Skip to content

Update vignette #779

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 26 commits into from
Jun 4, 2025
Merged
Show file tree
Hide file tree
Changes from 23 commits
Commits
Show all changes
26 commits
Select commit Hold shift + click to select a range
eb3e372
Update readme with new flow
ldecicco-USGS Jun 3, 2025
ce5a007
Merge branch 'add_docs' into 'develop'
ldecicco-USGS Jun 3, 2025
9c1db9b
Merge branch 'add_docs' into 'develop'
ldecicco-USGS Jun 3, 2025
8150512
make sure this works on gitlab
ldecicco-USGS Jun 3, 2025
eab31be
Remove bbox from daily
ldecicco-USGS Jun 3, 2025
a5394a7
Merge branch 'develop' of github.com:DOI-USGS/dataRetrieval into upda…
ldecicco-USGS Jun 3, 2025
5b011c8
Add API key info to readme
ldecicco-USGS Jun 3, 2025
7871762
add discrete stuff
ldecicco-USGS Jun 3, 2025
b16744d
Merge branch 'develop' of https://code.usgs.gov/water/dataRetrieval i…
ldecicco-USGS Jun 3, 2025
3bf91ce
put bbox back in daily
ldecicco-USGS Jun 3, 2025
6eb3ca0
Vignette updates
ldecicco-USGS Jun 4, 2025
70c3719
Merge branch 'develop' of github.com:DOI-USGS/dataRetrieval into upda…
ldecicco-USGS Jun 4, 2025
77b5c23
Add help section
ldecicco-USGS Jun 4, 2025
aaffa73
Update vignettes/read_USGS_functions.Rmd
ldecicco-USGS Jun 4, 2025
2b0ea51
Update vignettes/read_USGS_functions.Rmd
ldecicco-USGS Jun 4, 2025
69b5d04
Update vignettes/read_USGS_functions.Rmd
ldecicco-USGS Jun 4, 2025
095c5d5
Update vignettes/read_USGS_functions.Rmd
ldecicco-USGS Jun 4, 2025
da75aec
Update vignettes/read_USGS_functions.Rmd
ldecicco-USGS Jun 4, 2025
f478f2f
Update vignettes/read_USGS_functions.Rmd
ldecicco-USGS Jun 4, 2025
7bb6683
Update vignettes/read_USGS_functions.Rmd
ldecicco-USGS Jun 4, 2025
8943ec3
Update README.Rmd
ldecicco-USGS Jun 4, 2025
4c090c0
Update vignettes/read_USGS_functions.Rmd
ldecicco-USGS Jun 4, 2025
35c918e
Update vignettes/read_USGS_functions.Rmd
ldecicco-USGS Jun 4, 2025
69c7d1b
some updates from reviewers
ldecicco-USGS Jun 4, 2025
1c16310
pull from PR
ldecicco-USGS Jun 4, 2025
4972ee0
vignette updates
ldecicco-USGS Jun 4, 2025
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion R/read_USGS_daily.R
Original file line number Diff line number Diff line change
Expand Up @@ -67,7 +67,6 @@ read_USGS_daily <- function(monitoring_location_id = NA_character_,
parameter_code = NA_character_,
statistic_id = NA_character_,
properties = NA_character_,
bbox = NA,
time_series_id = NA_character_,
daily_id = NA_character_,
approval_status = NA_character_,
Expand All @@ -78,6 +77,7 @@ read_USGS_daily <- function(monitoring_location_id = NA_character_,
limit = 10000,
skipGeometry = NA,
time = NA_character_,
bbox = NA,
convertType = TRUE){

message("Function in development, use at your own risk.")
Expand Down
5 changes: 4 additions & 1 deletion R/read_USGS_data.R
Original file line number Diff line number Diff line change
Expand Up @@ -71,7 +71,10 @@ read_USGS_data <- function(service,

if(convertType) return_list <- cleanup_cols(return_list)

return_list <- return_list[order(return_list$time, return_list$monitoring_location_id), ]
# Add other time series services when they come online
if(service %in% c("daily")){
return_list <- return_list[order(return_list$time, return_list$monitoring_location_id), ]
}

return_list <- rejigger_cols(return_list, args[["properties"]], service)

Expand Down
28 changes: 21 additions & 7 deletions README.Rmd
Original file line number Diff line number Diff line change
Expand Up @@ -27,7 +27,7 @@ library(dataRetrieval)
[![](http://cranlogs.r-pkg.org/badges/dataRetrieval)](https://cran.r-project.org/package=dataRetrieval)
[![](http://cranlogs.r-pkg.org/badges/grand-total/dataRetrieval)](https://cran.r-project.org/package=dataRetrieval)

The `dataRetrieval` package was created to simplify the process of loading hydrologic data into the R environment. It is designed to retrieve the major data types of U.S. Geological Survey (USGS) hydrology data that are available on the Web, as well as data from the Water Quality Portal (WQP), which currently houses water quality data from the Environmental Protection Agency (EPA), U.S. Department of Agriculture (USDA), and USGS. Direct USGS data is obtained from a service called the National Water Information System (NWIS).
The `dataRetrieval` package was created to simplify the process of loading hydrologic data into the R environment. It is designed to retrieve the major data types of U.S. Geological Survey (USGS) hydrology data that are available on the Web, as well as data from the Water Quality Portal (WQP), which currently houses water quality data from the Environmental Protection Agency (EPA), U.S. Department of Agriculture (USDA), and USGS.

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

No opinion on whether this sentence is kept or removed, but the new APIs are still part of NWIS -- alongside the rest of the apis on api.waterdata.usgs.gov . We need to think through the branding better, frankly.


# Introduction

Expand All @@ -37,17 +37,19 @@ If you have additional questions about these changes, email CompTools@usgs.gov.

# What would you like to do?

1. Get instantaneous USGS discharge data. Start here: `?readNWISuv`
1. Get instantaneous USGS data (for example, discharge sensor data). Start here: `?readNWISuv`

2. Get daily USGS discharge data. Start here: `?readNWISdv`
2. Get daily USGS data (for example, mean daily discharge). Start here: `?read_USGS_daily`

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

❤️


3. Get USGS groundwater data. Start here: `?readNWISgwl`

4. Get discrete water quality data. Start here: `?readWQPdata`
4. Get discrete water quality data from a cooperative service that integrates publicly available water-quality data from the USGS, EPA, and over 400 state, federal, tribal, and local agencies. Start here: `?readWQPdata`

4. Discover USGS data (not including discrete water quality data). Start here: `?whatNWISdata`
5. Get USGS discrete water quality data. Start here: `?read_USGS_samples`

6. Find Hydro Network-Linked Data Index (NLDI) data. Start here: `?findNLDI`
6. Discover USGS time series metadata. Start here: `?read_USGS_ts_meta`

7. Find Hydro Network-Linked Data Index (NLDI) data. Start here: `?findNLDI`

For additional tutorials, see:

Expand All @@ -70,6 +72,18 @@ install.packages("dataRetrieval")

The USGS is planning to modernize all web services in the near future. For each of these updates, `dataRetrieval` will create a new function to access the new services.

## API Token

You can register an API key for use with USGS water data APIs. There are now limits on how many queries can be requested per IP address per hour. If you find yourself running into limits, you can request an API token here: <https://api.waterdata.usgs.gov/signup/>

Then save your token in your .Renviron file like this:

```
API_USGS_PAT = "my_super_secret_token"
```

You can use `usethis::edit_r_environ()` to edit find and open your .Renviron file. You will need to restart R for that variable to be recognized.

## New Features

### Style
Expand Down Expand Up @@ -160,7 +174,7 @@ Water Quality Portal. Washington (DC): National Water Quality Monitoring Council

# Package Support

The Water Mission Area of the USGS supports the development and maintenance of `dataRetrieval`, and most likely further into the future. Resources are available primarily for maintenance and responding to user questions. Priorities on the development of new features are determined by the `dataRetrieval` development team. This software was last released with USGS record: IP-147158.
The Water Mission Area of the USGS supports the development and maintenance of `dataRetrieval`, and most likely further into the future. Resources are available primarily for maintenance and responding to user questions. Priorities on the development of new features are determined by the `dataRetrieval` development team.

```{r disclaimer, child="DISCLAIMER.md", eval=TRUE}
```
Expand Down
50 changes: 35 additions & 15 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -11,8 +11,7 @@ retrieve the major data types of U.S. Geological Survey (USGS) hydrology
data that are available on the Web, as well as data from the Water
Quality Portal (WQP), which currently houses water quality data from the
Environmental Protection Agency (EPA), U.S. Department of Agriculture
(USDA), and USGS. Direct USGS data is obtained from a service called the
National Water Information System (NWIS).
(USDA), and USGS.

# Introduction

Expand All @@ -28,18 +27,25 @@ If you have additional questions about these changes, email

# What would you like to do?

1. Get instantaneous USGS discharge data. Start here: `?readNWISuv`
1. Get instantaneous USGS data (for example, discharge sensor data).
Start here: `?readNWISuv`

2. Get daily USGS discharge data. Start here: `?readNWISdv`
2. Get daily USGS data (for example, mean daily discharge). Start here:
`?read_USGS_daily`

3. Get USGS groundwater data. Start here: `?readNWISgwl`

4. Get discrete water quality data. Start here: `?readWQPdata`
4. Get discrete water quality data from a cooperative service that
integrates publicly available water-quality data from the USGS, EPA,
and over 400 state, federal, tribal, and local agencies. Start here:
`?readWQPdata`

5. Discover USGS data (not including discrete water quality data).
Start here: `?whatNWISdata`
5. Get USGS discrete water quality data. Start here:
`?read_USGS_samples`

6. Find Hydro Network-Linked Data Index (NLDI) data. Start here:
6. Discover USGS time series data. Start here: `?read_USGS_ts_meta`

7. Find Hydro Network-Linked Data Index (NLDI) data. Start here:
`?findNLDI`

For additional tutorials, see:
Expand Down Expand Up @@ -68,6 +74,21 @@ The USGS is planning to modernize all web services in the near future.
For each of these updates, `dataRetrieval` will create a new function to
access the new services.

## API Token

You can register an API key for use with USGS water data APIs. There are
now limits on how many queries can be requested per IP address per hour.
If you find yourself running into limits, you can request an API token
here: <https://api.waterdata.usgs.gov/signup/>

Then save your token in your .Renviron file like this:

API_USGS_PAT = "my_super_secret_token"

You can use `usethis::edit_r_environ()` to edit find and open your
.Renviron file. You will need to restart R for that variable to be
recognized.

## New Features

### Style
Expand Down Expand Up @@ -170,15 +191,15 @@ NWIScitation <- create_NWIS_bib(dv)
NWIScitation
#> U.S. Geological Survey (2025). _National Water Information System data
#> available on the World Wide Web (USGS Water Data for the Nation)_.
#> doi:10.5066/F7P55KJN <https://doi.org/10.5066/F7P55KJN>, Accessed Mar
#> 25, 2025,
#> doi:10.5066/F7P55KJN <https://doi.org/10.5066/F7P55KJN>, Accessed Jun
#> 03, 2025,
#> <https://waterservices.usgs.gov/nwis/dv/?site=09010500&format=waterml%2C1.1&ParameterCd=00060&StatCd=00003&startDT=1851-01-01>.
print(NWIScitation, style = "Bibtex")
#> @Manual{,
#> title = {National Water Information System data available on the World Wide Web (USGS Water Data for the Nation)},
#> author = {{U.S. Geological Survey}},
#> doi = {10.5066/F7P55KJN},
#> note = {Accessed Mar 25, 2025},
#> note = {Accessed Jun 03, 2025},
#> year = {2025},
#> url = {https://waterservices.usgs.gov/nwis/dv/?site=09010500&format=waterml%2C1.1&ParameterCd=00060&StatCd=00003&startDT=1851-01-01},
#> }
Expand All @@ -202,14 +223,14 @@ WQPcitation <- create_WQP_bib(SC)
WQPcitation
#> National Water Quality Monitoring Council (2025). _Water Quality
#> Portal_. doi:10.5066/P9QRKUVJ <https://doi.org/10.5066/P9QRKUVJ>,
#> Accessed Mar 25, 2025,
#> Accessed Jun 03, 2025,
#> <https://www.waterqualitydata.us/data/Result/search?siteid=USGS-05288705&count=no&pCode=00300&mimeType=csv>.
print(WQPcitation, style = "Bibtex")
#> @Manual{,
#> title = {Water Quality Portal},
#> author = {{National Water Quality Monitoring Council}},
#> doi = {10.5066/P9QRKUVJ},
#> note = {Accessed Mar 25, 2025},
#> note = {Accessed Jun 03, 2025},
#> year = {2025},
#> url = {https://www.waterqualitydata.us/data/Result/search?siteid=USGS-05288705&count=no&pCode=00300&mimeType=csv},
#> }
Expand All @@ -229,8 +250,7 @@ The Water Mission Area of the USGS supports the development and
maintenance of `dataRetrieval`, and most likely further into the future.
Resources are available primarily for maintenance and responding to user
questions. Priorities on the development of new features are determined
by the `dataRetrieval` development team. This software was last released
with USGS record: IP-147158.
by the `dataRetrieval` development team.

# Disclaimer

Expand Down
2 changes: 2 additions & 0 deletions _pkgdown.yml
Original file line number Diff line number Diff line change
Expand Up @@ -40,6 +40,8 @@ navbar:
menu:
- text: Tutorial
href: articles/tutorial.html
- text: New USGS Functions
href: articles/read_USGS_functions.html
- text: Background
href: articles/dataRetrieval.html
- text: Pivot Data
Expand Down
16 changes: 8 additions & 8 deletions man/read_USGS_daily.Rd

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

20 changes: 10 additions & 10 deletions tests/testthat/tests_userFriendly_fxns.R
Original file line number Diff line number Diff line change
Expand Up @@ -160,16 +160,16 @@ test_that("WQP qw tests", {
nameToUse <- "Specific conductance"
pcodeToUse <- "00095"

# INFO_WQP <- readWQPqw("USGS-04024315", pcodeToUse,
# startDate = "", endDate = "", legacy = FALSE)
# expect_is(INFO_WQP$Activity_StartDateTime, "POSIXct")
#
# INFO2 <- readWQPqw("WIDNR_WQX-10032762", nameToUse,
# startDate = "", endDate = "", legacy = FALSE)
# expect_is(INFO2$Activity_StartDateTime, "POSIXct")
#
# df <- readWQPqw("USGS-04193500", parameterCd = "00665", legacy = FALSE)
# expect_true(nrow(df) > 0)
INFO_WQP <- readWQPqw("USGS-04024315", pcodeToUse,
startDate = "", endDate = "", legacy = FALSE)
expect_is(INFO_WQP$Activity_StartDateTime, "POSIXct")

INFO2 <- readWQPqw("WIDNR_WQX-10032762", nameToUse,
startDate = "", endDate = "", legacy = FALSE)
expect_is(INFO2$Activity_StartDateTime, "POSIXct")

df <- readWQPqw("USGS-04193500", parameterCd = "00665", legacy = FALSE)
expect_true(nrow(df) > 0)

df2 <- readWQPqw("USGS-05427718", parameterCd = "all")
expect_true(nrow(df2) > 0)
Expand Down
Loading
Loading