Skip to content

Unable to download time-series data #16

@nikosGeography

Description

@nikosGeography

Hi, for the past week or so it seems that there is an issue when using the bm_raster() for downloading time-series data (i.e., more than one image). I am getting this error: Error in httr2::req_perform(): ! HTTP 401 Unauthorized. Sometimes I get the error at later stage of downloading (> 50% of the images) and sometimes at an early stage (after 2-3 images being downloaded). Below is the code:

pacman::p_load(blackmarbler, sf, terra, lubridate, RPushbullet)

wd <- "path/"

#### Define NASA bearer token
bearer <- "bearer"

### ROI
shapefile_list <- list.files(wd, pattern = "\\.shp$", full.names = TRUE)

# Read all shapefiles into a list
roi_sf1 <- st_read(shapefile_list)
plot(roi_sf1)
provoliko <- st_crs(roi_sf1)$Wkt

roi_sf  <- st_transform(roi_sf1, crs = 4326)

#### time-series data
r_monthly <- bm_raster(roi_sf = roi_sf,
                       product_id = "VNP46A3",
                       date = seq.Date(from = ymd("2018-01-01"), 
                                       to = ymd("2023-12-31"), 
                                       by = "month"),
                       variable = "AllAngle_Composite_Snow_Free",
                       quality_flag_rm = c(1, 2),
                       bearer = bearer)

pbPost("note", "download completed", "download completed")

r_monthly_terra_rpj <- r_monthly |> project(provoliko, 
                                               method = "average",
                                               res = 430)

r <- ifel(r_monthly_terra_rpj < 0.5, 0, r_monthly_terra_rpj)

writeRaster(r, 
            paste0(wd, names(r_monthly_terra_rpj),".tif"))

When I use the below code (i.e., to download a single image):

# single image
### Monthly data: raster for April 2018
r_201804 <- bm_raster(roi_sf = roi_sf,
                      product_id = "VNP46A3",
                      date = "2018-09", # The day is ignored
                      variable = "AllAngle_Composite_Snow_Free", # NearNadir, OffNadir, AllAngle
                      quality_flag_rm = c(1, 2),
                      bearer = bearer)

pbPost("note", "download completed", "download completed")

plot(r_201804)

r_201804_rast_rpj <- r_201804 %>% project(provoliko, 
                                               method = "average")
res(r_201804_rast_rpj)
crs(r_201804_rast_rpj)

writeRaster(r_201804_rast_rpj, paste0(wd, "ntl.tif"), overwrite = TRUE)

I get no error, just a warning:
Warning messages: 1: Failed to open https://ladsweb.modaps.eosdis.nasa.gov/archive/allData/5000/VNP46A3/2018/061.csv: The requested URL returned error: 404

but the imagery is properly downloaded.

I tried to update the bearer key, I uninstalled and installed the package, I tried different areas with simpler polygons (just a rectangle) but I can't download time-series data. Is this the package's problem or NASA's?

Thank you.

> sessionInfo()
R version 4.4.2 (2024-10-31 ucrt)
Platform: x86_64-w64-mingw32/x64
Running under: Windows 10 x64 (build 19045)

Matrix products: default

attached base packages:
[1] stats     graphics  grDevices utils     datasets  methods   base     

other attached packages:
[1] RPushbullet_0.3.4  lubridate_1.9.4    terra_1.8-21       sf_1.0-19         
[5] blackmarbler_0.2.4

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions