Using AnnotationHub with job schedular causes an error

[kokitsuyuzaki]

When using AnnotationHub in parallel with job schedular (OpenGridEngine and Slurm), whether a job can be calculated was stochastical and some jobs were failed with the error attempt to write a readonly database.

I don't know why but since some jobs can be calculated,
by performing the job queue, again and again, I could finally get the output.

As an alternative approach, by not using the job schedular,
I could perform the calculation without any error,
but it takes an enormous amount of time, so I want to know the reason for the error.

I wrote an Snakefile as follows:

rule all:
	input:
		expand('{id}.RData', id=list(range(24)))

rule ah:
	output:
		'{id}.RData'
	log:
		'logs/{id}.log'
	shell:
		'Rscript ah.R {id} >& {log}'

wrote the R-script (ah.R) as follows:

outfile = commandArgs(trailingOnly=TRUE)[1]
library("AnnotationHub")
ah = AnnotationHub()
id = names(ah)[sample(length(ah), 1)]
out = ah[[id]]
save(out, file=outfile)

and finally performed the snakemake workflow as follows:

# OpenGridEngine
snakemake -j 24 --cluster "qsub -S /bin/bash"

or

# Slurm
snakemake -j 24 --cluster sbatch

The error message was as follows:

Loading required package: BiocFileCache
Loading required package: dbplyr
snapshotDate(): 2019-10-29
downloading 1 resources
retrieving 1 resource
Error: failed to load resource
  name: AH39322
  title: E031-H3K23me2.imputed.pval.signal.bigwig
  reason: 1 resources failed to download
In addition: Warning messages:
1: Couldn't set cache size: attempt to write a readonly database
Use `cache_size` = NULL to turn off this warning.
2: Couldn't set synchronous mode: attempt to write a readonly database
Use `synchronous` = NULL to turn off this warning.
3: download failed
  hub path: ‘https://annotationhub.bioconductor.org/fetch/44762’
  cache resource: ‘AH39322 : 44762’
  reason: attempt to write a readonly database
Execution halted

[RongrongZhao]

Hi, I met the same problem when using AnnotationHub in parallel. Have you solved this problem? Thanks.