You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
{{ message }}
This repository was archived by the owner on Nov 1, 2021. It is now read-only.
localDataset=require'dataset.Dataset'localopt=lapp[[Got stuck in torch-dataset with batchSize == 128(options) --batchSize (default 128) how many images in a mini-batch?]]-- create tmp csv file containing lots of rowslocaltmpcsv=paths.tmpname() ..'.csv'f=io.open(tmpcsv, 'w')
f:write('filename\n')
fori=1,300dof:write(paths.tmpname() ..'\n')
endf:close()
dataset=Dataset(tmpcsv)
getBatch, numBatches, reset=dataset.sampledBatcher({
batchSize=opt.batchSize,
inputDims= {10, 256},
verbose=true,
poolSize=4,
get=function(x)
returntorch.FloatTensor(10,256)
end,
processor=function(res, processorOpt, input)
returntrueend,
})
print('before getBatch')
localbatch=getBatch()
print('finish getBatch')
Strangely the program works when batchSize is 64, but got stuck in getBatch() when batchSize is 128.
I came across this problem for several different problems which use custom get function and load data with other non-default method like image.load, where batchSize 64 works but not 128.