Memory Issue

[SpaceLearner]

Hi Karl, thanks for providing this wonderful solution. I find when processing a large dataset like droid, it will exceeds the memory. I wonder if there any way to make more efficient memory use. Thanks !

[varunvijamz]

Yeah, there seems to be a memory leak somewhere. I'm not sure what the root cause is but I was able to fix it by restarting the workers:
pool = Pool(processes=self._n_workers, maxtasksperchild=10)