Allowed Memory Size Exhausted

[fhandke-fugro]

Hello,

While trying to create a volume via API we are getting:

[2025-02-04 14:03:32] production.ERROR: Allowed memory size of 134217728 bytes exhausted (tried to allocate 5242880 bytes) {"userId":2,"exception":"[object] (Symfony\\Component\\ErrorHandler\\Error\\FatalError(code: 0): Allowed memory size of 134217728 bytes exhausted (tried to allocate 5242880 bytes) at /var/www/vendor/league/flysystem/src/DirectoryListing.php:67) [stacktrace] #0 {main}

In the admin logs and it fails to create the volume. The volume uses an s3 as storage disk and the images are in a folder with a lot of images (100000). Do you have an explanation for that and/or a workaround?

Thanks a lot!

[mzur]

Can you please share the full stack trace? The problem is that the PHP application consumes too much memory because it wants to store the full directory listing of the 100k files. Usually, something like this can be optimized with lazy loading and/or looping over the data but I need the full stack trace to see where exactly this happens.

As a workaround, you can first create the volume with only a couple of files (e.g. 10k). Then, when the volume exists, you can add the remaining images in (10k) chunks.

[fhandke-fugro]

Hello Martin,

Thanks for the quick response. Can you give me a hint on how to access the full stack trace?
Also note that we did not try to add all 100000 images to the volume, we just tried to add one image inside the folder.

[mzur]

I see. This definitely sounds like it has potential for optimization. If there is no stack trace below the line you already posted then there probably is no further information. I assume you got this from the laravel.log file or the logs in the admin area?

[fhandke-fugro]

I got this from the logs in the admin area and yes, there is no other info there pointing to the error in more detail.

[mzur]

Thanks. While this makes it hard to pin down the exact source of the error, I had a suspicion. I've released a code change in v3.94.3 that could fix the issue. Please update your instance (once the updated Docker images were built) and try again. Also mind the new migrations required by v3.94.2.

[fhandke-fugro]

Hello Martin. Apparently it wasn't too dificult to pin down. Just tested it and it worked now :). Thanks a lot!