Out of Memory Exception during full dataset download

[budhasree-m]

I understand the data is huge with about 10,211,478 rows. I choose to wait for the full dataset to download. But, it failed with "Out of Memory" exception after progressing to 80%. How much memory is significant memory? I have tried it twice after clearing cache.

Update: I managed to download as two separate files using Gender filter.

[thedivtagguy]

Hi @budhasree-m, thanks for reporting this! Yes this was/is a tricky thing to implement in browser. Could you let me know a little more details about what your system specs are and which browser this is? It'll help me understand this better.

[budhasree-m]

Hi @thedivtagguy Apologies for the delay in response. below are the details.

Internet Bandwidth ~ 100 to 140Mbps
Disk Space Available: 30GB
Browser: Chrome Version: 114.0.5735.91

Also, a suggestion if it's feasible to add a check before download to assess user machine/browser capability with their consent.

Also, I noticed if text identifier ("") is avoided & file is downloadable as .DAT extension with pipe delimiter(|) instead of comma(,), it can reduce the file size from 1.96GB to 1.74 GB. I understand text identifier ("") is significant for different file formats. In that case, if 2 to 3 format options are available to download, that would be great.

Otherwise, I really admire the work done in relation to design. Thanks to you & @Vonter for all the inspiration.
Happy learning.

[budhasree-m]

Thank you for providing a compressed download option. The issue can be closed.