Is there anyway to download a small portion of mC4?

[sbmaruf]

Hi! Not sure if it is the right place to ask this question. Since the mT5 repo doesn't have any Discussion bar, I'm asking it here.

Is there any way we can download a small portion of mC4? More than 20T seems really large space. It would be nice if we can download the dataset partially (language-wise or even language-wise random samples).

[craffel]

Not really - unless I'm mistaken the only way to do this would be to manually modify the code so that it only processed one crawl, but that would still be very compute intensive (@adarob can confirm). FWIW there is a rumor that AI2 will release mC4 like they did for C4. Maybe @dodgejesse can comment.

[dodgejesse]

Yes, we're working on releasing mC4 in the next few weeks. We plan to allow downloading each of the languages separately, and we're considering releasing a downsampled version that'll be smaller (for convenience, e.g. when prototyping models).

[sbmaruf]

Thanks a lot for the reply. Eagerly waiting for the dataset.