Training data

[boxydog]

What training data is actually used to train the model?

When I look in training.sqlite3, it looks like just nytimes?

$ sqlite3 train/data/training.sqlite3 
SQLite version 3.40.0 2022-11-16 12:10:08
Enter ".help" for usage hints.
sqlite> select source, count(1) from en;
nyt|60000

I'm looking to count popular and rare ingredients.

[strangetom]

Currently using three sources: NYT, BBC and Cookstr

$ sqlite3 train/data/training.sqlite3 
SQLite version 3.37.2 2022-01-06 13:25:41
Enter ".help" for usage hints.
sqlite> SELECT source, count(*) FROM en GROUP BY source;
bbc|15000
cookstr|15000
nyt|30000

The specific rows used training and evaluation are randomised each time the model is trained, but the randomisation is stratified so it's the same proportional split per source.

[boxydog]

Thanks. I'm a dummy, I didn't include the group-by.