Creating your own datasets for real world problems

[ChristopherMSmith71]

Love the class!

In the class, all of the datasets were already created. However, out in the ugly real world, you don't get datasets created for you.

How do you create datasets (PDF document, Classification) or even (Text document: Classification). You can't use a simple .csv file because there are commas all over the text document. Each .pdf or .txt, is in its own file.

I did find that there is tf.keras.preprocessing.image_dataset_from_directory where they will take images out of the file and give them the label of whatever the directory name is with the labels="inferred" argument. That is great for images, but how do you do it for .pdf or .txt files? How do you get the document as the feature, and the classification as the label?

[ChristopherMSmith71]

There is a tf.keras.utils.text_dataset_from_directory as well. However, in general, how do you create your own dataset?