Create datamodules from custom data

[LorenzoF6]

I want to try Anomalib with my custom data.
It is correct to orgnazie my data in a MVTEC like structure like the image below, beacuase for my category of object i have 4 indipendent type of defect

CustomData/
├── category_1/
│ ├── train/
│ │ ├── good/
│ │ │ ├── image1.png
│ │ │ ├── image2.png
│ │ │ └── ...
│ ├── test/
│ │ ├── anomaly_type_1/
│ │ │ ├── image1.png
│ │ │ ├── image2.png
│ │ │ └── ...
│ │ ├── anomaly_type_2/
│ │ │ ├── image1.png
│ │ │ ├── image2.png
│ │ │ └── ...
│ ├── ground_truth/
│ │ ├── anomaly_type_1/
│ │ │ ├── image1_mask.png
│ │ │ ├── image2_mask.png
│ │ │ └── ...
│ │ ├── anomaly_type_2/
│ │ │ ├── image1_mask.png
│ │ │ ├── image2_mask.png
│ │ │ └── ...
└── ...

And the use the class Folder to create the custom datamodules and use the option to automaticaly split the normal data for train and test? Or it is convinient to put some normal images in a dedicate normal directory under test?
Thanks very much

[abc-125]

Hello, Folder dataset uses just one type of defect, so you can either move all defects to one folder or create a custom class.

In both cases, you can use automatic split or put some normal images in a separate folder.

[LorenzoF6]

thansk, but if i want to separate my defects ( i have fout type) how the model can know the difference from the different effects?
and if i want to do classification, i think that's correct to put all all defects in one folder but if i want to do segmentation it isn't correct to maintain the different defects separate?

[LorenzoF6]

infact, if you see in the 2022 documentation (v1 realease) in the folder class page, it is shown that every type of defetc have it own folder

[LorenzoF6]

So now, with the new realese, if i want to train a model with a custom dataset, the dataset should have a directorty tree like this:
DatasetName:

• Normal
• Abnormal
  Or it is possible to maintain a mvteclike structure? Because in the documentation it is not well explained

Thansk

[abc-125]

how the model can know the difference from the different effects

It won't. All the models available in anomalib are trained with normal images only. Anomalous images are used only for testing.

if i want to do segmentation it isn't correct to maintain the different defects separate

All defects in one folder for either classification or segmentation should be fine.

Or it is possible to maintain a mvteclike structure?

You can make your version of the MVTecAD class. I think you need to replace categories with category_1, also make sure you added your class to all __init__.py. After this, you can initialize it like this:

dataset = MyDataset(
        ...     root=Path("./CustomData"),
        ...     category="category_1"
        ... )

[LorenzoF6]

ok thanks.
So if i have one "category" is better using the folder class
instead, if i have multiple "categories" of defect is better to customize the mvtec class.

But, this it isn't the same thing as using folder class and change the source of the abnormal_dir every time i want to perform the analysis on the different defects? In this, i using the folder class and every time i change the destination of the abnormal_dir, mainting at the same time, the mvtec like structure

[abc-125]

change the source of the abnormal_dir every time i want to perform the analysis on the different defects?

I think it should work in your case (if you want to calculate results per type of defect).

MVTecAD class calculates results for all defects together as if they are in the same folder; it just allows you to have a different folder structure.

[github-actions]

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.

[samet-akcay]

Thanks to a community contributor, @manuelkonrad, we have merged a new Tabular feature in #2713, which could be quite useful for your use-case. Here is the link to the documentation
https://anomalib.readthedocs.io/en/latest/markdown/guides/reference/data/datamodules/image/tabular.html