Store small 'view' files in SQLite containers

[lfcnassif]

Continuation of #174. We can target:

1. HTML reports produced by Parsers via MakePreviewTask
2. HEIC and other image types configured to generate a faster rendering preview
3. Video mosaics produced by VideoThumbTask
4. Other things stored in iped/view folder, not sure if we have more..

In ExportFileTask, we use a threshold of 16MB to store small items into SQLite, because the Xerial-sqlite library copies the whole BLOB to memory when reading it, even if using the InputStream API, unfortunately... For larger files, we kept them in case folder. It also helps the HexViewer to work faster, otherwise it would have to wait large files to be copied from SQLite to a temp folder before rendering the content, since it needs the whole data to be available because of its SEEK feature.

[lfcnassif]

Instead of storing large files in case folder, another much more complex option would be to break large files into several small BLOBs, so we could implement the SeekableInputStream interface to read just the needed blocks/blobs and to use a limited amount of memory when reading the whole file. It's challenging because of atomicity and coherence issues, and maybe might lead to a FS implementation using SQLite. For the sake of simplicity and to avoid having to maintain a complex code, we suspended this idea in the past.

[aberenguel]

As additional information, processing a 33GB UFDR from an iPhone extraction with PListParser and NSKeyedArchiverParser caused the file count in the case view folder to increase from 240 to 134114.

[lfcnassif]

As additional information, processing a 33GB UFDR from an iPhone extraction with PListParser and NSKeyedArchiverParser caused the file count in the case view folder to increase from 240 to 134114.

Could you check how many are HTML (parsers) and JPG (heic/videos) in iped/view folder?

[aberenguel]

Could you check how many are HTML (parsers) and JPG (heic/videos) in iped/view folder?

• 134114 HTML files (133874 from PListParsers and 240 from other Parsers/Tasks)
• 2334 JPG files
• 1 TMP file (empty file - iped/view/1/F/1FA5801F1B6D665B442EE8C7612A5145.tmp generated from file transparent.heic)

[lfcnassif]

To avoid bloating the case folder, I propose modifying MakePreviewTask. We could add a condition to check the size of the generated file. If it's below a certain threshold (e.g., 500KB), we could store its content directly using ExportFileTask.getLastInstance().insertIntoStorage instead of saving it as a separate file.

Be careful, this method would set or replace item data. That's not the case, what we want is to store the item preview. I think we will need a new column or table in the SQLite schema, and adjust code reading from iped/view accordingly.