MHTMLExtractor

MHTMLExtractor is a high-performance, standalone Python utility to extract files from MHTML (MIME HTML) documents. These documents are typically a snapshot of a web page and might contain images, scripts, styles, and the web page itself as a single file.

Features

• High Performance: Optimized memory usage and processing speed with automatic buffer sizing
• Dry-run Mode: Analyze MHTML files without extracting to preview contents
• Comprehensive Statistics: Detailed extraction statistics and timing information
• Type Safety: Full type hints for better code quality and IDE support
• Flexible Filtering: Selectively skip extraction of certain file types (CSS, images, etc.)
• Smart Filename Handling: Intelligent filename generation with conflict resolution
• Efficient Processing: Optimized string operations and memory management
• Progress Reporting: Detailed logging with configurable verbosity levels

Performance Improvements

• Adaptive Buffer Sizing: Automatically optimizes buffer size based on file size
• Linear String Operations: Uses list-based concatenation for O(n) performance instead of O(n²)
• Efficient Link Updates: Optimized HTML link replacement using regex substitution
• Memory Optimization: Processes files in chunks to handle large MHTML files efficiently

Prerequisites

• Python 3.7+ (with type hint support)

Usage

To use the MHTML Extractor, simply run the script and provide the necessary arguments:

usage: MHTMLExtractor.py [-h] [--output_dir OUTPUT_DIR] [--buffer_size BUFFER_SIZE] 
                         [--clear_output_dir] [--no-css] [--no-images] [--html-only] 
                         [--dry-run] [--verbose] [--quiet]
                         mhtml_path

positional arguments:
  mhtml_path              Path to the MHTML document.

optional arguments:
  -h, --help              show this help message and exit
  --output_dir OUTPUT_DIR
                          Output directory for the extracted files. (default: current directory)
  --buffer_size BUFFER_SIZE
                          Buffer size for reading the MHTML file. (default: 8192)
  --clear_output_dir      If set, clears the output directory before extraction.
  --no-css                If set, CSS files will not be extracted.
  --no-images             If set, image files will not be extracted.
  --html-only             If set, only HTML files will be extracted.
  --dry-run               If set, analyze the MHTML file without extracting files.
  --verbose, -v           Enable verbose logging output.
  --quiet, -q             Suppress all output except errors.

Examples

1. Extract all files from an MHTML document:

python MHTMLExtractor.py example.mhtml

2. Extract files to a specific directory:

python MHTMLExtractor.py example.mhtml --output_dir ./extracted

3. Extract only HTML files:

python MHTMLExtractor.py example.mhtml --html-only

4. Dry-run analysis (preview without extracting):

python MHTMLExtractor.py example.mhtml --dry-run --verbose

5. Extract without CSS and images:

python MHTMLExtractor.py example.mhtml --no-css --no-images

6. High-performance extraction with custom buffer:

python MHTMLExtractor.py large_file.mhtml --buffer_size 65536 --verbose

New Features

Dry-Run Mode

Use --dry-run to analyze MHTML files without extracting them. This shows you:

• What files would be extracted
• File types and sizes
• Extraction statistics
• Performance metrics

Enhanced Statistics

The tool now provides comprehensive statistics including:

• Number of files by type (HTML, CSS, images, other)
• Total data size processed
• Extraction time
• Files skipped due to filters

Improved Performance

• Auto-optimization: Buffer size automatically optimized based on file size
• Memory efficiency: Reduced memory usage for large files
• Faster processing: Optimized string operations and regex patterns

Better Error Handling

• Detailed error messages with specific causes
• Graceful handling of corrupted MHTML files
• Input validation and permission checks
• Proper exit codes for automation

Code Quality Improvements

• Type hints: Full type annotations for better IDE support and code safety
• Documentation: Comprehensive docstrings for all methods
• Constants: Extracted magic numbers to named constants
• Validation: Input validation for all parameters
• Logging: Configurable logging levels (quiet, normal, verbose)

Technical Details

Performance Optimizations

• Adaptive Buffering: Buffer size automatically adjusted based on file size (1KB - 1MB range)
• Linear String Operations: Uses list-join method instead of string concatenation for O(n) performance
• Efficient Regex: Optimized regular expressions for content parsing and link updates
• Smart Processing: Only processes complete MHTML parts to avoid partial data issues

Enhanced Error Handling

• Input Validation: Validates file existence, permissions, and parameter ranges
• Graceful Degradation: Continues processing even if individual parts fail
• Specific Exceptions: Different exception types for different error scenarios
• Detailed Logging: Comprehensive error messages with context

Code Quality Features

• Type Safety: Complete type hints using typing module
• Immutable Data: Uses @dataclass for structured data with proper types
• Path Handling: Uses pathlib.Path for robust cross-platform path operations
• Constants: All magic numbers and strings extracted to named constants

Notes

• Purpose: This script is designed to extract files (like images, CSS, and HTML content) from MHTML documents. MHTML is a web page archive format that's used to combine multiple resources from a web page into a single file.

• Performance: The script efficiently reads the MHTML file in adaptive chunks (auto-optimized from 1KB to 1MB) to handle even very large files without consuming excessive memory.

• Dry-Run Analysis: Use --dry-run to preview what would be extracted without actually writing files. Perfect for analyzing unknown MHTML files.

• Statistics: Comprehensive extraction statistics including file counts by type, total size, and processing time.

• Error Handling: Robust error handling with specific error types and detailed messages for troubleshooting.

• Handling Conflicts: If potential filename conflicts arise (two extracted resources having the same name), the script handles it by appending a counter to the filename.

• File Naming: The filenames for the extracted files are based on the Content-Location from the MHTML headers with sanitization for filesystem safety. If unavailable, UUID-based filenames are generated. A hash derived from the original URL is appended to ensure uniqueness.

• Link Updates: Once extraction is complete, the script updates the links within the extracted HTML files to ensure they point to the new filenames of the extracted resources (unless using --html-only).

• Filtering Options: The script provides command-line flags to optionally exclude CSS files, image files, or to extract only HTML files.

• Dependencies: The script uses only Python's built-in libraries, so no additional installation is required. Requires Python 3.7+ for type hint support.

• Cross-Platform: Works on Windows, macOS, and Linux with proper path handling.

Performance Benchmarks

Typical performance improvements over the original version:

• Memory Usage: 60-80% reduction for large files
• Processing Speed: 2-3x faster for files > 10MB
• String Operations: 10x faster link replacement for large HTML files

License

This script is provided as-is under the MIT License. Use it at your own risk.