Skip to content

This project is a robust, multi-threaded tool for generating and analyzing all possible Turkish Airlines flight routes that visit all 6 continents, specifically designed for the Turkish Airlines 1 Million Miles Challenge

Notifications You must be signed in to change notification settings

Amsal1/google-flights-scrapper

Repository files navigation

Turkish Airlines 1M Miles Challenge Route & Flight Search Tool

Overview

Th3. Run the Tool

python3 comprehensive_flight_search.py
  • The tool will generate all valid routes and begin searching for Turkish Airlines flights for each segment.
  • Progress and results are saved continuously. You can safely interrupt and resume at any time.
  1. Review Resultsct is a robust, multi-threaded tool for generating and analyzing all possible Turkish Airlines flight routes that visit all 6 continents, specifically designed for the Turkish Airlines 1 Million Miles Challenge. It was developed by extensively modifying and extending the google-flights-scraper library to support advanced route generation, flight search, and robust, resumable execution.

Features

1. Comprehensive Route Generation

  • All 6 Continents: Generates all possible multi-continent routes, ensuring every route visits all 6 continents (Asia, Europe, Africa, North America, South America, Oceania).
  • Easy Visa Countries: Only includes countries that are easy for Indian citizens to visit (visa-free, e-visa, or visa-on-arrival), maximizing practical feasibility.
  • Major Cities: Uses major cities per country for realistic flight segment planning.

2. Multi-Threaded, High-Performance Flight Search

  • Parallel Processing: Uses Python's ThreadPoolExecutor to search for flights on multiple routes in parallel, dramatically speeding up the search process.
  • Per-Thread Scrapers: Each thread uses its own Google Flights scraper instance to avoid conflicts and maximize throughput.
  • Rate Limiting: Configurable delay per thread to avoid being rate-limited by Google Flights.

3. Turkish Airlines-Only Filtering & Early Termination

  • Strict Airline Filter: Only considers flights operated by Turkish Airlines (no codeshares), and only those routing via Istanbul (IST).
  • Early Termination: As soon as any segment in a route cannot be flown with Turkish Airlines, the entire route is discarded immediately, saving time and resources.

4. Resumable & Robust Execution

  • Continuous Progress Saving: Progress is saved after every route to flight_search_progress.json and results to flight_search_results.json using atomic file writes.
  • Crash/Interruption Recovery: On restart, the tool loads previous progress and resumes from where it left off, skipping already-completed routes.
  • Thread-Safe File I/O: All file operations are protected by locks to ensure data integrity in multi-threaded execution.

5. Output & Reporting

  • JSON Results: All complete Turkish Airlines routes (with all segments valid) are saved in a structured JSON file, including route details, flight segments, and total cost.
  • Human-Readable Summaries: The tool prints summaries of the best routes, progress statistics, and efficiency metrics to the console.
  • Efficiency Stats: Reports on early termination savings, processing speed, and completion rates.

6. Flexible Date Configuration

  • Centralized Date Management: Configure departure date in one place in the main() function for all flights.
  • Multiple Date Formats: Supports various input formats (e.g., "2 Oct 2025", "2025-10-02", "October 2, 2025").
  • Automatic Format Conversion: Automatically converts to proper formats for search logic and Google Flights interface.
  • No Leading Zeros: Properly formats single-digit dates without leading zeros (e.g., "October 2, 2025" not "October 02, 2025").

7. Extensive Modifications & Improvements

  • Full Combinatorial Route Coverage: Implements generate_all_possible_combinations for exhaustive route generation.
  • Robust Error Handling: Handles all exceptions gracefully, marking failed routes as processed to avoid infinite retries.
  • Atomic File Writes: Ensures no data loss even if the process is killed or crashes.
  • Highly Configurable: Thread count, rate limits, and batch sizes are easily adjustable for different hardware or dataset sizes.

Usage

Setting Your Departure Date

Before running the tool, you can easily configure your preferred departure date by editing one line in the main() function:

# In comprehensive_flight_search.py, main() function:
departure_date = "2 Oct 2025"  # Change this to your preferred date

Supported Date Formats:

  • "2 Oct 2025" (recommended)
  • "2025-10-02"
  • "October 2, 2025"
  • "02/10/2025"
  • "25 Dec 2025"
  • And more standard formats...

The system automatically converts your input to the correct formats:

  • Search logic uses: "Thu, Oct 2" (short format)
  • Google Flights uses: "October 2, 2025" (full format)

Running the Tool

  1. Install Dependencies

    • Clone this repo and install requirements (see requirements.txt).
    • Ensure you have Python 3.9+.
  2. Configure Departure Date (Optional)

    • Edit the departure_date variable in the main() function of comprehensive_flight_search.py.
    • Use any standard date format (examples above).
  3. Run the Tool

    python3 comprehensive_flight_search.py
    • The tool will generate all valid routes and begin searching for Turkish Airlines flights for each segment.
    • Progress and results are saved continuously. You can safely interrupt and resume at any time.
  4. Review Results

    • Results are saved in flight_search_results.json (all complete Turkish Airlines routes).
    • Progress is tracked in flight_search_progress.json.
    • Summaries and statistics are printed to the console after each run.

File Structure

  • comprehensive_flight_search.py — Main script for route generation, multi-threaded search, and reporting.
  • route_planner.py — Route generation logic (all possible 6-continent combinations).
  • utils.py — Country, city, and visa data utilities.
  • google_flights/google_flights.py — Modified Google Flights scraper (airline filtering, robust error handling).
  • flight_search_progress.json — Progress tracking (auto-generated).
  • flight_search_results.json — Results (auto-generated).

Key Modifications to google-flights-scraper

  • Thread Safety: Refactored to allow multiple concurrent scraper instances.
  • Airline Filtering: Added strict Turkish Airlines-only and IST routing filters.
  • Date Configuration: Added centralized date management with automatic format conversion for both search logic and Google Flights interface.
  • Robust Error Handling: Improved handling of missing data, timeouts, and Google Flights quirks.
  • Atomic File Writes: Ensured all progress/results are saved safely in multi-threaded environments.

Example Output

  • Top 5 Cheapest Complete Turkish Airlines Routes:
    • Printed to console after each run, showing total cost, route, and flight segment details.
  • JSON Results:
    • Each result includes route, all flight segments, total cost, and status.

How to Extend or Modify

  • Change Departure Date: Simply edit the departure_date variable in the main() function.
  • Change Visa Rules: Edit utils.py to update which countries are considered "easy visa".
  • Adjust Threading/Performance: Change max_workers and rate_limit_delay in comprehensive_flight_search.py.
  • Add More Cities: Update COUNTRY_MAJOR_CITIES in utils.py.

Credits


License

This project is for educational and research purposes. Please respect the terms of service of Google Flights and Turkish Airlines. Not affiliated with Turkish Airlines or Google.

About

This project is a robust, multi-threaded tool for generating and analyzing all possible Turkish Airlines flight routes that visit all 6 continents, specifically designed for the Turkish Airlines 1 Million Miles Challenge

Topics

Resources

Stars

Watchers

Forks

Releases

No releases published

Packages

No packages published

Languages