Here's a README.md for your project:

Coupon Data Scraper

This project is a web scraper that extracts coupon data from the AliExpress coupon page on Coupons. It collects coupon IDs from the page, fetches detailed information for each coupon via an API, and saves the valid coupons to a JSON file. https://lilacchio.github.io/coupon-data/coupons.json

Features

• Scrapes coupon data from ANY WEBSITE (Demo: Aliexpress) on coupons.com.
• Fetches detailed information for each coupon using an API endpoint.
• Handles retries for failed API requests.
• Processes multiple coupon IDs concurrently with configurable concurrency.
• Saves the valid coupons to a JSON file.

Setup

1. Clone the repository:

2. Install the required dependencies:

3. Configure the CONFIG object inside the index.js file if needed:

   • TARGET_URL: The URL to scrape coupon data from.
   • API_BASE: The base URL for the API to fetch coupon details.
   • OUTPUT_FILE: The name of the file to save the valid coupons.
   • CONCURRENCY: Number of concurrent requests to make when fetching coupon details.
   • RETRIES: The number of retry attempts for failed API requests.
   • REQUEST_TIMEOUT: The timeout for each API request.

4. Run the scraper:

   node index.js

   This will start scraping, fetching the coupon details, and saving the results in a file named coupons.json.

Output

• The scraper will output a JSON file named coupons.json with the following structure:

[
  {
    "title": "Coupon Title",
    "type": "Percentage Off",
    "isVerified": true,
    "code": "COUPONCODE"
  },
  ...
]

• Each object represents a valid coupon, with:
  • title: The title of the coupon.
  • type: The type of discount (e.g., "Percentage Off", "Free Shipping").
  • isVerified: Whether the coupon is verified or not.
  • code: The coupon code (if available).

Contributing

If you have suggestions or improvements, feel free to open an issue or submit a pull request.

License

This project is open-source and available under the MIT License.

Feel free to adjust the details (like the URL, API base, and other configurations) as needed for your project!