Reddit Travel Scraper

A Python-based Reddit scraper that monitors travel-related subreddits and sends alerts to Slack for posts from the last 24 hours.

Setup

1. Install dependencies:

   pip install -r requirements.txt

2. Configure environment variables:

   cp .env.example .env

   Then edit .env with your actual credentials.

3. Get Reddit API credentials:

   • Go to https://www.reddit.com/prefs/apps
   • Click "Create App"
   • Choose "script" type
   • Copy client ID and secret

4. Get Slack webhook URL:

   • Go to https://api.slack.com/messaging/webhooks
   • Create a webhook for your workspace
   • Copy the webhook URL

Usage

Run the scraper:

python scripts/run_scraper.py

Configuration

Edit your .env file to customize:

• SUBREDDITS_TO_MONITOR: Comma-separated list of travel subreddits (default: backpacking,solotravel,travel,TravelHacks,digitalnomad,etc.)
• POST_LIMIT_PER_SUBREDDIT: Number of posts to fetch per subreddit (default: 5)
• REQUEST_DELAY: Delay between API requests in seconds (default: 2.0)

Features

• 24-hour filter: Only scrapes posts from the last 24 hours
• Travel-focused filtering: Prioritizes posts with travel-related keywords
• Smart content detection: Identifies questions and relevant travel content
• Slack integration: Sends formatted alerts to your Slack channel