RSSHub

A high-performance CLI RSS feed aggregator built with Go, featuring concurrent feed processing via configurable worker pools, PostgreSQL storage, and runtime configuration updates.

Features

• Concurrent Feed Processing - Efficient worker pool architecture for parallel RSS feed fetching
• Dynamic Configuration - Change fetch intervals and worker count without restarting
• PostgreSQL Storage - Reliable persistence with automatic migrations
• Clean Architecture - Well-structured codebase following domain-driven design principles
• Docker Support - Easy deployment with Docker Compose
• Graceful Shutdown - Proper cleanup and resource management

Requirements

• Go 1.21+
• PostgreSQL 15+
• Docker & Docker Compose (optional)

Installation

Using Docker Compose (Recommended)

git clone <repository-url>
cd rsshub
cp .env.example .env
docker-compose up -d

Local Installation

git clone <repository-url>
cd rsshub
go build -o rsshub ./cmd/rsshub

Quick Start

1. Start PostgreSQL (if running locally)

docker run -d \
  --name rsshub-postgres \
  -e POSTGRES_USER=postgres \
  -e POSTGRES_PASSWORD=changeme \
  -e POSTGRES_DB=rsshub \
  -p 5432:5432 \
  postgres:15-alpine

2. Set Environment Variables

export POSTGRES_HOST=localhost
export POSTGRES_PORT=5432
export POSTGRES_USER=postgres
export POSTGRES_PASSWORD=changeme
export POSTGRES_DBNAME=rsshub

3. Build and Run

go build -o rsshub ./cmd/rsshub
./rsshub --help

Usage

Add RSS Feeds

./rsshub add --name "tech-crunch" --url "https://techcrunch.com/feed/"
./rsshub add --name "hacker-news" --url "https://news.ycombinator.com/rss"

Start Background Fetching

In terminal 1:

./rsshub fetch
# Output: The background process for fetching feeds has started (interval = 3 minutes, workers = 3)

Manage Configuration (while fetcher is running)

In terminal 2:

# Change fetch interval
./rsshub set-interval 2m

# Adjust worker pool size
./rsshub set-workers 5

View Feeds and Articles

# List all feeds
./rsshub list

# List 5 most recent feeds
./rsshub list --num 5

# Show latest articles from a feed
./rsshub articles --feed-name "tech-crunch" --num 10

# Delete a feed
./rsshub delete --name "tech-crunch"

Stop the Aggregator

Press Ctrl+C in the terminal running rsshub fetch for graceful shutdown.

CLI Commands

Command	Description	Example
fetch	Start background RSS fetching	./rsshub fetch
add	Add new RSS feed	./rsshub add --name "feed" --url "https://..."
set-interval	Change fetch interval	./rsshub set-interval 5m
set-workers	Resize worker pool	./rsshub set-workers 10
list	List RSS feeds	./rsshub list --num 5
delete	Delete RSS feed	./rsshub delete --name "feed"
articles	Show latest articles	./rsshub articles --feed-name "feed" --num 5
--help	Display help	./rsshub --help

Configuration

Environment variables (see .env.example):

Variable	Description	Default
CLI_APP_TIMER_INTERVAL	Feed fetch interval	3m
CLI_APP_WORKERS_COUNT	Number of concurrent workers	3
POSTGRES_HOST	PostgreSQL host	localhost
POSTGRES_PORT	PostgreSQL port	5432
POSTGRES_USER	Database user	postgres
POSTGRES_PASSWORD	Database password	changeme
POSTGRES_DBNAME	Database name	rsshub

Architecture

The project follows Clean Architecture principles:

rsshub/
├── cmd/
│   └── rsshub/         # Application entry point
├── internal/
│   ├── domain/         # Business entities and interfaces
│   ├── app/            # Application services and use cases
│   ├── adapter/        # External adapters (DB, RSS parser)
│   └── cli/            # CLI command handlers
├── migrations/         # Database migrations
├── docker-compose.yml  # Container orchestration
└── .env.example       # Environment configuration template

Example RSS Feeds

• TechCrunch: https://techcrunch.com/feed/
• Hacker News: https://news.ycombinator.com/rss
• BBC World: http://feeds.bbci.co.uk/news/world/rss.xml
• Ars Technica: http://feeds.arstechnica.com/arstechnica/index
• The Verge: https://www.theverge.com/rss/index.xml

Development

Run with Race Detection

go run -race ./cmd/rsshub

Format Code

gofumpt -w .

Build

go build -o rsshub ./cmd/rsshub