Web Scraper for ML Models

Project Overview

I created a web scraping website that scrapes a news website and returns live news data in a CSV format based on the selected genre.

Live Link

Visit the Web Scraper

Tech Stack

• Backend: Go
  A statically typed, compiled programming language designed for simplicity and efficiency, ideal for building scalable web applications.

• Framework: Fiber
  A fast and minimalist web framework for Go, which provides a simple API to create web applications and APIs with improved performance.

• Web Scraping: Colly
  A powerful web scraping framework for Go that allows you to crawl websites and extract data easily and efficiently.

• Frontend:

  • Shadcn
    A UI component library designed for building beautiful and responsive user interfaces with ease.

  • Tailwind CSS
    A utility-first CSS framework that provides low-level utility classes for rapid UI development, promoting consistent design.

  • Daisy UI
    A component library that works with Tailwind CSS, providing pre-designed UI components that are easy to customize and implement.

Features

• Scrape news articles based on user-selected genres.
• Generate downloadable CSV files with the latest news data.
• Live updates to ensure data is current.

Future Ideas

• Make an ML Model to create an AI/ML project using the data!

Special Thanks to The Hindu