⚡ Multi-Web Product Scraper (Stealth Mode | Blazing Fast | NLP for queries | Cache Management ) ⚡

Search Amazon, Flipkart, and Myntra — all at once, way before your coffee is ready.

Built with async Playwright for stealth, speed.

---

🚀 What's Cookin’ :

• 🔍 Multi-Site Support: Scrapes all products from Amazon, Flipkart, and Myntra ( more coming soon . . . ).
• 🧠 Natural Language Processing : Uses all-MiniLM-L6-v2 model for understanding user queries.
• 🛢️ Cache Management : Uses aiosqlite to cache the searched results to avoid computation expences.
• 📈 150+ Products in 18 Seconds: Yup, tested — ~152 products loaded in one go ( 18-20s sec ).
• 🖼️ Photo Gallery : Product photos are saved with dedicated folder.
• ⚡ SuperFast Scraping: Built with asyncio, aiohttp, and Playwright, scrapes around 150 products within 20 sec 💨
• 🕶️ Headless & Stealth Mode: Runs in the background and mimics human behavior to dodge bot detection.
• 🧠 Why This Slaps: Manually opening 3 sites, scrolling forever, and remembering deals? Nah. Just type what you want — this gives you 150+ options instantly.

---

💻 How To Run

Clone this repo :

git clone https://github.com/Aniket-16-S/product-Sraper.git

cd product-Sraper 

install dependencies :

pip install -r requirements.txt

playwright install

Run the async scraper :

python main_scraper.py

Enter your product keyword and let it cook 🔥 for 20 secs. Note : Please Wait for 40 to 50 sec for initial setup on first run.

---

📂 Output

Products from all 3 sites shown in your terminal.

Images of all products store in your File Mansger.

Clean, structured format — easy to compare.

Great for research, trend tracking, or flexing that dev power.

---

🔮 Future Goals

🛍️ Add support for more sites like Ajio, Meesho, Nykaa, etc.

🎛️ GUI version for non-coders

---

⭐ Why This Could Blow Up

Tired of opening 10 tabs to compare products ? This solves that.

Has advance features like Natural Language Processing for query matching and Cache Management.

Get more than 100 products scraped at your terminal in 16 - 20 secs. ( assuming decent network connectivity. )

It’s async and hence faster.

With a clean UI or web version, this could go viral as a price-compare tool.

Dev-friendly = easy stars + forks on GitHub.

Great side-project flex for portfolios or hackathons.

---

⚠️ Legal Note

This project is for educational use only. Respect each site’s terms of service and robots.txt. The author is not responsible for any misuse or violations of third-party terms and conditions.

---

🤝 Contributions Welcome

Got new site targets or bug fixes? Open a PR or drop ideas in issues!

Let’s build the ultimate cross-site scraper 🧃