Skip to content

sh-alireza/General-Crawler

Folders and files

NameName
Last commit message
Last commit date

Latest commit

 

History

10 Commits
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 

Repository files navigation

Data Crawler

This is a simple data crawler that crawls data from a website and stores it in a database. Then you can download your data in your local storage. For now it only supports crawling data from houzz, amazon(only some products), google-images, youtube.

Installation

Go to the root of the project and run the following commands:

pip install -r requirements.txt
pip install -e .

Usage

crawler

Contributing

Pull requests are welcome. You can add your own website crawler and downloader to help the project grow.

About

General Crawler with inquirer

Topics

Resources

License

Stars

Watchers

Forks

Packages

No packages published