-
Notifications
You must be signed in to change notification settings - Fork 37
Open
Description
Context
The CityProtect scraper could use a rework. It works well enough in what it does but is outdated and the instructions in the README are confusing
Requirements
- Should work the same as it currently does
- Should use functional programming so it can be called upon by scrapers as a module
- The
configs.py
method should be scrapped in favor of passing function parameters to make things more straightforward - There should be a template detailing how a user may call upon the module in their own scraper. See Open Data Template
- The CityProtect scraper and CityProtect_Bulk script should probably live in one folder together
- Reworked README
- Cleanup the old or useless comments
Docs
- The README should be clear and concise in how to setup and run the CityProtect scraper
Open questions
- There's a lot of adding and condition statements to retrieve file names are urls. Perhaps this could be reworked with datetime somehow to make it simpler?
- The CityProtect_Bulk scraper seems to use an API, perhaps this could be looked into implementing into the CityProtect scraper as well?
Metadata
Metadata
Assignees
Labels
No labels