TSE data utilities

Scraps all the from the election results from Tribunal Superior Eleitoral files at:

Please read their policy and docs before using the tool.

TSE Docs:

English terms took (mostly) from:

Prerequisites

Python 3.6+
Python Poetry: https://python-poetry.org/docs/#installation
- Or use brew install poetry
Install the dependencies poetry install
Activate shell/virtual environment poetry shell

Run scrapy crawl divulga to update all the files
- Scrapping is incremental and can be peformed continuously to get latest data if available,
- Files are downloaded to data/download/...
Run scrapy crawl urna to download all the original files transmitted from the voting machines (bulletins, logs, etc),
- Beware that it's above 472k electoral sections with 6 files and about 200kb per section, totalling 2.8 million files 90gb
Edit tse/setting.py to customize paths, network usage, narrow down filters, etc.

Name		Name	Last commit message	Last commit date
Latest commit History 142 Commits
.vscode		.vscode
data		data
notebooks		notebooks
tse		tse
.gitignore		.gitignore
README.md		README.md
poetry.lock		poetry.lock
pyproject.toml		pyproject.toml
scrapy.cfg		scrapy.cfg