Our first production Crawly spider

[dogweather]

https://github.com/public-law/oregon-revised-statutes-crawler

Code review / critique is appreciated.

Coding it was a very smooth experience. No unpleasant surprises; everything worked as expected. The Domo library and the new dbg debugger contributed the smooth experience. In some ways, it was easier than Scrapy/Python.

The spider creates a 60mb JSON lines file. It uses one API call and approx. 500 web pages as its source.

[serpent213]

Thank you for the example, learned a few things!

Unfortunately the service seems to be down right now...