project questions

[h0h0h0]

Did you intend of the project to become a template for others to use? I grabbed this because I was looking for a scrapy/playwright template that was setup already. It came up in the search. Or was this just a scratchpad for you?

1. if you intended for it to be a template we can setup github templates
2. We can make the tutorial.txt a little better. I can send a patch for that.
3. I wonder if there is a way to make create spider already have the playwright stuff included. I did not realize i needed to add the playwright configurations to a spider. it would have been nice to have that in the tutorial.txt or something.

Anyway, I don't know what the intention was. If it was not meant for others to use and consume, my apologies!! I stumbled and used this in error. Otherwise I would send a patch with the above changes.

Thanks dude!

[ibrooz94]

It was initially just a scratchpad, I also put it out here to provide insight to others so yes, we can setup Github templates.

For number 3. It's possible for enable the playwright configurations for spiders globally. I set it up individually so the other spiders not requiring playwright can run without getting slowed down.

scrapy_first/bookscraper-src/bookscraper/spiders/quotespider.py

Lines 11 to 17 in 6c8dfd1

	"DOWNLOAD_HANDLERS": {
	"https": "scrapy_playwright.handler.ScrapyPlaywrightDownloadHandler",
	"http": "scrapy_playwright.handler.ScrapyPlaywrightDownloadHandler",
	},
	"PLAYWRIGHT_LAUNCH_OPTIONS":{
	"headless": False,
	}

You just need to move the lines above to the settings.py

Awaiting your patch. Thanks!