Automate List Updates

[indolering]

It would be ideal if we used the atom feed to update to refresh the official list and run new tests. I'm not sure if you would want it to automatically push to the repository or package mangers, but it would be good if the delta between updates was minimized.

[lupomontero]

Hi @indolering. Thanks for the suggestion. Any ideas as to how you would go about doing this? I'm open to ideas ;-)

[indolering]

Well, you would need to start by fully automating the entire build, test, and (eventually) update process. I would start with Gulp ;). After that, I would be willing to figure out how to wire up the Atom feed to a Github webhook.

[stash]

Would love to see this! I'm willing to contribute some effort to implementing it. ;)

I'm wondering if we can automate this via TravisCI or perhaps negotiate with Salesforce/Heroku to get a charity dyno. I have a meeting with some folks there on Monday and will inquire.

Backstory: I'm a maintainer of tough-cookie (https://github.com/salesforce/tough-cookie), which is owned by Salesforce, which has its own public suffix implementation. If I do a drop-in replacement of psl, it works perfectly and so I'd like to stop re-inventing the wheel. For reference, here's tough-cookie's public suffix implementation: https://github.com/salesforce/tough-cookie/blob/7d66ffde12af5cbad40c3642f3c339fa82e6e381/lib/pubsuffix.js

[stash]

Oooh, hey, TravisCI has a "cron" thing now so maybe a daily poll is good enough? https://docs.travis-ci.com/user/cron-jobs/

https://gist.github.com/willprice/e07efd73fb7f13f917ea might cover the pushing to github part, with a similar idea for npm (i.e., store the auth tokens in an encrypted env var)

[lupomontero]

Hi @stash,

Thanks for your comments. I will totally look into this. This is a feature that has been requested a few times and makes a lot of sense. I'll explore implementation options over the next few days and will post ideas/thought here.

Best,

L

[SnidelyWhiplash]

@lupomontero Any chance there's been some movement on this..?

[lupomontero]

... really sorry for having taken such a long time to look into this... doing so right now...

Many thanks for the references @stash 👍

I'll report back in the next few hours... 🤞 ...

[lupomontero]

In the end I've taken quite a bit longer than I initially expected... 🙈 ... but I think this should do for now. I've added a custom CI script that will automatically fetch the latest PSL list, rebuild the module and send the changes as a PR, which should notify me. I can then double check, merge it in and publish a new release.

I've scheduled a weekly job on Travis CI to to do this. The first run resulted in #50 😉

To help with this, I ended up writing commit-and-pr 😜

Let's see how this goes and if it seems sensible I'm happy to consider further automation.

[controversial]

@lupomontero Those PRs haven’t been merged in 8+ months; are there plans to further automate this process?