A Better openelections.net

[dwillis]

The openelections.net site is badly in need of a refresh. In particular, it doesn't provide updated information about what data we have, what states are in progress and how to use our data collections. Some things I'd like to propose:

• Using activity & issue information from the GitHub API to provide a sense of what is being worked on.
• Add state pages that serve as a hub for what we have for every state.
• Lessen the need to manually update the website.
• Eliminate the need for a backend database for any portion of the website (replace if needed with Google Sheets?)
• Add a section to the docs on how to use our data and summarize that for openelections.net.

I'd love to hear comments on these ideas as well as any other ideas you might think useful. There's a Google doc for brainstorming: https://docs.google.com/document/d/1I90PupYP_5R5rqxnzxbG7hr-uaH2gRo7I6iQhFH9E48/edit#

[miklb]

I mentioned on Twitter that I'm interested in contributing on any Jekyll/plumbing issues.

I'm still not 100% on where GitHub Actions fall with organizational repos but something could be wired together to regularly force a build to pull in new data and rebuild. That would help address "lessen need to manually update" which could include pulling from Sheets, GitHub API and any other data sources.

What type of DB is being used now that you're interested in moving away from?

[dwillis]

@miklb Yeah, I think GitHub Actions might be a good way to go. It would be interesting to see if we could use it to update state-based pages with repository activity. The only database that the site uses (and it uses data exported from it at that) is a Postgres database of election metadata - details about when elections have been held and what offices are on the ballot. I can produce a dump of that and probably will convert it to SQLite.

[miklb]

I've run Jekyll on Travis CI to run a RAKE task to pull JSON via API request and commit it back to the repo in the _data directory. Then "dynamically" populate pages from that. I have plans to do that in an Action instead and believe it would work here as well.

Basically a cron task to pull in new data every X hours/days.

edit Jekyll on GitHub pages specifically.

[dwillis]

I think my preference would be to use Google Sheets as a backend for metadata and GitHub's API for information about specific activities. That likely means coming up with a structure that works well for providing summary information about an election in every state as well as state-specific information.

[miklb]

Were you thinking via Javascript then or still pulling in the data and storing as JSON?

[dwillis]

I think pulling JSON from Google Sheets using Javascript is probably the way to go, although I'm also thinking about SQLite as a backend that we could maintain and publish (maybe even multiple SQLite dbs).

[miklb]

If you go SQLite you could use
https://github.com/simonw/datasette as an api. Or perhaps I’m over thinking it.

[dwillis]

No, I think that's a possibility for sure.

[miklb]

I have some bandwidth to contribute still if there are any tasks I can help with. Should I just look at open issues or wait for you to gather more feedback?

[dwillis]

@miklb I think what I'd like to do is to gather ideas about what sort of things we should have on the site. Like, what should appear on a state page? Which activity should we display on the home page? Maybe a google doc would kick things off: https://docs.google.com/document/d/1I90PupYP_5R5rqxnzxbG7hr-uaH2gRo7I6iQhFH9E48/edit

[miklb]

Excellent idea. I’m not sure I have answers to those questions, but can help look for them.

[nbdavies]

Here's something I mocked up, for how elections/results could be presented interactively (using Vue.js for example), and hosted statically wherever:
https://nbdavies.github.io/openelections-website-wi/

The election details are pulled from JSON on Github (the same data that's in @openelections/openelections-metadata).

Source here: https://github.com/nbdavies/openelections-website-wi/tree/gh-pages

That doesn't tackle the problem of showing what's been updated recently, or what's available/converted vs what isn't. Those kinds of things might require a Github API key, and might not be amenable to a frontend-only site.

[mroswell]

@nbdavies I like the concept. The filters seem to behave in a someone wonky fashion. Took me a long time to figure out that they were "or" filters. Maybe best to add checkboxes to make that clear.

[dwillis]

@nbdavies I agree - I like this, too. I wonder if this could serve as the basis for state pages going forward.