Thinking about UTRS deployment process

[supertassu]

The UTRS deployment process needs to improve. Main pain points currently are:

• We have no staging site, all changes merged to master are immediately deployed to production

One way to solve this would be to create a separate branch, stable (or production, or something else), that is used to run the production site. That way we can have a completely separate manually controller branch for production and we can still cherry-pick important fixes to it from master if needed.

Also, we need to create a staging site from the master branch. (see #61) That way we can play around with things and hopefully find breakage before it's on production.

• Built assets are commited to Git

See #240

• The process is too manual.

You need to manually run php artisan down/up, update dependencies, execute database migrations and perform changes to .env.

---

I'm thinking about writing some kind of script that will automate most of the process, with a human watching that it doesn't break. Something that will automatically update dependencies, execute migrations, etc. when it detects those files have been changes. The script will probably live in a separate repository.

We could also experiment with having each version be a separate clone and symlink the current version to webroot when ready and that way make the downtime shorter.

[supertassu]

I'm planning to work on this soon, so I've drafted some more detailed plans. @dqwiki does this look good / am I missing anything obvious?

• Two main use cases:
  • Human-watched deployment to production
  • Automated deployments to a testing site
• Versioning
  • have a separate checkout directory for each version that is deployed, symlink the current running version to a path that's configured on the web server
  • easier rollbacks, just revert the symlink change
  • can prepare assets/install dependencies without downtime
• Dependencies
  • Install new dependencies if lock files change, otherwise just use existing
  • keep separate folder for dependency folders, symlink it to place? that would allow to prune old versions even if dependencies haven't changed
• Asset building
  • Do kind of same thing as with dependencies, change if dependencies, asset files or frontend templates change, otherwise use the assets from last version
  • Move assets from the main github repo to locally hosted on the server
• Automation
  • try to automate as much as possible, with still keeping a human in the loop to unbreak broken things
  • suggestion: target a two-command approach, optionally aliased into one command for speed: one to prepare a new version, another to deploy it
  • one command to deploy to a previous version
  • nightly(?) cronjob to prune old versions that haven't been used in x days and isn't the previous one
  • automatically detect if dependencies need updating/migrations that need to be run and do that
  • database migrations are tricky, see below
• Logging
  • need to be able to (publicly/searchable) log the timestamp and version of all versions
  • integrate with https://sal.toolforge.org/utrs? and slack
  • log a verbose log of actions to a file on disk
• Database migrations
  • are a pain point due to the ability to rollback.
  • take a backup beforehand, let a human deal with issues
• Development version
  • Automate updates, with a github webhook or similar
  • Integrate with github commit status checks? not a hard requirement