Skip to content

Production Deployment

Jump to bottom
Stuart Olivera edited this page Jan 11, 2018 · 11 revisions

Below are steps & notes to get BrickHack's hosting infrastructure setup. This assumes you already have Dokku running on a machine and can SSH into the box. DNS should likely be set up as well but isn't required for bare minimum functionality.

For example purposes, we'll be talking about the brickhack-stage app, but the same goes for brickhack-prod (just with slightly different environment variable values).

If you have any questions at all, please don't hesitate to reach out to Stuart! This doc is very much a work in progress but we want to keep it as up to date as possible.

Deploying

Travis will automatically deploy to staging or production based on the branch.

  • develop branch -> brickhack-stage Dokku app
  • master branch -> brickhack-prod Dokku app

Dokku

Currently used and required Dokku plugins (other than the defaults):

Dokku Setup Steps

dokku apps:create brickhack-stage
dokku config:set brickhack-stage MYSQL_DATABASE_SCHEME=mysql2
dokku mysql:create brickhack-stage
dokku mysql:link brickhack-stage brickhack-stage
dokku redis:create brickhack-stage
dokku redis:link brickhack-stage brickhack-stage
dokku checks:disable brickhack-stage worker
dokku config:set brickhack-stage \ [environment variables]

Where [environment-variables] is a list of all environment variables:

BUILDPACK_URL=https://github.com/heroku/heroku-buildpack-ruby.git \
ENVIRONMENT="" \
AWS_BUCKET="" \
AWS_ACCESS_KEY_ID="" \
AWS_SECRET_ACCESS_KEY="" \
AWS_REGION="" \
ROLLBAR_ACCESS_TOKEN="" \
ROLLBAR_ENV="" \
SECRET_KEY_BASE="" \
MLH_KEY="" \
MLH_SECRET="" \
DOKKU_DEPLOY_HOOKS_PREFIX=/app \
GOOGLESHEETS_KEY="" \
SPARKPOST_API_KEY=""

You can generate secrets via bundle exec rake secret

Once all configuration is set, add Dokku as a remote & run an initial deploy.

Initial deploy

First, we have to disable our CHECKS. Since our initial deploy won't have a working database, our checks will fail and block deploys.

On the server, run:

dokku checks:disable brickhack-stage web

Then, do a local deploy to Dokku (skipping Travis):

git remote add dokku dokku@csh-cloud.oweb.co:brickhack-stage
git push dokku master

By doing a test push locally, you'll be able to easily see the progress of the build and any errors. Debugging deploys via Travis leads to scrambled logs and excess build failing emails (for everyone).

Once this succeeds, return back to the server to re-enable our web checks and seed the now-prepared database.

dokku checks:enable brickhack-stage web
dokku run brickhack-stage bin/rails db:seed

Validating initial deploy

  • Deploy should succeed without any red flags in the build log
  • Should be able to submit an application on the website & receive an immediate confirmation email

If everything works, verify any configs for Travis (web UI and .travis.yml) and do a test deploy via Travis.

Nginx Config

Usually, there's no need to modify the nginx config for the apps. However, we have a few special cases.

  1. Create the directory /$DOKKU_ROOT/$APP/nginx.conf.d/ (eg /home/dokku/brickhack-stage/nginx.conf.d/)
  2. Add files ending in .conf (such as rewrites.conf) that you want loaded
  3. Restart nginx: dokku nginx:build-config $APP

Sidekiq

Sidekiq's web UI will throw a 502 Gateway error out of the box on production. To fix this, increase the nginx buffer size.

Create proxy_buffer.conf with the following:

# Fix for Sidekiq web console
proxy_buffer_size   128k;
proxy_buffers   4 256k;
proxy_busy_buffers_size   256k;

Resumes

Support decently-sized resumes.

Create upload.conf with the following:

client_max_body_size 2M;

Gallery

To do things like redirect /gallery to /gallery/, an nginx rule is needed.

Create a rewrites.conf with the following:

rewrite ^/gallery?$ http://$host/gallery/ permanent;

Don't forget to restart nginx after both of these files have been added: dokku nginx:build-config $APP

Production Branches

By default, Dokku apps use the master branch. However, for apps such as our staging app, we use the develop branch. For this, we need to specify local & remote branches at the time we push:

git push dokku develop:master

MySQL

Database URL

By default, dokku-mysql sets the DATABASE_URL to use mysql://, but rails needs mysql2://.

To fix this before linking, just run dokku config:set brickhack-stage MYSQL_DATABASE_SCHEME=mysql2.

Otherwise if the database is already linked, you must manually update the URL:

  1. Copy the current url: dokku config:get brickhack-stage DATABASE_URL
  2. Set the url, but replace mysql with mysql2: dokku config:set brickhack-stage DATABASE_URL=...

The dokku-mysql README has more info on this as well.

MySQL Timezone Tables (Groupdate)

Update: Looks like dokku-mysql has timezone information by default, so this shouldn't be necessary.

In order to support groupdate, timezone tables must be created.

mysql_tzinfo_to_sql /usr/share/zoneinfo | mysql -u $OPENSHIFT_MYSQL_DB_USERNAME -p mysql

Environment Variables

Various services require environment variables to operate. These variables are set via Dokku's dokku config:set tool.

Resumes

Resumes are stored locally in development and on AWS in production.

E-mail

Currently, emails are queued via Sidekiq and then sent by Sparkpost's servers.

Create a Sparkpost API key with Transmissions: Read/Write and Message Events: Read-only permissions, limited to the production server's IP address. SMTP is not required, as email is sent over the Sparkpost API rather than SMTP.

SPARKPOST_API_KEY=""

Rollbar

Rollbar captures and notifies of errors in production, and requires a server-side access token.

ROLLBAR_ACCESS_TOKEN=""
ROLLBAR_ENV="" # staging or production

My MLH

My MLH provides us authentication & initial application information.

MLH_KEY=""
MLH_SECRET=""

Slack

Invites to Slack can be automatically sent.

SLACK_SUBDOMAIN=""
SLACK_API_TOKEN=""
SLACK_SIGNUP_URL=""

Where the subdomain is the subdomain piece of the Slack URL (e.g. subdomain.slack.com), the API token is a legacy token generated by an admin or owner, and the sign up URL is a fallback link that users can click to join the workspace. The sign up URL is provided by Slack's admin interface, or by contacting support.

Google Sheets

The Google Sheets API is used to grab up-to-date schedule information. To obtain an API key:

  1. Visit https://console.developers.google.com/apis/credentials
  2. In the "Create Credentials" dropdown, click "API Key"
  3. Go to the "Library" tab on the right
  4. Search for "Google Sheets" and enable the API
GOOGLESHEETS_KEY=""

Action Mailer Assets

By default, all assets included in emails will be served from https://brickhack.io. However for staging, it is desirable to have staging assets used. To do so, set the environment variable:

RAILS_MAILER_DOMAIN="staging.brickhack.io

Skylight

Skylight provides detailed performance analytics for our app.

SKYLIGHT_AUTHENTICATION=""
Clone this wiki locally