Pulsar stack upgrade: Heroku-18 to Heroku-22

[twang15]

Heroku-18 is end-of-life soon in May 1st, 2023.
https://help.heroku.com/X5OE6BCA/heroku-18-end-of-life-faq

Need to upgrade to Heroku-22, here is the step-by-step official guide:
https://devcenter.heroku.com/articles/upgrading-to-the-latest-stack

[twang15]

Previous reference:
#62
#245

[twang15]

Possible failures:

• Elastic search: Pulsar bug: heroku-18, elasticsearch does not work properly #247
• Active-model-serializer: Pulsar bug fix: active-model-serializer does not work properly #242

[twang15]

bundle install:

• if there is no Gemfile.lock or Gemfile is newer, running this will generate Gemfile.lock.
• if Gemfile.lock is newer than Gemfile, running this will install the exact dependencies locked by previous run of "bundle install" or "bundle update"

bundle update

• always generate a new Gemfile.lock

[twang15]

The previous heroku-18 was built with bundle 1.17.3 and the Gemfile does not work any more. I don't know when my bundler was updated to 2.5. Afterwards, running bundle install w/ the following revision in staging code failed with error message below:

# revision in staging repo
commit a6e326eafb5ea70d58d17de5a93b4c6c7c70f50c (HEAD -> master, origin/master, origin/HEAD)
Author: tao <twang15@ncsu.edu>
Date:   Mon Mar 14 14:03:22 2022 -0700

# error message
Try us-west-2 for less expense

Using elasticsearch-model (5.1.0) from https://github.com/elastic/elasticsearch-rails.git (at 5.x)
NoMethodError: undefined method `spec' for nil:NilClass
An error occurred while installing elasticsearch-model (5.1.0), and Bundler cannot continue.
Make sure that `gem install elasticsearch-model -v '5.1.0'` succeeds before bundling.

The solution is to uninstall bundler 2.5 and install 1.17.3 with the following code:

gem uninstall bundler
gem install bundler -v '1.17.3'

and then change Gemfile by replacing git:// with https:// for git repo of elasticsearch-model and elasticsearch-rails:

  diff --git a/Gemfile b/Gemfile
  index 387b4e9d..01be72ec 100644
  --- a/Gemfile
  +++ b/Gemfile
  @@ -13,15 +13,17 @@ gem 'kaminari'
   #    2.x       →       2.x
   #    5.x       →       5.x
   #    master    →       master
  -gem 'elasticsearch-model', github: 'elastic/elasticsearch-rails', branch: '5.x'
  -gem 'elasticsearch-rails', github: 'elastic/elasticsearch-rails', branch: '5.x'
  -#gem 'elasticsearch-model', github: 'elastic/elasticsearch-rails', branch: 'master'
  -#gem 'elasticsearch-rails', github: 'elastic/elasticsearch-rails', branch: 'master'
  +gem 'elasticsearch-model', git: 'https://github.com/elastic/elasticsearch-rails.git', branch: '5.x'
  +gem 'elasticsearch-rails', git: 'https://github.com/elastic/elasticsearch-rails.git', branch: '5.x'
  +#gem 'elasticsearch-model', github: 'elastic/elasticsearch-rails', branch: '5.x'
  +#gem 'elasticsearch-rails', github: 'elastic/elasticsearch-rails', branch: '5.x'
  +##gem 'elasticsearch-model', github: 'elastic/elasticsearch-rails', branch: 'master'
  +##gem 'elasticsearch-rails', github: 'elastic/elasticsearch-rails', branch: 'master'

[twang15]

to rebuild staging site without any source modification:

  git add Gemfile Gemfile.lock
  git commit -m "Heroku-18 build on 04-20-2023 without change to source code. Bundler 1.17.3, ruby 2.6.5."
  git push

  #There was a failure message:
  
  remote: !	Your app is using the Heroku-18 stack, which reaches end-of-life on April 30th, 2023.
  remote: !
  remote: !	Builds on Heroku-18 will not be possible from May 1st, 2023.
  remote: !
  remote: !	Please upgrade the stack of your app as soon as possible:
  remote: !	https://devcenter.heroku.com/articles/upgrading-to-the-latest-stack
  remote: !
  remote: !	Heroku-18 EOL FAQ: https://help.heroku.com/X5OE6BCA/heroku-18-end-of-life-faq
  remote: !
  remote: !	THIS IS A ONE-OFF NOTICE. THE NEXT DEPLOY WILL FUNCTION WITHOUT INTERRUPTION.
  remote: !
  remote: !	The next warning of this kind will occur in: 1 day(s).
  remote:
  To https://git.heroku.com/staging-pulsar-encode.git
   ! [remote rejected]   master -> master (pre-receive hook declined)
  error: failed to push some refs to 'https://git.heroku.com/staging-pulsar-encode.git'

  git push #2nd try w/o modification succeeds
  # rerun w/o error
  remote:   _                    _                __  ___    ______ ____  _
  remote:  | |                  | |              /_ |/ _ \  |  ____/ __ \| |
  remote:  | |__   ___ _ __ ___ | | ___   _ ______| | (_) | | |__ | |  | | |
  remote:  | '_ \ / _ \ '__/ _ \| |/ / | | |______| |> _ <  |  __|| |  | | |
  remote:  | | | |  __/ | | (_) |   <| |_| |      | | (_) | | |___| |__| | |____
  remote:  |_| |_|\___|_|  \___/|_|\_\\__,_|      |_|\___/  |______\____/|______|
  remote:
  remote: This app is using the Heroku-18 stack, which is deprecated:
  remote: https://devcenter.heroku.com/changelog-items/2432
  remote:
  remote: From April 30th 2023, Heroku-18 will be end-of-life, and apps using it
  remote: will no longer receive security updates, and be run at your own risk.
  remote:
  remote: From May 1st 2023, builds will no longer be allowed for Heroku-18 apps.
  remote:
  remote: Please upgrade to a newer stack as soon as possible:
  remote: https://devcenter.heroku.com/articles/upgrading-to-the-latest-stack
  remote: https://help.heroku.com/X5OE6BCA/heroku-18-end-of-life-faq
  remote:
  remote: **Verifying deploy... done.**
  To https://git.heroku.com/staging-pulsar-encode.git
     a6e326ea..e2634776  master -> master

[twang15]

with "self.root = true", the following code causes error "SystemStackError (stack level too deep)":

  class ChipseqExperimentSerializer < ActiveModel::Serializer
    embed :ids
    #self.root = false
    self.root = true
  
    attributes :id,
               :document_ids,
               :description,
               :name,
               :notes,
               :submitter_comments,
               :target_id,
               :chipseq_starting_biosample_ids,
               :upstream_identifier,
               :upstream_identifier,
               :wild_type_control_id,
               :created_at,
               :updated_at
  
  
    has_one :user
    has_many :control_replicates
    has_many :replicates
    has_many :chipseq_starting_biosamples
    #has_many :documents
  end

The error you're encountering suggests that there is an infinite recursion or circular reference happening somewhere in the serialization process. This can happen when you have a circular reference in your associations or if you're embedding a serializer that also embeds the current serializer.

When you set self.root = true, it means that the serialized output will include a root element with the name of the serializer. This can cause issues when you have circular references in your associations, since the serializer will keep trying to embed itself in an infinite loop.

To fix this issue, you can try setting self.root = false instead of self.root = true. This will remove the root element from the serialized output and may help to prevent the infinite recursion.

You can also try to debug the circular reference by commenting out the associations one by one and see which one is causing the issue. Once you identify the problematic association, you can try to remove it or find a way to break the circular reference.

[twang15]

production rebuild w/o code change, 04/20/2023

commit f17e3c6 (HEAD -> master, prod-heroku/master, origin/master, origin/HEAD)
Author: tao twang15@ncsu.edu
Date: Thu Apr 20 20:24:23 2023 -0700

Heroku-18 build on 04-20-2023 without change to source code. Bundler 1.17.3, ruby 2.6.5.

[twang15]

https://devcenter.heroku.com/articles/upgrading-to-the-latest-stack#rolling-back

In case of rolling back, for staging, roll back to v389 with "heroku rollback v389"

tao1@MacBook-Pro staging-pulsar-encode % heroku releases
=== staging-pulsar-encode Releases - Current: v389
v389  Deploy 7e987119                       taowang9@stanford.edu                2023/04/20 19:43:16 -0700 (~ 1h ago)
v388  Deploy 720ecab2                       taowang9@stanford.edu                2023/04/20 19:34:11 -0700 (~ 1h ago)
v387  Deploy e2634776                       taowang9@stanford.edu                2023/04/20 19:25:26 -0700 (~ 1h ago)
v386  Update DATABASE by heroku-postgresql  heroku-postgresql@addons.heroku.com  2023/03/15 14:35:41 -0700
v385  Update DATABASE by heroku-postgresql  heroku-postgresql@addons.heroku.com  2023/01/18 13:45:30 -0800
v384  Update DATABASE by heroku-postgresql  heroku-postgresql@addons.heroku.com  2022/10/05 14:46:24 -0700
v383  Update DATABASE by heroku-postgresql  heroku-postgresql@addons.heroku.com  2022/07/20 14:37:21 -0700
v382  Update DATABASE by heroku-postgresql  heroku-postgresql@addons.heroku.com  2022/04/29 16:31:15 -0700
v381  Update DATABASE by heroku-postgresql  heroku-postgresql@addons.heroku.com  2022/04/27 16:13:00 -0700
v380  Set S3_BUCKET config vars             taowang9@stanford.edu                2022/03/15 07:31:15 -0700
v379  Deploy a6e326ea                       taowang9@stanford.edu                2022/03/14 14:04:00 -0700
v378  Deploy e491e602                       taowang9@stanford.edu                2022/03/14 13:54:54 -0700
v377  Set S3_BUCKET config vars             taowang9@stanford.edu                2022/03/14 13:34:05 -0700
v376  Set S3_BUCKET config vars             taowang9@stanford.edu                2022/03/14 13:21:23 -0700
v375  Set S3_BUCKET config vars             taowang9@stanford.edu                2022/03/14 10:59:08 -0700

for production, roll back to v707 with "heroku rollback v707"

  tao1@MacBook-Pro pulsar_lims % heroku releases
  === pulsar-encode Releases - Current: v707
  v707  Deploy f17e3c61                        taowang9@stanford.edu                2023/04/20 20:26:50 -0700 (~ 24m ago)
  v706  Update DATABASE by heroku-postgresql   heroku-postgresql@addons.heroku.com  2023/03/20 11:09:40 -0700
  v705  Update DATABASE by heroku-postgresql   heroku-postgresql@addons.heroku.com  2023/02/13 10:13:36 -0800
  v704  Update DATABASE by heroku-postgresql   heroku-postgresql@addons.heroku.com  2022/11/07 10:08:41 -0800
  v703  Update DATABASE by heroku-postgresql   heroku-postgresql@addons.heroku.com  2022/08/22 11:09:26 -0700
  v702  Update DATABASE by heroku-postgresql   heroku-postgresql@addons.heroku.com  2022/05/16 11:10:37 -0700
  v701  Deploy a7fa3aea                        taowang9@stanford.edu                2022/03/15 07:45:15 -0700
  v700  Set AWS_SECRET_ACCESS_KEY config vars  taowang9@stanford.edu                2022/03/15 07:30:17 -0700
  v699  Set AWS_ACCESS_KEY_ID config vars      taowang9@stanford.edu                2022/03/15 07:29:51 -0700

[twang15]

tao1@MacBook-Pro staging-pulsar-encode % rbenv install 3.1.4
ruby-build: definition not found: 3.1.4

See all available versions with `rbenv install --list'.

If the version you need is missing, try upgrading ruby-build:

brew update && brew upgrade ruby-build