ActiveModel::MissingAttributeError on custom `select`

[jrochkind]

originally from @guilhermegazzinelli

---

Hi Jonathan,

I had the same issue, but with custom types. The error is as follows:

*** ActiveModel::MissingAttributeError Exception: missing attribute: settings

In another investigation I got:

ActiveModel::MissingAttributeError: missing attribute: settings from /(...)/ruby/3.1.3/lib/ruby/gems/3.1.0/gems/activemodel-7.0.4.2/lib/active_model/attribute.rb:226:in 'value'

This appeared when I do:

Form.select(:id, :name).last

I'll try to investigate deeper and post here my findings.

Code Excerpt

class Form < ApplicationRecord
  include AttrJson::Record
  attr_json_config(default_container_attribute: :settings)
  attr_json :form_settings, Json::FormSettings.to_type, default: nil 
  attr_json :name,:string, default: "Default name"
end

class Json::FormSettings
  include AttrJson::Model

  attr_json :name, :string, default: nil #
  attr_json :department, ActiveModel::Type::Value.new, array: true,  default: nil #
end

More info:

• Rails 7.0.4.2
• Bundler version 2.4.6
• attr_json (2.0.0)

[jrochkind]

Hi @guilhermegazzinelli I have reproduced this.

You may not be able to do select unless you include your json column , in this case settings. I'm investigating further.

It does work if you do Form.select(:id, :settings).last. But should it be possible without including :settings in the select? I'm thinking through it, testing other analogous ActiveRecord things to see how they work, and generally experimenting.

I'll note a couple other probably unrelated oddities in your example:

• You do select(:name)... but :name is an attr_json, it's not actually a column in your database. So you won't be able to include it in the select, which is just for database columns. Or is it both a database column and you use it as an attr_json? Cause that would probably be a problem too.

• attr_json :department, ActiveModel::Type::Value.new, array: true, That is a weird thing to do. You mean to declare an array of... raw json-compatible data? I guess that would work to do it... but it's odd. And I'm not sure what array: true gets you. Not sure what you're trying to do there or if this is the best way, but probably not related to this issue so we don't have to discuss it more here. (Feel free to open up a Discussion if you'd like to).

[jrochkind]

OK, so let's look at how ordinary ActiveRecord (without json/attr_json) behaves when selecting only certain attributes.

Let's say we have an ordinary AR model like this:

class CreateForms < ActiveRecord::Migration[7.0]
  def change
    create_table :forms do |t|
      t.string :name
      t.timestamps
    end
  end
end

class Form < ApplicationRecord
end

Form.create(name: "My Name")

Form.last.name # => "My Name"

What happens if we do a select without the name column?

form = Form.select(:id).last
# we got this far
form.name
# NOW raises...

Raises: missing attribute: name (ActiveModel::MissingAttributeError)

So in ordinary Rails, if you try to access an attribute you did not include in a select, it will raise at the point you try to access it.

It does let you set form.name = "something" even if it wasn't included in the select; it will override whatever is in the DB.

So at best we want to match this -- if you don't include settings (in your example configuration; normally json_attributes) in the select, you will not be able to access any of your attr_json values; it will raise when you do. This is expected.

But you should be able to do the fetch -- right now it raises when you just do Form.select(:id), you can't even succesfully fetch. This is a bug I would like to fix.

You will not be able to do a select(:name) when name is an attr_json -- we can't support that. If you want any of your attr_json, you need to include :settings in the select.

Should you be able to set your attr_json attributes if you didn't include :settings in the select? I'm not sure, have to think about it/experiment. I would lean towards maybe not, but am curious what the code will do as-is, once I fix the mentioned bug.

[jrochkind]

I like to compare to "peer" project https://github.com/madeintandem/jsonb_accessor

What does it do if you do a select without including the json column? It looks like it fetches fine (of course). Then if you access a json-backed attribute? It does let you access the attribute without an error -- and get nil even if there actually was something in the DB. But if you try to set the json-backed attribute, you get a ActiveModel::MissingAttributeError.

That it lets you access and get nil is actually surprising to me; it doens't strike me as preferable though.

Consistency with AR ordinary attributes, which raise on access if they weren't included in the select, seems fine and possibly preferable to me.

It avoids raising on the fetch with select missing json column, by a guard check:

https://github.com/madeintandem/jsonb_accessor/blob/add9d3ff96d756c220c0c06ee331e19562f38cc9/lib/jsonb_accessor/macro.rb#L105

Which seems a fine thing to emulate.

[jrochkind]

OK, after #193, you will be able to do a custom select that does not include your json container -- the fetch will work, and you will be able to access other non-attr-json attributes so long as they were included in the select.

If you try to access an attr_json attribute backed by a json column that was NOT included in the select, you will still get a ActiveModel::MissingAttributeError. This is by design.

You can't select an attr_json attribute, like you tried to do above with :name. attr_json does not support that, and there isn't any good way for it to support that. After #193 if you try that, you'll get an exception PG::UndefinedColumn: ERROR: column "name" does not exist (ActiveRecord::StatementInvalid) -- this is considered correct.

Thank you for your report @guilhermegazzinelli, I appreciate it!

[guilhermegazzinelli]

@jrochkind
Thanks for your Fix, but even after upgrade I still have some problems:

Quoting:

If you try to access an attr_json attribute backed by a json column that was NOT included in the select, you will still get a ActiveModel::MissingAttributeError. This is by design.

My problem appears, in the same previous scenario, when you do:

f = Form.select(:id, :name) # select only non attr_json attributes
f.to_json 
# => Raises ActiveModel::MissingAttributeError: missing attribute: settings
# from /home/ggr/.asdf/installs/ruby/3.1.3/lib/ruby/gems/3.1.0/gems/activemodel-7.0.4.2/lib/active_model/attribute.rb:226:in `value'

If I call only f.to_json(only: [:id, :name]) it works.

About this:

You can't select an attr_json attribute, like you tried to do above with :name. attr_json does not support that, and there isn't any good way for it to support that. After #193 if you try that, you'll get an exception PG::UndefinedColumn: ERROR: column "name" does not exist (ActiveRecord::StatementInvalid) -- this is considered correct.

It was my mistake, I tried to simplify the model for the POC and didn't realized that the select was including :name. This doesn't happen on the real scenario.

More info:

f.attributes
# => {"id"=>7129, "name"=>"Checklist de abertura de loja 02", "form_settings"=>nil, "kind"=>nil}
f.serializable_hash
# => ActiveModel::MissingAttributeError: missing attribute: settings
# from .../3.1.3/lib/ruby/gems/3.1.0/gems/activemodel-7.0.4.2/lib/active_model/attribute.rb:226:in `value'

Thanks for your wonderful work.

[jrochkind]

OK, thanks @guilhermegazzinelli , it looks like I fixed a problem with loading the model at all with custom select, but there's another problem with to_json and serializable_hash on the model when you have a custom select not including the json_attributes.

This surprises me, but we should be able to get it figured out somehow. I probably won't have time to work on it until next week.

PR's not required but always welcome and can help get things solved quicker -- including a PR just for a failing test, even without a solution!

Can you file a separate ticket making it clear there is a problem with calling #serializable_hash when you have a custom select not including the attr_json column? And the complete stack trace would be great too! Hopefully I will be able to easily reproduce.

There may be other problems waiting too, I can only fix the problems we find, I don't know every possible thing that might have a bug -- I am surprised about this one!

[jrochkind]

Oh okay I sort of understand what's going on.

This is potentially a bit difficult to fix, although there might be some ways.

Can you tell me more about your use case? I want to think about what might be the best solution.

Why are you doing a partial select? Why are you then wanting to #to_json?

If there are attributes backed by the json column, when you've left the json column out of the select, would you expect them to show up in the to_json output with nil results, or would you expect the keys to be left out of the json results, or you don't care?

[guilhermegazzinelli]

Hi Jonathan, Thanks for the fast reply. This week I won't be able to address a PR for a failing test or a solution but I'll try in the next week. Hope I can help with this bug. Answering yours questions:

Can you tell me more about your use case?

In my scenario I do a partial select in the index, so the application don't need a big amount of settings stored in the attr_json. I render the json directly from this array of fetched fields. In database backed fields, if you do `to_json` non fetched data comes as nil. As mentioned before, this seems related to `/3.1.0/gems/activemodel-7.0.4/lib/active_model/attribute.rb:226:in `value': missing attribute: last_sign_in_at (ActiveModel::MissingAttributeError)` this also occurs in the Devise lib with the `:last_sign_in_at`. Next week I'll do a deeper search, and open a proper ticket. About being a Rails issue:

either way I'll do the work to report the Issue to Rails if I can't find an existing one.

If you open an issue there could you reference me or this issue so we can follow the progress? Thanks for all. And hope I can help you.