default_scope leaks into ThroughAssociation queries

[oehlschl]

We've had Goldiloader pinned at 4.2.0 for a while now due to some SQL errors we encountered in specs when upgrading to 4.2.1, believing it was an issue with our associations. But it looks like Goldiloader 4.2.1+ (including current version 5.4.0) has a bug where a model's default_scope incorrectly leaks into queries for has_many :through associations with scopes, causing SQL errors due to invalid table references in ORDER BY clauses.

Specifically, when a model with a default_scope has a has_many :through association with a scope, Goldiloader's ThroughAssociationPatch (introduced in 4.2.1) incorrectly applies the default_scope's ORDER BY clause to the through association's target query, resulting in SQL errors:

PG::UndefinedTable: ERROR:  missing FROM-clause entry for table "blogs"
LINE 1: ...ROM "posts" WHERE "posts"."id" IN (1, 2) ORDER BY "blogs"."posit...
                                                                    ^

To reproduce:

• Goldiloader 4.2.1+
• Rails 7.1.5
• Ruby 3.4.6

# app/models/blog.rb
class Blog < ApplicationRecord
  # This default_scope causes the bug
  default_scope { order(position: :asc) }

  has_many :blog_posts, dependent: :destroy

  # These through associations with scopes trigger the bug
  has_many :published_posts, -> { where(status: 'published') },
           through: :blog_posts,
           source: :post

  has_many :draft_posts, -> { where(status: 'draft') },
           through: :blog_posts,
           source: :post
end

# app/models/blog_post.rb (join table model)
class BlogPost < ApplicationRecord
  belongs_to :post
  belongs_to :blog
end

# app/models/post.rb
class Post < ApplicationRecord
  # Target model
end

# blogs table
create_table "blogs" do |t|
  t.integer "position"
  t.string "name"
  # ... other columns
end

# blog_posts table (join table)
create_table "blog_posts" do |t|
  t.integer "blog_id"
  t.integer "post_id"
  # ... other columns
end

# posts table
create_table "posts" do |t|
  t.string "status"
  t.string "title"
  # ... other columns
end

# Create test data
blog = Blog.create!(position: 1, name: 'My Blog')
post = Post.create!(status: 'published', title: 'Hello World')
blog.blog_posts.create!(post: post)

# Load blog_posts (triggers ThroughAssociationPatch logic)
blog.blog_posts.each { |bp| bp.update!(archived: true) }

# Now when Goldiloader tries to eager load published_posts, it fails
blog.published_posts.to_a
# => PG::UndefinedTable: ERROR:  missing FROM-clause entry for table "blogs"

It looks like this bug was triggered by the addition of the ThroughAssociationPatch module added in Goldiloader 4.2.1 (present in 5.4.0 at lib/goldiloader/active_record_patches.rb:180-198):

module ThroughAssociationPatch
  def auto_include?
    through_association = owner.association(through_reflection.name)
    auto_include_self = super

    return false unless auto_include_self
    return true if through_association.auto_include?

    # This logic allows auto-including after through association is loaded
    through_association.loaded? && Array.wrap(through_association.target).none? do |record|
      record.new_record? || record.changed? || record.destroyed?
    end
  end
end

When this patch enables auto-including for the :published_posts association, Goldiloader/Rails' preloader constructs a query that incorrectly includes the source model's (Blog) default_scope ORDER BY clause in the target model's (Post) query:

-- expected SQL
SELECT "posts".*
FROM "posts"
INNER JOIN "blog_posts" ON ...
WHERE "posts"."status" = 'published';

-- actual SQL
SELECT "posts".*
FROM "posts"
WHERE "posts"."id" IN (1, 2)
ORDER BY "blogs"."position" ASC  -- bug: blogs not in FROM clause

We attempted the following workarounds, but none of them prevented the issue:

• auto_include(false) on the association scope - still tries to eager load
• Table-qualified WHERE clause: where(posts: { status: 'published' }) - same error
• Scope with arity > 0: -> (record) { where(status: 'published') } - same error
• unscope(:order) in association scope - default_scope still applied
• auto_include(false)` on the join table association - same error

Regarding a possible fix, the through association query should not inherit the source model's default_scope when constructing queries for the target model. The ORDER BY clause from Blog.default_scope should not appear in queries against the posts table. Instead, the default_scope from the source model (Blog) leaks into the target model's (Post) query, causing invalid SQL with table references in ORDER BY clauses for tables not in the FROM clause.

The ThroughAssociationPatch should probably ensure that when enabling auto-include for through associations, the default_scope from the source model is not applied to the target model's query. That way, the preloader should only apply scopes defined directly on the association and the target model's own default_scope.

Thanks in advance, and hopefully this is clear. We've been working on a patch for this as well, which I'll open up separately.

[oehlschl]

Worth noting again that auto_include(false) on these associations unfortunately doesn't fix the issue, so without a patch or without disabling Goldiloader more broadly, this issue will continue.

Also notably, this is arguably a Rails bug triggered by Goldiloader. auto_include(false) doesn't solve the bug because it only disables Goldiloader's automatic eager loading optimization, but the scope leakage seemingly occurs deeper in Rails' ActiveRecord::Associations::Preloader when it constructs SQL queries for through associations. The ThroughAssociationPatch enables auto-including associations after their through association has been loaded, which triggers Rails' preloader to batch-load peer associations; during this batching, Rails incorrectly reuses scopes from one through association in queries for other through associations. This means the bug is actually likely a Rails issue with how it handles multiple scoped has_many :through associations via the same join table when a default_scope is present, and Goldiloader just happens to trigger this problematic code path.

We've only tested on Rails / ActiveRecord 7.1, and haven't yet verified whether this issue still exists in new Rails versions.