Skip to content

Column names getting double-quoted when importing from sqlite using --with "quote identifiers" #1651

Open
@funkypenguin

Description

@funkypenguin

Hi,

Beginner here - I'm trying to automate the importing of a Radarr/Sonarr SQLite database into Postgres 17 with pgloader v3.6.9 on alpine edge.

/app $ pgloader -V
pgloader version "3.6.9"
compiled with SBCL 2.3.6
/app $

I'm running:

pgloader --with "quote identifiers" --with "data only" /config/radarr.db "postgresql://radarr:radarr@localhost/radarr_main"

But I see the following failure:

KABOOM!
UNDEFINED-COLUMN: Database error 42703: column ""Id"" of relation "MovieFiles" does not exist
CONTEXT: PL/pgSQL function inline_code_block line 6 at FOR over SELECT rows
QUERY:
DO $$
DECLARE
  n integer := 0;
  r record;
BEGIN
  FOR r in
       SELECT 'select '
               || trim(trailing ')'
                  from replace(pg_get_expr(d.adbin, d.adrelid),
                               'nextval', 'setval'))
               || ', (select greatest(max(' || quote_ident(a.attname) || '), (select seqmin from pg_sequence where seqrelid = ('''
               || pg_get_serial_sequence(quote_ident(nspname) || '.' || quote_ident(relname), quote_ident(a.attname)) || ''')::regclass limit 1), 1) from only '
               || quote_ident(nspname) || '.' || quote_ident(relname) || '));' as sql
         FROM pg_class c
              JOIN pg_namespace n on n.oid = c.relnamespace
              JOIN pg_attribute a on a.attrelid = c.oid
              JOIN pg_attrdef d on d.adrelid = a.attrelid
                                 and d.adnum = a.attnum
                                 and a.atthasdef
        WHERE relkind = 'r' and a.attnum > 0
              and pg_get_expr(d.adbin, d.adrelid) ~ '^nextval'
              and c.oid in (select oid from reloids)
  LOOP
    n := n + 1;
    EXECUTE r.sql;
  END LOOP;

  PERFORM pg_notify('seqs', n::text);
END;
$$;

I have confirmed that the column Id does exist in the MovieFiles table

I notice in the output that the column name (""Id"" ) is double-quoted - could this be an error introduced by using --with "quote identifiers" ?

Without --with "quote identifiers", I only get this far:

/app $ pgloader --with "data only" /config/radarr.db "postgresql://radarr:radarr@localhost/radarr_main"
2025-03-19T11:46:07.014999Z LOG pgloader version "3.6.9"
2025-03-19T11:46:07.016999Z LOG Data errors in '/tmp/pgloader/'
2025-03-19T11:46:07.077998Z LOG Migrating from #<SQLITE-CONNECTION sqlite:///config/radarr.db {1005C81D13}>
2025-03-19T11:46:07.077998Z LOG Migrating into #<PGSQL-CONNECTION pgsql://radarr@localhost:5432/radarr_main {1005F0F183}>
KABOOM!
SIMPLE-ERROR: pgloader failed to find anything in schema "public" in target catalog.
An unhandled error condition has been signalled:
   pgloader failed to find anything in schema "public" in target catalog.




What I am doing here?

pgloader failed to find anything in schema "public" in target catalog.

/app $

I also see lots of these, which are warnings but don't stop processing:

2025-03-19T11:43:53.330994Z WARNING Source column "public"."NamingConfig"."\"ReplaceIllegalCharacters\"" is casted to type "bigint" which is not the same as "boolean", the type of current target database column "public"."NamingConfig"."ReplaceIllegalCharacters".
2025-03-19T11:43:53.330994Z WARNING Source column "public"."NamingConfig"."\"ColonReplacementFormat\"" is casted to type "bigint" which is not the same as "integer", the type of current target database column "public"."NamingConfig"."ColonReplacementFormat".
2025-03-19T11:43:53.330994Z WARNING Source column "public"."NamingConfig"."\"RenameMovies\"" is casted to type "bigint" which is not the same as "boolean", the type of current target database column "public"."NamingConfig"."RenameMovies".

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions