Skip to content

Column names getting double-quoted when importing from sqlite using --with "quote identifiers" #1651

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
funkypenguin opened this issue Mar 19, 2025 · 0 comments

Comments

@funkypenguin
Copy link

Hi,

Beginner here - I'm trying to automate the importing of a Radarr/Sonarr SQLite database into Postgres 17 with pgloader v3.6.9 on alpine edge.

/app $ pgloader -V
pgloader version "3.6.9"
compiled with SBCL 2.3.6
/app $

I'm running:

pgloader --with "quote identifiers" --with "data only" /config/radarr.db "postgresql://radarr:radarr@localhost/radarr_main"

But I see the following failure:

KABOOM!
UNDEFINED-COLUMN: Database error 42703: column ""Id"" of relation "MovieFiles" does not exist
CONTEXT: PL/pgSQL function inline_code_block line 6 at FOR over SELECT rows
QUERY:
DO $$
DECLARE
  n integer := 0;
  r record;
BEGIN
  FOR r in
       SELECT 'select '
               || trim(trailing ')'
                  from replace(pg_get_expr(d.adbin, d.adrelid),
                               'nextval', 'setval'))
               || ', (select greatest(max(' || quote_ident(a.attname) || '), (select seqmin from pg_sequence where seqrelid = ('''
               || pg_get_serial_sequence(quote_ident(nspname) || '.' || quote_ident(relname), quote_ident(a.attname)) || ''')::regclass limit 1), 1) from only '
               || quote_ident(nspname) || '.' || quote_ident(relname) || '));' as sql
         FROM pg_class c
              JOIN pg_namespace n on n.oid = c.relnamespace
              JOIN pg_attribute a on a.attrelid = c.oid
              JOIN pg_attrdef d on d.adrelid = a.attrelid
                                 and d.adnum = a.attnum
                                 and a.atthasdef
        WHERE relkind = 'r' and a.attnum > 0
              and pg_get_expr(d.adbin, d.adrelid) ~ '^nextval'
              and c.oid in (select oid from reloids)
  LOOP
    n := n + 1;
    EXECUTE r.sql;
  END LOOP;

  PERFORM pg_notify('seqs', n::text);
END;
$$;

I have confirmed that the column Id does exist in the MovieFiles table

I notice in the output that the column name (""Id"" ) is double-quoted - could this be an error introduced by using --with "quote identifiers" ?

Without --with "quote identifiers", I only get this far:

/app $ pgloader --with "data only" /config/radarr.db "postgresql://radarr:radarr@localhost/radarr_main"
2025-03-19T11:46:07.014999Z LOG pgloader version "3.6.9"
2025-03-19T11:46:07.016999Z LOG Data errors in '/tmp/pgloader/'
2025-03-19T11:46:07.077998Z LOG Migrating from #<SQLITE-CONNECTION sqlite:///config/radarr.db {1005C81D13}>
2025-03-19T11:46:07.077998Z LOG Migrating into #<PGSQL-CONNECTION pgsql://radarr@localhost:5432/radarr_main {1005F0F183}>
KABOOM!
SIMPLE-ERROR: pgloader failed to find anything in schema "public" in target catalog.
An unhandled error condition has been signalled:
   pgloader failed to find anything in schema "public" in target catalog.




What I am doing here?

pgloader failed to find anything in schema "public" in target catalog.

/app $

I also see lots of these, which are warnings but don't stop processing:

2025-03-19T11:43:53.330994Z WARNING Source column "public"."NamingConfig"."\"ReplaceIllegalCharacters\"" is casted to type "bigint" which is not the same as "boolean", the type of current target database column "public"."NamingConfig"."ReplaceIllegalCharacters".
2025-03-19T11:43:53.330994Z WARNING Source column "public"."NamingConfig"."\"ColonReplacementFormat\"" is casted to type "bigint" which is not the same as "integer", the type of current target database column "public"."NamingConfig"."ColonReplacementFormat".
2025-03-19T11:43:53.330994Z WARNING Source column "public"."NamingConfig"."\"RenameMovies\"" is casted to type "bigint" which is not the same as "boolean", the type of current target database column "public"."NamingConfig"."RenameMovies".
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant