[Bug] dbt docs generate: can't decode byte 0xfc in position 6

[cedrikschueler]

Is there an existing issue for this?

• I have searched the existing issues

Current Behavior

I am currently generating source.yml files from a Python script, which basically queries ALL_TAB_COLS from a source_db.
This is done for four different schemas on the source_db.

However, for one (always the same) schema, I get the following error when running dbt docs generate:

Runtime Error
  'utf-8' codec can't decode byte 0xfc in position 6: invalid start byte

'0xfc' appears to be the German 'ü'. This character is not present in my entire project folder, but might be somewhere on the database.

The strange behavior is that I can successfully create the source descriptions (one file per table) for all other schemas, and if I delete a random part of the source files for my failing schema, it works at some point.

I am happy for any advise for further debugging or solutions :)

Expected Behavior

No response

Steps To Reproduce

No response

Relevant log output using --debug flag enabled

Environment

- OS: Windows 11
- Python: 3.11.9
- dbt: 1.9.9

What Oracle database version are you using dbt with?

19

Additional Context

No response

[cjbj]

If your script failing when querying ALL_TAB_COLS, or when you eventually query a user table? Review https://python-oracledb.readthedocs.io/en/latest/user_guide/sql_execution.html#querying-corrupt-data

[cedrikschueler]

My python script successfully creates all sources.yml files. Thus it also queries the ALL_TAB_COLS successfully (using sqlalchemy engine and pd.read_sql).

My folder structure is like this:

schema1/
tab1_src.yml
tab2_src.yml
schema2/
tab3.yml

etc.

All those yml files seem to be valid and saved in utf-8 encoding.
The bug occurs when I run dbt docs generate.

Example workflow:

1. python my_script.py (successfully runs, queries ALL_TAB_COLS and creates all those yml files)
2. dbt docs generate (fails with abovementioned error)
3. Fix attempt: create all files except those for schema1 --> dbt docs generate runs successfully
4. Reverse attempt: create only those files for schema1 --> dbt docs generate fails again
5. Delete an arbitrary part of the yml files in schema1 --> fails

[aosingh]

@cedrikschueler

Could you share dbt debug logs ?

dbt --debug docs generate

[cedrikschueler]

I cannot share the whole output.

But an anonymous example:

with tables as (
...
),
columns as (
...
)
from tables
< SOME JOINS HERE - ALL LOOK LEGIT >
        where (<TABLE SCHEMA MATCHES ANY OF MY SCHEMAS>)
        order by
        tables.table_schema,
        tables.table_name,
        ordinal_position

Opening a new connection, currently in state init
SQL status: OK in 10.000 seconds
oracle adapter: Rolling back transaction.
On MY_DATABASE.information_schema: Close
Encountered an error while generating catalog: Runtime Error 'utf-8' codec can't decode byte 0xfc in position 6: invalid start byte

The only thing looking suspicious is the SQL status OK after 10 seconds, which looked like a timeout for me first. But on different runs it's sometimes also 6.000 seconds. Nevertheless it's always an X.000 number.