[Kafka] DatabaseTransientException: "_kafka_msg_id" conflicts with system column pattern

[amotl]

Problem

When evaluating the CrateDB destination adapter with the Kafka source, ingestr is failing on the venerable InvalidColumnNameException ¹.

dlt.pipeline.exceptions.PipelineStepFailed: Pipeline execution failed at stage load when processing package 1751729700.3065002 with exception:

<class 'dlt.destinations.exceptions.DatabaseTransientException'>
"_kafka_msg_id" conflicts with system column pattern
CONTEXT:  io.crate.metadata.ColumnIdent.validateColumnName(ColumnIdent.java:365)
io.crate.metadata.ColumnIdent.validForCreate(ColumnIdent.java:581)
io.crate.metadata.ColumnIdent.fromNameSafe(ColumnIdent.java:224)
[...]

-- https://github.com/crate/cratedb-examples/actions/runs/16105196388/job/45439612842?pr=1013#step:3:3789

References

• SQL DDL error InvalidColumnNameException["_foo" conflicts with system column pattern] crate/crate#15161
• Ingestr: Add runnable example loading from Kafka crate/cratedb-examples#1013

Footnotes

1. This time, that label (InvalidColumnNameException) does NOT appear within the error traceback. ↩