Support other database integrations like Oracle for dagster storage #20295
amaralvieira
started this conversation in
Ideas
Replies: 0 comments
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Uh oh!
There was an error while loading. Please reload this page.
-
Hi! Right now we can use MySQL and PostgreSQL for dagster storage (events, runs, scheduler).
With support for other databases (in my case I would be interested in Oracle), teams could more rapidly develop production-grade pipelines without the limitations of SQLite: all too frequent "database locked" errors, no retries... An example: when developing locally I have to limit the dbt threads otherwise the dagster runs often fail on SQLite concurrent writes, while the dbt runs have succeeded.
It is probably a stupid question, but would a more generic dagster db adapter work, even with any caveats of not being optimised for any particular db? It is already sqlalchemy under the hood.
I would be happy to test, especially for anything Oracle related.
Beta Was this translation helpful? Give feedback.
All reactions