SQLite vs PostgreSQL for storage

[dagsir[bot]]

I see that runs history are stored in a history directory in . db files in DAGSTER_HOME. Can these be configured to persist to another database? Like having a PostgreSQL backend.

Are there any advantages of the others over the default aside the use of the assets page? Eg. Some DBs are good for multiprocessing or simultaneous running and scheduling compared to the default SQLite

---

The question was originally asked in Dagster Slack.

[yuhan]

Yes, you can use PostgreSQL - how to configure it on your instance is described here: https://docs.dagster.io/deployment/dagster-instance#postgres-storage

In our experience PostgreSQL is much better for performance (esp once you start using multiprocessing or run many pipelines at once)

[digitalghost-dev]

Updated like in case anyone came here from Google: https://docs.dagster.io/deployment/oss/oss-instance-configuration#dagster-storage

[rtol5]

hi @yuhan – if i created a Dagster environment using SQLite and i want to migrate to Postgres, is there a way to do so? would simply porting the tables/data over to a Postgres db and pointing Dagster to it be enough?

[SpencerSharp]

I have the same question and wondering if you ever figured this out.

[tspencerdata]

I also have this question