Using Alembic without ORM at a decent scale

[bobsbarrels]

Not long ago I joined a small but kind of old team that move a fair amount of data around, they have hundreds of schemas and maybe few thousand tables as they move data through a pipeline. It's big not huge. I come from a back ground that was very ORM heavy, mostly Django, but non-zero SqlAlchemy.

This team only writes Alembic migrations basically from scratch*. Probably about ~1-per-day . Then they make custom classes to map to those migrations (not using an ORM of any kind). Some of the migrations/tables are pretty big but nothing out of control, they do make it work, but it's a non-trivial number of engineering hours. *They do have their own very primitive custom partially-baked migrations/class generator for part of the db, but it was made by a previous generation of engineers from the late-teens and everything generated out of it requires so much rework it isn't worth discussing.

As someone who expects to only climb in to migrations in extreme circumstances, being able to trust my ORM in all but the most exotic cases, this is usage is wild to me, but everyone here is telling me it's fine.

Maybe this is normal outside of Django world?

Is this a "normal" use case? Do Alembic users often just "roll their own" migrations?

[CaselIT]

Hi,

I would suggest you take a look at https://alembic.sqlalchemy.org/en/latest/autogenerate.html
To work with it alembic does not need you to use the sqlalchemy orm, but it uses the sqlalchemy tables and columns that underpin the sqlalchemy orm classes.

Autogenerate does not claim to be perfect, manual editing is expected to be required at times, but it should provide a good starting point.

Of course if the project is not using tables or the table definition is partial and does not fully represent the db then I don't think autogenerate will be of much use.

[elena]

Hi,

Thank you for the response.

I've tried out autogenerate already and if I can get the value proposition through to the team hope to implement classes from it.

I'm actually interested if there's a reason why people would use just the migrations?

Is this a pattern that anyone is aware of?

[CaselIT]

I'm not sure what you are asking.

through to the team hope to implement classes from it.

I don't follow what classes you are referring here.

why people would use just the migrations?

I'm not sure what your mean here. alembic is a tool to apply schema changes to a database.