The great restructuring, episode 15: the monorepo strikes back, single SciML version

[ChrisRackauckas]

As SciML has grown, so has its structural needs. Back in 2017, we realized that DifferentialEquations.jl was too large and so it needed to be split into component packages for OrdinaryDiffEq, StochasticDiffEq, etc. in order to allow for loading only a part of the system as DiffEq itself was too big. In 2021, we expanded SciML to have LinearSolve, NonlinearSolve, etc. as separate interfaces so they can be independently used, and as such independent packages. In 2024 we split many of the solver packages into independent packages like OrdinaryDiffEqTsit5, in order to accomodate users who only wanted lean dependencies to simple solvers. And now we have hundreds of packages across hundreds of repos, though many of the subpackages are in the same repo.

If we had started this system from scratch again, I don't think we'd have so many repos. With the subrepo infrastructure of modern Julia, keeping OrdinaryDiffEq and StochasticDiffEq together would have been nice. In fact, DelayDiffEq.jl touches many of the OrdinaryDiffEqCore internals, so it should be versioned together with OrdinaryDiffEq. This leads to issues like JuliaLang/julia#55516 being asked of the compiler team.

The core issue here is that the module structure does not necessarily reflect the public/private boundaries, but rather the module system is designed for separate loading, separate compilation, conditional dependencies, and ultimately handling startup/latency issues. This means that it really does not make sense for OrdinaryDiffEqTsit5 and OrdinaryDiffEqCore to have different versions, since they really should be handled in lock-step, and any versioning that isn't matching is always suspect. This leads to odd issues with bumping, and nasty resolution of manifests. The issue is that the semver system is designed around package boundaries being built around public interfaces with announcing public breakage, but this simply does not work out when the common breakage is non-public internals simply because the package boundary is at some internal function, again not because it's a generally good idea for the packages but instead because it's a requirement to achieve lower startup times.

So what can we do? The proposal is not so simple, but it can work. Instead of waiting on a fix for JuliaLang/julia#55516, we can do some clever tricks on the tools that we now have in Julia 2025 land in order to pull this off. It would look like this:

• We can have a single basic repository that contains all of the core code. For example, OrdinaryDiffEq contains what is now the OrdinaryDiffEqCore code
• We then make all of the add on code into extensions. For example, OrdinaryDiffEqTsit5 would effectively be an empty package, that only exists to have a Project.toml for declaring new dependencies, and an OrdinaryDiffEqOrdinaryDiffEqTsit5 extension to OrdinaryDiffEq would be created to trigger on using OrdinaryDiffEqTsit5 that actually loads the extension code.
• Importantly, this means that the code lives in OrdinaryDiffEq.jl, and thus the solver code only has one version, the OrdinaryDiffEq version. All solver code is then single versioned and updated in lock step. The only reason to bump the version on the pseudo packages is to bump dependency versions. Now because the code does not actually live in OrdinaryDiffEqTsit5, the only way to do a code update is to first make a version of OrdinaryDiffEqTsit5 that is a major bump with the new dependency versions, and then update OrdinaryDiffEq.jl to allow the new OrdinaryDiffEqTsit5 major. This means that every release of the psuedo packages is a considered a breaking release, but the only times they need a release are for compat bumps.

With this, the user code would look like:

using OrdinaryDiffEq, OrdinaryDiffEqTsit5
function lorenz!(du, u, p, t)
    du[1] = 10.0(u[2] - u[1])
    du[2] = u[1] * (28.0 - u[3]) - u[2]
    du[3] = u[1] * u[2] - (8 / 3) * u[3]
end
u0 = [1.0; 0.0; 0.0]
tspan = (0.0, 100.0)
prob = ODEProblem(lorenz!, u0, tspan)
sol = solve(prob, Tsit5())
using Plots;
plot(sol, idxs = (1, 2, 3))

Note that using OrdinaryDiffEqTsit5 would be a requirement, as OrdinaryDiffEq would no longer trigger any solvers. If this isn't nice, we could have the single version package be OrdinaryDiffEqCore, with a higher level OrdinaryDiffEq that just uses a few solvers.

Small Questions

1. The issue "Note that using OrdinaryDiffEqTsit5 would be a requirement, as OrdinaryDiffEq would no longer trigger any solvers. If this isn't nice, we could have the single version package be OrdinaryDiffEqCore, with a higher level OrdinaryDiffEq that just uses a few solvers. " - Is that to janky?
2. Any potential issues not considered?

Big Question

If we go to monorepo, what's in and what's out? The advantage of having more things in a single repo is clear. Most of the issues around downstream testing, keeping package versions together, etc. are all gone. You know your code is good to go if it passes tests in this repo because that would have "everything".

However, what is everything? Should we have the following:

• DifferentialEquations
• NonlinearSolve
• LinearSolve
• ...

Or if we're going to do this, should we just have a single SciML?

using SciML, OrdinaryDiffEqTsit5
function lorenz!(du, u, p, t)
    du[1] = 10.0(u[2] - u[1])
    du[2] = u[1] * (28.0 - u[3]) - u[2]
    du[3] = u[1] * u[2] - (8 / 3) * u[3]
end
u0 = [1.0; 0.0; 0.0]
tspan = (0.0, 100.0)
prob = ODEProblem(lorenz!, u0, tspan)
sol = solve(prob, Tsit5())
using Plots;
plot(sol, idxs = (1, 2, 3))

Then you can just ask, "what version is your SciML?"

I think that would actually be really nice because splitting the interface between SciMLOperators, SciMLBase, DiffEqBase, NonlinearSolveBase, OptimizationBase, OrdinaryDiffEqCore, etc. are also somewhat arbitrary distinctions and moving code at the interface level there has always been a multi-repo mess due to the aritificiality of the split and release process w.r.t. semver at this level. Additionally, SciMLSensitivity is a package adding sensitivity analysis to all solvers, even NonlinearSolve.jl, so it would neatly fit as an extension to all of the packages. If that's the case, is the core package here just SciMLBase, and everything else is an extension to SciMLBase?

That said, how far do we go? Is ModelingToolkit in there? Is Catalyst in there? Symbolics.jl and SymbolicUtils.jl? Do we then restructure the whole SciML documentation as a single Vitepress doc? Or build many different Documenter docs from one repo?

Some CI questions

1. Testing the subpackages: do we put the tests all in the main repo, or associate them with the psuedo packages? That way users could trigger subsets of tests?
2. Could the tests know/understand the dependency graph, so say a change to SciMLBase itself triggers all tests, while changing something in OrdinaryDiffEqCore only triggers the OrdinaryDiffEq tests and other downstream of that? That would be pretty essential for managing the growth of the CI budget, since more tests is more costs.
3. Are there downsides with precompilation, download sizes, and system image building to consider?
4. How will the VS Code Language Server feel about this?

Conclusion

This is going to be a lot of work, so I'm looking for comments before commencing.

@devmotion @oscardssmith @isaacsas @TorkelE @thazhemadam @asinghvi17

[KristofferC]

A drawback to extensions is that you cannot really access symbols defined in them. So they have to work pretty much purely by method extensions. Does all the code that is desired to be put into extensions do that here?

In the code given as:

With this, the user code would look like:
sol = solve(prob, Tsit5())

does the Tsit5 symbol come from OrdinaryDiffEqTsit5, which is then dispatched on in the extension? What about solvers that need more complicated constructors, you then start to need to put more code into the packages that are supposed to be empty, which works against the idea of having all code in the base package.

Note that using OrdinaryDiffEqTsit5 would be a requirement, as OrdinaryDiffEq would no longer trigger any solvers.

I think the requirement is so that you get access to the Tsit5 symbol, no?

[ranocha]

What about packages that extend SciML packages but are not part of SciML? We cannot depend on package extensions, can we?

Moreover, I would have expected Tsit5() to be defined in OrdinaryDiffeqTsit5.jl. What if we change the algorithm interface? This will still require synchronizing versions across packages.
Edit: Just saw that a related comment was posted above.

[TorkelE]

Keeping a look at this, but given the magnitude of things I will have to think through things before I can contribute any useful opinion.

From experience, working with extensions is a proper pain. E.g. If I have function f in CatalystBifurcationKitExt, I cannot do CatalystBifurcationKitExt.f or Catalyst.f or something to reach it. I do agree that coupling versions would make things a lot easier though, and maybe given the niche use case of extensions it can work out.

I guess we have no idea when a solution to JuliaLang/julia#55516 could drop?

[hodgensc]

regarding "Small question" #1: Say I was still in the troubleshooting phase of a project and getting my infrastructure in place and I'm trying out different solvers to see which is most appropriate to use. Would the workflow in this suggestion be to 1) test with OrdinaryDiffEqCore because it has a bunch of the solvers packaged in, then 2) when I figure out which solver I need I would switch to OrdinaryDiffEq and import only that solver?

[ChrisRackauckas]

does the Tsit5 symbol come from OrdinaryDiffEqTsit5, which is then dispatched on in the extension? What about solvers that need more complicated constructors, you then start to need to put more code into the packages that are supposed to be empty, which works against the idea of having all code in the base package.

Well there's two different choices that could be made here.

1. It's still defined in the core package, in which case we'd need to manually guard against solve(prob, Tsit5()) not having loaded the extension. The downside here is that these manual load checks will be nasty to get right and have to be manually maintained (it's somewhat what's already in place in LinearSolve.jl)
2. It's defined in the extension, in which case yes it's not just an empty piece of code. The nice thing here is that would handle some dependency issues, since yes some algorithms may have a default argument that depends on a dependency (like threads = Threads.PolyesterThreads() or something of the sort), and so it needs to be in the subpackage in order to not take the dependency.

I would be inclined to go with 2 for a mostly different reason than those, which is that we know large files and just loading large amounts of code is one of the biggest latency issues right now, and that's not necessarily going to get any smaller. So having a giant file of algorithm struct constructors would be something that would add up in terms of load time. IIRC the OrdinaryDiffEq algorithms page was like 0.15 seconds itself, so all solvers together could be a floor of like 0.4 seconds to loading (in its current implementation) just from the algorithm structs. So, putting those into the package shims could be the right thing to do not just for handling dependencies but also to make the loading floor lower.

[MasonProtter]

I think I'm generally in favour of this. One thing I do want to mention though is that on this issue:

Small Questions

1. The issue "Note that using OrdinaryDiffEqTsit5 would be a requirement, as OrdinaryDiffEq would no longer trigger any solvers. If this isn't nice, we could have the single version package be OrdinaryDiffEqCore, with a higher level OrdinaryDiffEq that just uses a few solvers. " - Is that to janky?

Why not keep the same behaviour we currently have? using OrdinaryDiffEqCore loads no solvers, and using OrdinaryDiffEq loads the (ordinary) kitchen sink.

---

Beyond that, I want to flag the fact that there's still currently a lot of UX issues with extensions, e.g. various things are broken with them in 1.10 (crytic errors about being unable to merge manifests when running test suites for example), and most users don't know how to actually access an extension module (i.e. Base.get_extension is kinda wonky to use).

I don't think any of that are reasons to not do it, but they are maybe reasons to be cautious about this.

I would suggest perhaps starting with a smaller set of consolidations. For instance, we could turn OrdinaryDiffEq back into a single-version monorepo using the techniques described above, and see how that goes. Or we could try this as a project to separate out the solvers in e.g. StochasticDiffEq.

Doing this on a smaller scale might help us find sharp edges and work out solutions to them before we go all in and consoldiate everything into one mega-repo.

[ChrisRackauckas]

I guess we have no idea when a solution to JuliaLang/julia#55516 could drop?

I was told today that it's unlikely to happen in v1.13, and so probably not within the next ~2 years. I don't think we can keep this version hell going on for that long without having a riot, so given that's not happening with new Base Julia tooling, we need to figure out a solution with duck tape and bubble gum.

That said, one non extension way to do this that was proposed is a purely CI solution. What @oscardssmith had sent me last night is that could do this through a CI-based lock step versioning system. Quoting:

1. New compat bot that when triggered makes a commit that bumps the versions of all packages in a repo and and bumps all compats so that the compats are in lockstep.
2. JuliaRegister bot support for bulk registration. (likey @JuliaRegistrator register_all() or something similar)
3. Tagbot and Dependabot support for multiple projects in a repo

This approach has a few advantages over introducing subpackages:

1. We can use this ~now rather than in ~2 years when we are comfortable lower bounding on 1.13
2. This isn't a Pkg/language change so if it doesn't work out, we haven't committed to a new feature

[JuliaRegistrator]

Error while trying to register: Action not recognized: register_all

[ChrisRackauckas]

Go away registrator, not now. Read the room.

[JuliaRegistrator]

Error while trying to register: Action not recognized: register_all

[ChrisRackauckas]

I would suggest perhaps starting with a smaller set of consolidations. For instance, we could turn OrdinaryDiffEq back into a single-version monorepo using the techniques described above, and see how that goes. Or we could try this as a project to separate out the solvers in e.g. StochasticDiffEq.

Doing this on a smaller scale might help us find sharp edges and work out solutions to them before we go all in and consoldiate everything into one mega-repo.

That's definitely the plan. I think we'd do just the DiffEq stuff first and see how that goes. But I'm curious then for the next step if people would prefer NonlinearSolve separate or together with it. At least I know I want all of DiffEq together, since DelayDiffEq makes no sense in a different repo and that has been an issue for at least 7 years at this point.

[ChrisRackauckas]

#1: Say I was still in the troubleshooting phase of a project and getting my infrastructure in place and I'm trying out different solvers to see which is most appropriate to use. Would the workflow in this suggestion be to 1) test with OrdinaryDiffEqCore because it has a bunch of the solvers packaged in, then 2) when I figure out which solver I need I would switch to OrdinaryDiffEq and import only that solver?

Somewhat other way around. using OrdinaryDiffEq as a top level would give you all of the solvers. You REPL play with that. And then when you realize you just want Tsit5, you do using OrdinaryDiffEqCore, OrdinaryDiffEqTsit5 in the package and now it just has the one dependency.

[hodgensc]

#1: Say I was still in the troubleshooting phase of a project and getting my infrastructure in place and I'm trying out different solvers to see which is most appropriate to use. Would the workflow in this suggestion be to 1) test with OrdinaryDiffEqCore because it has a bunch of the solvers packaged in, then 2) when I figure out which solver I need I would switch to OrdinaryDiffEq and import only that solver?

Somewhat other way around. using OrdinaryDiffEq as a top level would give you all of the solvers. You REPL play with that. And then when you realize you just want Tsit5, you do using OrdinaryDiffEqCore, OrdinaryDiffEqTsit5 in the package and now it just has the one dependency.

Ah! Okay, that makes sense and seems manageable for someone like me. Thanks.

[ChrisRackauckas]

What about packages that extend SciML packages but are not part of SciML? We cannot depend on package extensions, can we?

Yes, those couldn't be extensions. They would just have to be separate packages as always, with the same warts as before. But normally external packages should just be relying on public interfaces. Really the big issues are for example the solver packages that rely on the exact implementation of ODEIntegrator and such.

Though https://github.com/NumericalMathematics/PositiveIntegrators.jl is this kind of edge case that's using OrdinaryDiffEqCore internals... I'd prefer to just get that into the system at that point as its hard to ever make it fully robust if separate. We could at least keep the same downstream test infra for this kind of thing.

[gdalle]

One thing I want to draw attention to with a monorepo approach is testing. I have similar problems with DI, where you could argue that I run 14 downstream tests every time I bat an eyelid. Now imagine if every docstring modification triggered a full run of every single SciML test ever written. This would be nightmarish for user experience and energy consumption alike.
If we go down that road, we'll need a clever way to be selective on the test suites that run. It could probably be done through a mixture of PR labels and automatic GitHub Actions magic, for instance to detect which files have been modified and decide which subpackages should be tested as a result. But even for DI alone it's far from easy to set up.

As a side note, if we're discussing custody of the kids, I'd like to bring ADTypes into the DI repo, for exactly the same lockstep versioning reasons. These days, it is DI which defines the semantics of ADTypes, so it makes no sense that we're able to add something to ADTypes without having it immediately implemented in DI.