Slow initialization after updating to v0.90.0 #3381
Description
With the recent updates of Oceananigans, I noticed an unusual behavior of the model that often leads to much slower initialization.
Below is a minimum working example that demonstrates the issue:
using Oceananigans
using Oceananigans.Units
grid = RectilinearGrid(CPU(),
size = (3, 3, 3),
extent = (1, 1, 1),
topology = (Periodic, Bounded, Bounded))
model = HydrostaticFreeSurfaceModel(; grid)
simulation = Simulation(model, Δt=20minutes, stop_time=20days)
u, v, w = model.velocities
ζ = ∂x(v) - ∂y(u)
fields_slice = Dict("u" => u, "v" => v, "w" => w, "ζ" => ζ)
simulation.output_writers[:top] = NetCDFOutputWriter(model, fields_slice;
filename = "mwe.nc",
schedule = TimeInterval(0.5day),
overwrite_existing = true,
indices = (:, :, grid.Nz))
run!(simulation)
Running this code with Julia 1.9.3 and Oceananigans v0.90.0 gives an initialization time of ~ 15 minutes, much longer than common values of a few seconds. The same issue also appears on GPU.
This speed issue disappears either when ζ = ∂x(v) - ∂y(u) is replaced with ζ = Field(∂x(v) - ∂y(u)), or when ζ is the only variable in fields_slice. However, as pointed out by @tomchor, wrapping outputs in Field() tends to waste more memory (a very precious resource on the GPU), so it may be worthwhile to investigate further why this happens.