Skip to content

New regional example for the ACC #142

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Draft
wants to merge 43 commits into
base: main
Choose a base branch
from
Draft
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
43 commits
Select commit Hold shift + click to select a range
aeae520
creating new example for the ACC
Aug 14, 2024
705f078
Update acc_regional_simulation.jl
francispoulin Aug 15, 2024
bc31849
Update acc_regional_simulation.jl
francispoulin Aug 20, 2024
67c3eb7
Merge branch 'main' into fjp/acc_regional_model
simone-silvestri Dec 10, 2024
06ab5d4
Merge branch 'main' into fjp/acc_regional_model
simone-silvestri Dec 18, 2024
7741ea7
updated script
Dec 18, 2024
a398d0c
add different masking
simone-silvestri Dec 18, 2024
e377edf
updating script to get further
Dec 18, 2024
ef9d130
fix node problem
Dec 18, 2024
5e95104
fix phinodes
francispoulin Dec 18, 2024
9abbc32
add grid to varphinodes
francispoulin Dec 18, 2024
55f667b
fix varphinode for real
Dec 18, 2024
3f9dd7d
a working version on a gpu
Dec 19, 2024
47ec55e
change time step to 10 minutes
Dec 20, 2024
a6989e8
cleaning up callbacks
Dec 20, 2024
8cc2719
Update examples/acc_regional_simulation.jl
francispoulin Dec 22, 2024
04b24e9
Merge branch 'main' into fjp/acc_regional_model
simone-silvestri Jan 13, 2025
c341fdc
change to the restoring rate and the outputwriter schedule
simone-silvestri Jan 13, 2025
7862bf2
Merge remote-tracking branch 'origin/main' into fjp/acc_regional_model
simone-silvestri May 29, 2025
75d909f
update example
simone-silvestri May 29, 2025
0cb27c5
Merge branch 'main' into fjp/acc_regional_model
simone-silvestri May 29, 2025
4462fe0
run docs
simone-silvestri May 29, 2025
2101f85
run docs
simone-silvestri May 29, 2025
a53438e
Update examples/acc_regional_simulation.jl
simone-silvestri May 29, 2025
9f2a04d
add a dataset
simone-silvestri May 29, 2025
bb228d1
fix restoring
simone-silvestri May 29, 2025
291f381
jld2 writer
simone-silvestri May 29, 2025
7a30c07
remove the interior
simone-silvestri May 29, 2025
feb1e12
correctly compute restoring on GPU
simone-silvestri May 30, 2025
06f479f
pass it
simone-silvestri May 30, 2025
16957c2
Update acc_regional_simulation.jl
francispoulin May 30, 2025
a3ad47e
updating file names
francispoulin Jun 1, 2025
88fa6a6
try with passing the grid
simone-silvestri Jun 2, 2025
ad5bdb6
changing the name of the files to save
Jun 2, 2025
b0c23ea
modifying some names and using the visualization from near global sim…
Jun 3, 2025
43693b9
changing a few names and using the visualization from near global (fo…
francispoulin Jun 3, 2025
2275549
change coupled_simulation to simulation
francispoulin Jun 3, 2025
3f546de
remove two unnecessary lines
francispoulin Jun 3, 2025
2837862
fix alignments
francispoulin Jun 3, 2025
0738b8b
restore backend definition
francispoulin Jun 3, 2025
1fcd5ed
returning to model
francispoulin Jun 3, 2025
c343d9c
changing initial time step to 2*minutes
francispoulin Jun 3, 2025
5cd349c
Merge remote-tracking branch 'origin/main' into fjp/acc_regional_model
simone-silvestri Jun 3, 2025
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
14 changes: 8 additions & 6 deletions docs/make.jl
Original file line number Diff line number Diff line change
Expand Up @@ -17,9 +17,10 @@ const EXAMPLES_DIR = joinpath(@__DIR__, "..", "examples")
const OUTPUT_DIR = joinpath(@__DIR__, "src/literated")

to_be_literated = [
"single_column_os_papa_simulation.jl",
"one_degree_simulation.jl",
"near_global_ocean_simulation.jl"
# "single_column_os_papa_simulation.jl",
# "one_degree_simulation.jl",
# "near_global_ocean_simulation.jl"
"acc_regional_simulation.jl"
]

for file in to_be_literated
Expand All @@ -42,9 +43,10 @@ pages = [
"Home" => "index.md",

"Examples" => [
"Single-column ocean simulation" => "literated/single_column_os_papa_simulation.md",
"One-degree ocean simulation" => "literated/one_degree_simulation.md",
"Near-global ocean simulation" => "literated/near_global_ocean_simulation.md",
# "Single-column ocean simulation" => "literated/single_column_os_papa_simulation.md",
# "One-degree ocean simulation" => "literated/one_degree_simulation.md",
# "Near-global ocean simulation" => "literated/near_global_ocean_simulation.md",
"Regional ACC simulation" => "literated/acc_regional_simulation.md",
],

"Interface fluxes" => "interface_fluxes.md",
Expand Down
224 changes: 224 additions & 0 deletions examples/acc_regional_simulation.jl
Original file line number Diff line number Diff line change
@@ -0,0 +1,224 @@
using Oceananigans
using Oceananigans.Units
using Oceananigans.Grids: φnode
using ClimaOcean
using ClimaOcean.ECCO

using Printf
using CairoMakie
using CFTime
using Dates

arch = GPU()

Nx = 1440
Ny = 400
Nz = 40

z_faces = exponential_z_faces(; Nz, depth=6000)

grid = LatitudeLongitudeGrid(arch;
size = (Nx, Ny, Nz),
halo = (7, 7, 7),
z = z_faces,
latitude = (-80, -20),
longitude = (0, 360))

bottom_height = regrid_bathymetry(grid;
minimum_depth = 10,
interpolation_passes = 7,
major_basins = 1)

grid = ImmersedBoundaryGrid(grid, GridFittedBottom(bottom_height), active_cells_map=true)

start_date = DateTime(1993, 1, 1)
end_date = DateTime(1993, 12, 1)

#
# Restoring force
# φS φN -20
# -------------- | ------------------ | ------------ |
# no restoring 0 linear mask 1 mask = 1 1
#

const φN₁ = -23
const φN₂ = -25
const φS₁ = -78
const φS₂ = -75

@inline northern_mask(φ) = min(max((φ - φN₂) / (φN₁ - φN₂), zero(φ)), one(φ))
@inline southern_mask(φ, z) = ifelse(z > -20,
min(max((φ - φS₂) / (φS₁ - φS₂), zero(φ)), one(φ)),
zero(φ))

@inline function tracer_mask(λ, φ, z, t)
n = northern_mask(φ)
s = southern_mask(φ, z)
return max(s, n)
end

@inline function u_restoring(i, j, k, grid, clock, fields, p)
φ = φnode(i, j, k, grid, Face(), Center(), Center())
return - p.rate * fields.u[i, j, k] * northern_mask(φ)
end

@inline function v_restoring(i, j, k, grid, clock, fields, p)
φ = φnode(i, j, k, grid, Center(), Face(), Center())
return - p.rate * fields.v[i, j, k] * northern_mask(φ)
end

T_meta = Metadata(:temperature; start_date, end_date, dataset = ECCO4Monthly())
S_meta = Metadata(:temperature; start_date, end_date, dataset = ECCO4Monthly())

forcing = (T = DatasetRestoring(T_meta, grid; rate=1/5days, mask=tracer_mask),
S = DatasetRestoring(S_meta, grid; rate=1/5days, mask=tracer_mask),
u = Forcing(u_restoring; discrete_form=true, parameters=(; rate=1/5days)),
v = Forcing(v_restoring; discrete_form=true, parameters=(; rate=1/5days)))

momentum_advection = WENOVectorInvariant()
tracer_advection = WENO(order=7)

ocean = ocean_simulation(grid; forcing, momentum_advection, tracer_advection)
backend = JRA55NetCDFBackend(41)
atmosphere = JRA55PrescribedAtmosphere(arch; backend)
radiation = Radiation()
model = ocean.model

coupled_model = OceanSeaIceModel(ocean; atmosphere, radiation)
simulation = Simulation(coupled_model; Δt=2minutes, stop_time = 10days)

wall_time = [time_ns()]

function progress(sim)
ocean = sim.model.ocean
u, v, w = ocean.model.velocities
T = ocean.model.tracers.T

Tmax, Tmin = maximum(T), minimum(T)
umax = maximum(abs, u), maximum(abs, v), maximum(abs, w)
step_time = 1e-9 * (time_ns() - wall_time[1])

@info @sprintf("Time: %s, Iteration %d, Δt %s, max(vel): (%.2e, %.2e, %.2e), max(T): %.2f, min(T): %.2f, wtime: %s \n",
prettytime(ocean.model.clock.time),
ocean.model.clock.iteration,
prettytime(ocean.Δt),
umax..., Tmax, Tmin, prettytime(step_time))

wall_time[1] = time_ns()
end

simulation.callbacks[:progress] = Callback(progress, TimeInterval(4hours))

ocean.output_writers[:surface] = JLD2Writer(model, merge(model.tracers, model.velocities);
schedule = TimeInterval(5days),
filename = "acc_surface_fields",
indices = (:, :, grid.Nz),
overwrite_existing = true,
array_type = Array{Float32})
nothing #hide

# ### Spinning up the simulation
#
# As an initial condition, we have interpolated ECCO tracer fields onto our custom grid.
# The bathymetry of the original ECCO data may differ from our grid, so the initialization of the velocity
# field might cause shocks if a large time step is used.
#
# Therefore, we spin up the simulation with a small time step to ensure that the interpolated initial
# conditions adapt to the model numerics and parameterization without causing instability. A 10-day
# integration with a maximum time step of 1 minute should be sufficient to dissipate spurious
# initialization shocks.

run!(simulation)
nothing #hide

# ### Running the simulation
#
# Now that the simulation has spun up, we can run it for the full 2 years.
# We increase the maximum time step size to 10 minutes and let the simulation run for 2 years.

simulation.stop_time = 2*365days
simulation.Δt = 10minutes
run!(simulation)
nothing #hide

# ## Visualizing the results
#
# The simulation has finished, let's visualize the results.
# In this section we pull up the saved data and create visualizations using the CairoMakie.jl package.
# In particular, we generate an animation of the evolution of surface fields:
# surface speed (s), surface temperature (T), and turbulent kinetic energy (e).

u = FieldTimeSeries("surface.jld2", "u"; backend = OnDisk())
v = FieldTimeSeries("surface.jld2", "v"; backend = OnDisk())
T = FieldTimeSeries("surface.jld2", "T"; backend = OnDisk())
e = FieldTimeSeries("surface.jld2", "e"; backend = OnDisk())

times = u.times
Nt = length(times)

n = Observable(Nt)

land = interior(T.grid.immersed_boundary.bottom_height) .>= 0

Tn = @lift begin
Tn = interior(T[$n])
Tn[land] .= NaN
view(Tn, :, :, 1)
end

en = @lift begin
en = interior(e[$n])
en[land] .= NaN
view(en, :, :, 1)
end

un = Field{Face, Center, Nothing}(u.grid)
vn = Field{Center, Face, Nothing}(v.grid)

s = @at (Center, Center, Nothing) sqrt(un^2 + vn^2)
s = Field(s)

sn = @lift begin
parent(un) .= parent(u[$n])
parent(vn) .= parent(v[$n])
compute!(s)
sn = interior(s)
sn[land] .= NaN
view(sn, :, :, 1)
end

title = @lift string("ACC regional ocean simulation after ",
prettytime(times[$n] - times[1]))

λ, φ, _ = nodes(T)

fig = Figure(size = (1000, 1500))

axs = Axis(fig[1, 1], xlabel="Longitude (deg)", ylabel="Latitude (deg)")
axT = Axis(fig[2, 1], xlabel="Longitude (deg)", ylabel="Latitude (deg)")
axe = Axis(fig[3, 1], xlabel="Longitude (deg)", ylabel="Latitude (deg)")

hm = heatmap!(axs, λ, φ, sn, colorrange = (0, 0.5), colormap = :deep, nan_color=:lightgray)
Colorbar(fig[1, 2], hm, label = "Surface Speed (m s⁻¹)")

hm = heatmap!(axT, λ, φ, Tn, colorrange = (-1, 30), colormap = :magma, nan_color=:lightgray)
Colorbar(fig[2, 2], hm, label = "Surface Temperature (ᵒC)")

hm = heatmap!(axe, λ, φ, en, colorrange = (0, 1e-3), colormap = :solar, nan_color=:lightgray)
Colorbar(fig[3, 2], hm, label = "Turbulent Kinetic Energy (m² s⁻²)")

Label(fig[0, :], title)

save("snapshot_acc.png", fig)
nothing #hide

# ![](snapshot.png)

# And now we make a movie:

record(fig, "acc_region_surface.mp4", 1:Nt, framerate = 8) do nn
n[] = nn
end
nothing #hide

# ![](near_global_ocean_surface.mp4)