FDB and Polytope model data access and processing

Jupyter Notebook Examples Using MeteoSwiss NWP Data

This repository contains examples for using FDB and Polytope to access ICON forecast data and process it using meteodata-lab.

FDB allows to retrieve any multi-dimensional dataset of the recent real-time ICON NWP forecasts. FDB is designed to access full horizontal fields (feature extraction is not supported) and it is only accessible from within CSCS.

Polytope allows to efficiently extract specific features from the same real-time ICON NWP forecast, such as grid point data, time-series, vertical profiles or polygons. Polytope is an HTTP service and therefore access is not restricted to CSCS (it supports access from LabVM and ACPM).

---

📓 Notebooks

The following notebook demonstrates how to access ICON-CH1-EPS & ICON-CH2-EPS model data with FDB:

• How to access model data with FDB from CSCS and HTML rendered output

The following notebooks demonstrate various use cases to access ICON-CH1-EPS & ICON-CH2-EPS model data via Polytope:

Feature extraction:

• Bounding Box and HTML rendered output
• Polygon country cut-out and HTML rendered output
• Time Series and HTML rendered output
• Trajectory and HTML rendered output
• Vertical Profile and HTML rendered output

Full field retrieval:

• Full field and HTML rendered output

---

🚀 Getting Started

Option 1: FDB setup at CSCS

FDB data access requires the FDB libraries. In order to facilitate the use of the notebooks, we provide a pre-built environment with the required libraries, as well as jupyter kernel configuration for VSCode. To download this environment and create/configure a jupyter kernel to use it, run:

bash host/install_kernel.sh

If you would like to develop Python examples outside of the Juypter notebooks, see https://meteoswiss.atlassian.net/wiki/spaces/IW2/pages/144150401/Realtime+FDB+for+ICON#How-to-use-it how to use the FDB uenv environment.

Using VSCode

Ctrl-Shift P -> 'Notebook: Select Notebook Kernel' -> 'Select Another Kernel' -> 'Jupyter Kernel' -> 'Polytope demo'

Tip

• The python extension of VScode should not be in restricted mode.

• You might need to reload the window Ctrl-Shit P -> 'Developer: Reload Window' in order to let VSCode recognize the newly installed kernel.

Option 2: Python setup for Polytope

Deploy your own python environment using the following commands. Alternatively, when working at CSCS, you can use Option 1 mentioned above.

1. Install the poetry environment.

poetry install

2. Install the kernel.

poetry run python -m ipykernel install --user --name=polytope-env --display-name "polytope demo"

---

📚 Forecasts available in FDB

Note

The realtime FDB normally contains just the latest day of forecasts. This means the FDB requests should usually use date = today and time as either 0000, 0300, 0600, 0900 etc. (given the forecasts run every 3 hour) for ICON-CH1-ENS and 0000, 0600, 1200 etc. for ICON-CH2-ENS. The data is usually available in the FDB a couple of hours after the forecast start time. If the requests to FDB return no data, see these instructions for how to query yourself which data is currently archived in the FDB.

Development

See DEVELOPMENT.md for how to update and publish notebooks consistently.