realtime

LPT Set-Up for Real Time

These instructions are for setting up realtime LPT from scratch on a Linux system with an Anaconda Python environment.

1. Set up Python for LPT

Download Anaconda

First, download Anaconda Python for Linux using the download link at https://www.anaconda.com/. The current version is for Python 3.7. Subsequent Python versions should work as long as they maintain backwards compatibility. The current direct download link is here.

Install Anaconda

This is usually done by running the installer script (e.g., Anaconda3-2020.02-Linux-x86_64.sh). The script will ask for a directory where to install Python environments. Generally your home directory on your system will do. The Python environment on my system takes up 1.7 GB of space.

Set up Python environment

To set up a Python environment with the modules needed to run LPT, you can issue the following command:

conda create --name lpt numpy pandas matplotlib basemap netCDF4 h5py scipy wrf-python gdal

The command may first ask you to update Anaconda (optional). Answer yes when prompted with the list of modules to be installed, and it will install the environment. To activate the environment before running LPT, use source activate lpt.

2. Obtain the code

The code is on GitHub here.

Download

Navigate to a directory where you want to run LPT, and get the code using the following:

git clone https://github.com/brandonwkerns/lpt-python-public.git

This should download the code and put it under the directory lpt-python-public/.

NOTE: If you want to use a different directory name, you can call the git clone command like this:

git clone https://github.com/brandonwkerns/lpt-python-public.git custom_directory_name

When/if code updates are needed, doing a git pull anywhere under the directory should update all of the relevant files.

3. Set up realtime directory

Copy "MASTER" files

The code repository has "MASTER" files which are to be copied and edited. The idea is that the MASTER files may be updated on the GitHub repository. If you directly edit the "MASTER" files, it may cause issues when git pull is used.

Get into the realtime directory using cd lpt-public/realtime (or whatever custom directory you specified). Then, copy the following "MASTER" files:

cp lpt.cron.MASTER lpt.cron
cp download_cfs_rt.MASTER.sh download_cfs_rt.sh
cp lpt_run_cfs_fcst.MASTER.py lpt_run_cfs_fcst.py
cp download_cmorph_rt.MASTER.sh download_cmorph_rt.sh
cp lpt_crontab_wrapper.MASTER.sh lpt_crontab_wrapper.sh
cp lpt_run_cmorph.MASTER.py lpt_run_cmorph.py

Edit the new files for your system.

Edit the directories in the following lines:

In lpt.cron, for this script directory.

LPT_REALTIME_DIR=/path/to/this/realtime/script/directory

In download_cfs_rt.sh, for the directory you want to download the raw realtime data to.

download_parent_dir=/path/to/keep/your/data

In download_cmorph_rt.sh, for the directory you want to download the raw realtime data to.

workdir=/path/to/keep/your/data

In lpt_crontab_wrapper.sh, for this script directory and the path to your anaconda3 directory. You can get the anaconda3 directory using which anaconda and it's the path with taking out the bin/anaconda at the end.

WORKDIR=/path/to/this/realtime/script/directory/
ANACONDA_DIR=/path/to/anaconda3

In lpt_run_cmorph.py, for the path to CMORPH data (same as specified in download_cmorph_rt.sh).

dataset['raw_data_parent_dir'] = '/path/to/cmorph/rt/'

In lpt_run_cfs_fcst.py, for the path to CFS forecast data (same as specified in download_cfs_rt.sh).

dataset['raw_data_parent_dir'] = '/path/to/model_fcst_grib/cfs'