Share your best coding tips!!!

[sarambl]

Hi all! I've been reflecting today over how many good tips and tricks we are all learning and maybe we could share a bit and then we can go through what people have shared in the forum??
Please comment with your best tips :)

I'll start with a couple I realized are useful:

1. If you are moving your functions to packages (means moving them to a .py file and importing from there), it's incredibly useful to add this to the beginning of your notebook

%load_ext autoreload
%autoreload 2

This tells jupyter to check if the file you are importing has changed, and load the changes. If not you can edit the file until you are blue, but the notebook won't load the change until you restart the kernel.

2. Edit the attributes (Julias tip!)
   If you change the units of something, you can keep track by editing the notebooks. If for example you have an aerosol concentration in units of 'm-3', and you want it in 'cm-3', you may do:

if ds['my_aerosol_conc'].attrs['units']=='m-3':
       ds['my_aerosol_conc'] = ds['my_aerosol_conc']*1e-6
       ds['my_aerosol_conc'].attrs['units'] = 'cm-3'

ALSO USE E.G:

ds.mean(keep_attrs=True)

The if test here ensures that you don't accidentally do it twice by running the same code twice.

[theodorekhadir]

I wrote a function to create a custom colorbar with your favorite colors

from matplotlib.colors import LinearSegmentedColormap
import matplotlib.pyplot as plt

def build_cmap(name, colors, N=100, reverse=''):
    '''
    Build a custom colormap from a list of colors
    The colors are linearly interpolated
    The number of colors in the final colormap is N
    You can reverse the colormap by setting reverse='r'
    ''' 
    if reverse == 'r':
        colors = [colors[len(colors)-1-i] for i in range(0, len(colors))]
        cmap = LinearSegmentedColormap.from_list(name, colors, N=N)
    else:
        cmap = LinearSegmentedColormap.from_list(name, colors, N=N)
    return cmap

# example

cmap_GoldenTheo = build_cmap('GoldenTheo', 
        [(5/256, 41/256, 132/256),(175/256, 186/256, 219/256),
        (249/256, 240/256, 216/256),(237/256, 204/256, 142/256),
        (126/256, 90/256, 79/256)])

fig, ax = plt.subplots(figsize=(8, 1))
fig.subplots_adjust(bottom=0.5)

cb1 = mpl.colorbar.ColorbarBase(ax, cmap=cmap_GoldenTheo,
                                orientation='horizontal')

cb1.set_label('Enjoy!', fontsize=20)

[pzieger]

Nice! Here is a good paper on how to choose colorbars in geo/meteo sciences: Stauffer et al. "Somewhere Over the Rainbow: How to Make Effective Use of Colors in Meteorological Visualizations " https://journals.ametsoc.org/view/journals/bams/96/2/bams-d-13-00155.1.xml

[SPfaller]

Might be a simpler one, but when plotting data using (robust=True) has been incredibly useful for me in order to exclude weird data by only plotting the 95th percentile!

[sarambl]

ds['my_variable'].plot(robust=True)

ds['my_variable'].plot(vmax=10)

[sarambl]

Perfect example of nice-to-share !

[mateuszmatu]

When loading in a dataset in xarray (and probably pandas too), you can drop loading in certain variables to be RAM friendly!

data = xr.open_dataset(file, drop_variables=['variable_you_dont_need'])

[dominichr1]

try and except are useful sometimes e.g.

def divide_x_by_y(x, y):
    try:
        n = x / y
    except ZeroDivisionError:
        print("can't divdie by zero, return np.nan")
        n = np.nan
    print("result is", n)
    return n

[theodorekhadir]

We re gonna show try except at the python for newbies session

[dominichr1]

Having a look at all the nbextensions you can use in your notebooks - there are some really helpful ones.
some useful examples here

[dominichr1]

This function requires the fig and the name, the rest are given so-called default values/strings.
if the folder is not there, python will interact with the operating system and create a new folder - if the folder is there, it will just save the fig.

import os 

def save_fig(fig, name, path=r'C:\Users\DominicHeslinRees\Pictures', folder='plots', 
            format_fig=".jpeg"):    
    os.makedirs(path+"\\"+folder, exist_ok=True) #For value True leaves directory unaltered. 
    fig.savefig(path+"\\"+folder+"\\"+str(name)+str(format_fig), bbox_inches='tight')
    print("saved to/as: "+str(path+"\\"+folder+"\\"+str(name)+str(format_fig)))

[sarambl]

Great! I also recommend pathlib for dealing with these things.

from pathlib import Path

my_path = Path('my/path/')
my_filename = pa / 'file.png'

my_path.mkdir()

fig.savefig(my_filename)

https://docs.python.org/3/library/pathlib.html

[dominichr1]

If you want to work in full screen mode

from IPython.core.display import display, HTML
display(HTML("<style>.container { width:100% !important; }</style>"))

[EmmaAxe]

If you want to increase the size on the labels and the numbers on your plot:

Instead of 'Year' you can also write ds.long_name or ds.unit

axs1.set_xlabel('Year', fontsize=20)
axs1.set_ylabel('Year', fontsize=20)

axs1.set_ylabel(ds.long_name, fontsize=20)

axs1.tick_params(axis="x", labelsize=20)
axs1.tick_params(axis="y", labelsize=20)

To write a figure title and subfigure titles:

fig.suptitle('Text', fontsize=30)
axs1.set_title(" Text" , fontsize=25)

[sarambl]

Good point! Note that if you want to edit the size of everything, you can control the plot size by defining your figure first:
The dpi controls how big the plot will show up in your notebook, the figsize how large the plot is.

fig, ax = plt.subplots(1, figsize=[4,3], dpi=100)
dss_s['mmrso4'].isel(time=0).plot(robust=True, ax = ax)

fig, ax = plt.subplots(1,figsize=[10,8], dpi=50)
dss_s['mmrso4'].isel(time=0).plot(robust=True, ax = ax)

[fairyeszhuo]

Instead of putting function in util.py, that won't be updated in the notebook if you modify it, you can store functions in functions.ipynb, and run in your notebook %run functions.ipynb to use the functions, and run it again to get the update when you modify something in the functions.ipynb file.

[sarambl]

Hi! So I would really recommend the reload instead here! If you put the following at the top of your notebook, python will reload the file.py when you edit it.

%load_ext autoreload
%autoreload 2

[sarambl]

I uploaded the notebooks from todays tutorial here :) Have fun! Sorry it got so long!
https://github.com/pangeo-data/escience-2022/blob/main/tutorial/examples/notebooks/basic_python_tutorial/01-Some_basic_python.ipynb

[AntoineHaddon]

Correcting longitudes for plotting

There can be a problem due to longitudes 'wrapping around' and going from -180 to +180, leads to issue with pcolormesh trying to make a quadrilateral with corners -180 and 180, Solution is to 'unwrap' longitudes :

longitudePlt = np.unwrap(dataSet.longitude, period=360)

from https://stackoverflow.com/questions/40009652/using-python-basemap-pcolormesh-with-non-monotonic-longitude-jumps

[theodorekhadir]

For changing directly the coordinates of your dataset into a -180 to 180 lat grid:
ds = ds.assign_coords(lon=(((ds.lon + 180) % 360) - 180)).sortby('lon')

[fairyeszhuo]

Calculate monthly climatology and anomaly
clm = sst.sel(time=slice('1982-01-01','2020-12-01')).groupby('time.month').mean(dim='time')
anm = (sst.groupby('time.month') - clm)

[sarambl]

OMG SO COOL! Had no idea. Thanks @fairyeszhuo and @EmmaAxe !

[theodorekhadir]

Hello go see my tuto for plotting :)

[sarambl]

Plotting with multiple axis:

def fix_ax(c, ax3):
    ax3.yaxis.label.set_color(c)
    ax3.tick_params(axis='y', colors=c, )
    ax3.spines.right.set_color(c)


fi, ax1 = plt.subplots(1)
_ds.plot.hist(bins = np.logspace(-1,2), ax=ax1, alpha=.5)
ax1.set_ylabel('axis 1')

## Second axis: 
col = 'r'
ax2 = ax1.twinx()
p = _ds.plot.hist(bins = np.linspace(0,20), ax=ax2, color=col, alpha=.5)
fix_ax(col, ax2)
ax2.set_ylabel('axis 2')

## Third axis
col = 'm'
ax3 = ax1.twinx()
p = _ds.plot.hist(bins = np.linspace(2,10), ax=ax3, color =col, alpha=.5)
# Specify position of axis
ax3.spines.right.set_position(("axes", 1.18))
ax3.set_ylabel('axis 3')
fix_ax(col, ax3)

ax1.set_xlim([0,40])