Optimization with multiple losses

[praghav444]

I am trying to fit parameters to minimize the errors for both ET and T (see below) i.e., with multiple losses. ET and T can have different lengths. What modification do I need to make in function min_model to consider multiple losses during optimization?

from lmfit import Minimizer, Parameters
def min_model(params,df,obs_LE,weekly_T_obs_global):
    mod_LE,_,_,_,T_ET_pre = calLE_PM(df,params)
    weekly_LE_mod = weekly_average(DateTime,mod_LE)
    weekly_LE_obs = weekly_average(DateTime,obs_LE)
    weekly_T_ET_mod = weekly_average(DateTime,T_ET_pre)
    weekly_T_mod = weekly_LE_mod*weekly_T_ET_mod
    temp = pd.DataFrame({'weekly_T_obs_global':weekly_T_obs_global,'weekly_T_mod':weekly_T_mod,
                        'weekly_LE_obs':weekly_LE_obs,'weekly_LE_mod':weekly_LE_mod})
    temp = temp.dropna()
    weekly_T_obs_global_1 = temp['weekly_T_obs_global'].values
    weekly_T_mod_1 = temp['weekly_T_mod'].values
    weekly_LE_mod_1 = temp['weekly_LE_mod']
    weekly_LE_obs_1 = temp['weekly_LE_obs']
    loss_1 = (obs_LE - mod_LE) ** 2         # Loss 1
    loss_2 = (weekly_LE_obs_1 - weekly_LE_mod_1)**2    # Loss 2
    return loss_1

def fit_gsurf_params(df,weekly_T_obs_global, tag=1, min_fit_var='LE_c'):
    params = setup_params(tag)
    LE_obs = df[min_fit_var]
    minner = Minimizer(min_model, params, fcn_args=(df,LE_obs,weekly_T_obs_global))
    fit = minner.minimize()
    LE_pm,g_surf_mod,g_canopy_mod,g_soil_mod,_ = calLE_PM(df,fit.params)
    return g_surf_mod, g_soil_mod, g_canopy_mod, LE_pm

[newville]

1. use numpy.concatenate to merge multiple partial arrays to minimize.
2. do not take the square: return the fit residual (data-model). The minimization code will find the parameter values that minimize the residual in the least squares sense.

[praghav444]

Thank you so much for your response. It is working completely fine. Just to confirm did you mean the below?

def min_model(params,df,obs_LE,weekly_T_obs_global):
    mod_LE,_,_,_,T_ET_pre = calLE_PM(df,params)
    weekly_LE_mod = weekly_average(DateTime,mod_LE)
    weekly_LE_obs = weekly_average(DateTime,obs_LE)
    weekly_T_ET_mod = weekly_average(DateTime,T_ET_pre)
    weekly_T_mod = weekly_LE_mod*weekly_T_ET_mod
    temp = pd.DataFrame({'weekly_T_obs_global':weekly_T_obs_global,'weekly_T_mod':weekly_T_mod,
                        'weekly_LE_obs':weekly_LE_obs,'weekly_LE_mod':weekly_LE_mod})
    temp = temp.dropna()
    weekly_T_obs_global_1 = temp['weekly_T_obs_global'].values
    weekly_T_mod_1 = temp['weekly_T_mod'].values
    weekly_LE_mod_1 = temp['weekly_LE_mod']
    weekly_LE_obs_1 = temp['weekly_LE_obs']
    loss_1 = (obs_LE - mod_LE)
    loss_2 = (weekly_LE_obs_1 - weekly_LE_mod_1)
    final_loss = np.concatenate((loss_1,loss_2), axis=0)
    return final_loss

[praghav444]

@newville I have a follow up question. When we use numpy.concatenate to merge multiple partial arrays to minimize, how can we consider the weights for each partial array. For example, I want to normalize the loss for each partial array with the length of the array. For example, in my case, lets say length of ET array is 1000 and length of T is 10. So when we use numpy.concatenate then the parameters obtained can be more biased towards ET. How to deal with this situation?

[newville]

@praghav444 yes to

    loss_1 = (obs_LE - mod_LE)
    loss_2 = (weekly_LE_obs_1 - weekly_LE_mod_1)
    final_loss = np.concatenate((loss_1,loss_2), axis=0)

you always need to consider how to set the weight for each element of the residual, whether you've combined different loss sources or not. A common approach is to divide "data-model" by the uncertainty in the data, at each data point (or "for each observation").

If you are combining 10 residuals from one "loss source" and 100,000 from another, you have to ask yourself (and you have to answer!) whether each individual observation is of equal importance (or perhaps "confidence that the model will very accurately match the data")
or whether each loss source is equally important, or somewhere in between. We cannot tell you that.