MASE implementation.

[lukbast]

I'm studying part of the course about time series and I decided to implement MASE on my own. In odrer to do that I recreated formula from Hyndman and Koehler paper:

This is the code:

def mean_absolute_scaled_error(y_true, y_pred):
    """
    Implements MASE, inspired by Hyndman's and Koehler's paper 
    "Another look at measures of forecast accuracy"
    """
    e = y_true-y_pred # calculates e for all predictions
    
    # part under division line
    x = (tf.reduce_sum(tf.abs(y_true[1:] - y_true[:-1])) / len(y_true) - 1
    
    # calculates scaled error for all predictions
    q = e/x 
    
    return tf.reduce_mean(tf.abs(q))  # returns MASE

Used on naive forecast it returns result very close to one, 1.000213 to be exact. It seems to work correctly for me but your implementation is not only different but also calculates MASE in different manner.

In formula, e_t is a difference between precition and true value for a single timestep and q_t is a scaled error of prediction for a single timestep. Quote from paper:

Let Y_t denote the observation at time t and F_t denote the forecast of Y_t. Then define the forecast
error e_t = Y_t − F_t

After calculating scaled error for every sample, paper suggest to take absolute mean of them to get Mean Absolute Scaled Error.

Your implementation just take MAE of predictions then divide it by MAE taken form naive forecast. Why is that? Or I just misunderstood something?

[mrdbourke]

Heya,

The MASE (mean absolute scaled error) formula used in the notebook 10: https://github.com/mrdbourke/tensorflow-deep-learning/blob/main/10_time_series_forecasting_in_tensorflow.ipynb

# MASE implemented courtesy of sktime - https://github.com/alan-turing-institute/sktime/blob/ee7a06843a44f4aaec7582d847e36073a9ab0566/sktime/performance_metrics/forecasting/_functions.py#L16
def mean_absolute_scaled_error(y_true, y_pred):
  """
  Implement MASE (assuming no seasonality of data).
  """
  mae = tf.reduce_mean(tf.abs(y_true - y_pred))

  # Find MAE of naive forecast (no seasonality)
  mae_naive_no_season = tf.reduce_mean(tf.abs(y_true[1:] - y_true[:-1])) # our seasonality is 1 day (hence the shifting of 1 day)

  return mae / mae_naive_no_season

It takes the mean before the end.

Slightly different to the formula you're looking at but I believe they achieve the same result.

The notebook 10 implementation is based on Sktime's implementation: https://github.com/alan-turing-institute/sktime/blob/ee7a06843a44f4aaec7582d847e36073a9ab0566/sktime/performance_metrics/forecasting/_functions.py#L16