NAN values for gdp and gdppc

[lpicci96]

Functions in utils to get gdp and gdppc using WEO have null values for certain countries in 2022 - see AFG and LBN.
Need to add functionality so that the latest value is retrieved where the 2022 value is null

@jm-rivera do you know if the functions in the tracker repo address this issue?

[jm-rivera]

What would be the ideal behaviour you would like to see?

Are you looking for most recent non empty value?

[lpicci96]

Yes - set the gdp_year and take the most recent value if gdp_year value is null.
Is that a viable solution?

[jm-rivera]

The tracker may or may not have something. But sounds like it may be easiest to write something: -To keep only values under a certain year, -drop nans -(sort by if needed) drop duplicates keeping last (or groupy post sorting and keep last of group) And that would return the equivalent. It may be good to restrict how far you’re willing to go by filtering on both ends with the lower bound set relative to the requested year. (Like 5 or 10 years for example)

[lpicci96]

Issue solved with

def weo_to_dict(indicator: str, target_year: int = 2022, *, min_year: int = 2018) -> dict:
    """
    downloads weo data and returns a dictionary for a specific indicator and target year
    if target year value is null, the latest available year with data is used
    min year: earliest year acceptable to return a value
    """

    df = weo.WEO(f"{config.paths.raw_data}/weo_{WEO_YEAR}_{WEO_RELEASE}.csv").df

    df = (df
          .pipe(__clean_weo)
          .dropna(subset = ['value'])
          .loc[lambda d: (d.indicator == indicator)&(d.year>=min_year)&(d.year<=target_year),["iso_code", "year","value"]]
          .reset_index(drop=True)
          )

    return (df.loc[df.groupby(['iso_code'])['year'].transform(max) == df['year']]
            .set_index("iso_code")["value"]
            .to_dict()
            )

created new function to reduce code duplication.