Historical data in the database (read only) #1265
Description
The E12 team retain CSV master exports of data for previous cohorts. We'd like to migrate that into the same database as the main E12 platform so that we can manage access centrally using the same mechanisms (Entra ID) we do for cohort 6, 7, 8 etc.
Whilst there may be some value in showing that data within the platform itself, in practice the E12 team don't regularly use it for more than data access requests and sense checking new calculations.
We should consider this data "frozen" - we do not aim to change it. New data continues to be collected by the platform in the existing tables.
- Step 1: E12 team to produce a single aggregated CSV file of previous data
- May be "wide", i.e. columns with missing data for questions that have been added/modified across cohorts
- Step 2: Dev team creates a "historical" Django model for the CSV file
- One model unless it's really obvious that we are awkwardly shoving the data into a single one
- Important to have a Django model to have schema migration and governance
- Step 3: Dev team writes and runs a command to import the historical data