"CRUDE" examples -- building up the Flat Navigation Dispatch tools #1
Description
The task is to write streamlit apps of growing complexity to build up (and later demo) the CRUDE GUI tools (aka "Flat Navigation Dispatch"). (CRUDE is for CRUD + Execution).
The following are (or are planned to be) in examples/crude/:
take_01_salary.py: A simple app to demo the use of Mappings to handle complex typetake_02_salary.py: Like take01, but using i2.wrapper to transform the functiontake_03_salary.py: Like take02, but wheresalarykey entry is done via a selectboxtake_04_model_run.py: An example of how to transform (manually) a function that runs some data through a model (both too complex to enter directly and explicitly in forms) into a streamlit dispatchable function that uses a store (a Mapping) to manage complex data.take_05_model_run.py: Same as take_04_model_run, but where the dispatch is not as manual.
| name | func(s) | dispatch_prep | store_type | store_contents | key_entry | comments | |
|---|---|---|---|---|---|---|---|
| take_01_salary | multiplication | manual | dict | static | freeform | ||
| take_02_salary | multiplication | semi-auto | dict | static | freeform | ||
| take_03_salary | multiplication | semi-auto | dict | static | selectbox | ||
| take_04_model_run | apply_model | manual | dict | static | freeform | ||
| take_05_model_run | apply_model | semi-auto+ | dict | static | freeform | ||
Flat Navigation Dispatch front issue
Copied from front issue#12.
The design pattern I'm proposing is that when you need to dispatch a function like
train(learner, data)where learner and data are complex python objects, you do it by creating three components:
- CRUD for learners
- CRUD for data
- train component, which points to the learner and data CRUD components to get its inputs
The CRUD components are made of two parts
- For the creation of the object: A specific UI component
- For all other CRUD operations: A (py2)store that can handle the type of objects being made
Sometimes the creation will itself require some complex object inputs. For example, the case of model in:
run(model, data)
But a model is just the output of train(learner, data), so we fall back on the train component,
which in turn relies on the learner and data components.
The GUI navigation that the use of this pattern enables is "flat" in the sense that the process structure is less of a tree or DAG, and more of a "choose your action, and if the inputs you require don't exist, go choose the action that will make these inputs available".
This flatness may or may not be desirable from a UX point of view, but is certainly desirable from an architectural simplicity point of view, and we can get a great bang for the buck as far as component reuse.
Any code that appears in this form:
data = get_data(data_key)
prepped_data_1 = prep1(data, another_param)
prepped_data_2 = prep2(data, yet_another, and_another)
result = do_something(prepped_data_1, prepped_data_2, configs)
...can be dispatched to such a "flat GUI" dispatching get_data, prep1, prep2 and do_something functions, after wrapping these in a "dispatchable" form that uses stores to both solve the complex data entry problem and the action/navigation order problem. The store keys play the role of intermediate variables in the python code above.
Code example
A function such as:
from typing import Any
FVs = Any
FittedModel = Any
# to dispatch:
def apply_model(fvs: FVs, fitted_model: FittedModel, method='transform'):
method_func = getattr(fitted_model, method)
return method_func(list(fvs))would have a dispatchable form (automatically created by wrappers) as such:
# This is what we want our "dispatchable" wrapper to look like
# There should be real physical stores for those types (FVs, FittedModel) that need them
fvs_store = dict(
train_fvs_1=[[1], [2], [3], [5], [4], [2], [1], [4], [3]],
train_fvs_2=[[1], [10], [5], [3], [4]],
test_fvs=[[1], [5], [3], [10], [-5]],
)
fitted_model_store = dict(
fitted_model_1=MinMaxScaler().fit(fvs_store['train_fvs_1']),
fitted_model_2=MinMaxScaler().fit(fvs_store['train_fvs_2']),
)
# And a store for the result of the function if we want this result to be reused later
# (for example for further inputs)
apply_model_store = dict()
from typing import Any, Mapping, Tuple
FVsKey = str # really, should be a str from a list of options, given by list(fvs_store)
FittedModelKey = str # really, should be a str from a list of options, given by list(fitted_model_store)
Result = Any
# dispatchable function:
def apply_model_using_stores(
fvs: FVsKey,
fitted_model: FittedModelKey,
method: str = 'transform',
fvs_store: Mapping[FVsKey, FVs] = fvs_store,
fitted_model_store: Mapping[FittedModelKey, FittedModel] = fitted_model_store,
apply_model_store: Mapping[Tuple[FVsKey, FittedModelKey], Result] = apply_model_store
):
# get the inputs
fvs_value = fvs_store[fvs]
fitted_model_value = fitted_model_store[fitted_model]
# compute the function
result = apply_model(fvs_value, fitted_model_value)
# store the outputs
apply_model_store[fvs, fitted_model] = result
return result # or not