This repository contains the code and datasets for the paper:
Fidelity Isn’t Accuracy: When Linearly Decodable Functions Fail to Match the Ground Truth
Jackson Eshbaugh, Lafayette College, June 2025
Neural network outputs can often be well-approximated by linear models—but what does that tell us?
This project introduces the linearity score λ(f), a simple metric that quantifies how well a regression network’s predictions can be mimicked by a linear model. We show that this output-level diagnostic reveals important interpretability characteristics of learned functions—especially when fidelity (mimicking a network) diverges from accuracy (matching the ground truth).
📄 Read the paper: paper.pdf
This is a pre-submission draft. Feedback welcomed.
Email: jacksoneshbaugh@gmail.com
More at: jacksoneshbaugh.github.io
Let ( f ) be a trained regression network, and let ( \mathcal{L} ) be the space of affine functions. Define:
[ \lambda(f) := R^2(f, g^) = 1 - \frac{\mathbb{E}[(f(x) - g^(x))^2]}{\text{Var}(f(x))} ]
where ( g^* = \arg\min_{g \in \mathcal{L}} \mathbb{E}[(f(x) - g(x))^2] ).
In other words, λ(f) measures how well a linear model can mimic the predictions of a trained neural network. Unlike typical ( R^2 ), this score is not about matching the ground truth—it’s about measuring how linearly decodable the function learned by the network is from the input space.
All experiments and visualizations in the paper are contained in:
📓 lambda_linearity_score.ipynb
The notebook is fully self-contained and organized into:
- A reusable experimental framework
- Four datasets (synthetic + 3 real-world)
- Plots and tabulated results
To use λ(f) on your own data, modify the preprocessing and build_network() function and rerun the provided pipeline.
- Medical Cost Personal Dataset (Kaggle)
- Concrete Compressive Strength
- California Housing (scikit-learn)
- Synthetic: ( y = x \cdot \sin(x) + \varepsilon ), where ( \varepsilon \sim \mathcal{N}(0, \sigma^2) )
To run the notebook, install the following Python packages:
pip install tensorflow scikit-learn matplotlib pandas seaborn kagglehubTested with Python 3.11.
Special thanks to Professor Jorge Silveyra for the early discussions that helped spark this project.
© 2025 Jackson Eshbaugh. Released under the MIT License.