Lesson 10, Concept 1: Output Verification

[nigoshh]

This concept was super interesting, like the rest of the course (amazing job 😍). I am confused about a couple of things.

1. In the beginning of the video (about 2:20) Andrew says that, if we constrained the transformation to be public knowledge, we would violate input privacy. Is it because the transformation algorithm itself is considered as an input of the information flow?
2. In the end of the video Andrew talks about auditing algorithms for fairness and other ethical concerns. Andrew talks very abstractly about "using structured transparency to evaluate the algorithm directly". What does that mean, concretely? Is it black-box auditing, i.e. auditing an algorithm when we are only granted access to its input-output pairs? That does not seem like something which could satisfactorily cover all cases, though. Also, with black-box access it might be fairly easy to reconstruct the model (I'm no expert, but see e.g. https://www.usenix.org/conference/usenixsecurity16/technical-sessions/presentation/tramer), which would effectively violate input privacy again.

[ibarrond]

1. I guess so, if the transformation includes a pretrained model, you would be showing it. At the same time, even if there is no "pretraining", a public knowledge transformation would be subject to model inversion attacks (much like what you commented in point 2)
2. I see it more as adding a flow that grants us this information without revealing anything else. You could see this as a flow where the inputs are the algorithm to be audited, the fairness algorithm and the fairness data, and the only output is a boolean "fair"/"not fair". Checking the fairness could be done while preserving input privacy (using MPC/HE). Definitely the scenario you mention would be subject to model inversion attacks.