Each audio unit actually has 4 formats:
- Input and output scope formats for the input element.
 
- Input and output scope formats for the output element.
 
#47 is related to this but perhaps not general enough in its approach to solving this.
Perhaps we should change stream_format to always require taking the Scope and Element as arguments? cc @plietar