Device Model and Update Behavior

[ZhenyuWu323]

RPUConfig defines what device model and what update behavior is used, and we can apply different update algorithm like the "mixed precision" and "Tiki-taka" through Compounds that are used to define the update bahavior of RPUConig (Table IX of the tutorial paper). However, it also seems that the in-memory update behavior is also defined in analog_tile.update method called by the AnalogOptimizer. Where those in-memory training algorithm like "mixed precision" were actually implemented(It seems Compounds.py doesn't implement the algorithm itself), and how they will be called during in-memory training? What are the difference between them two? If I am going to customize the update behavior, should I derive a new analogtile or work on compunds?

[maljoras]

Hi @ZhenyuWu323,
For the in-memory training algorithm we currently use C++ code which is defined in the rpucuda library. For instance the TikiTaka implementation is here (for CPU. GPU has extra code, with many runtime optimizations). The python part simply calls the C++ code in the library so that you cannot see the actual implementation in python. However, if you want to customize it, you could simply write your own python code into the update function of the tile which then will be called in the optimizer. There is an example in the CustomTile code how to do that (you could simply derive class from that, see here ).