Merge branch 'tcmxx/docs'

tcmxx · tcmxx · commit 82a22c6446bf · 2018-08-14T15:00:53.000+03:00
diff --git a/Documents/Training-PPO.md b/Documents/Training-PPO.md
@@ -23,14 +23,14 @@ The example [Getting Started with the 3D Balance Ball Environment](Getting-Start
 ## Explanation of fields in the inspector
 We use similar parameters as in Unity ML-Agents. If something is confusing, read see their [document](https://github.com/Unity-Technologies/ml-agents/blob/master/docs/Training-PPO.md) for mode datails.
 
-### TrainerPPO.cs
+#### TrainerPPO.cs
 * `isTraining`: Toggle this to switch between training and inference mode. Note that if isTraining if false when the game starts, the training part of the PPO model will not be initialize and you won't be able to train it in this run. Also,
 * `parameters`: You need to assign this field with a TrainerParamsPPO scriptable object. 
 * `continueFromCheckpoint`: If true, when the game starts, the trainer will try to load the saved checkpoint file to resume previous training.
 * `checkpointPath`: the path of the checkpoint, including the file name. 
 * `steps`: Just to show you the current step of the training.
 
-### TrainerParamsPPO
+#### TrainerParamsPPO
 * `learningRate`: Learning rate used to train the neural network.
 * `maxTotalSteps`: Max steps the trainer will be training.
 * `saveModelInterval`: The trained model will be saved every this amount of steps.
@@ -45,12 +45,12 @@ We use similar parameters as in Unity ML-Agents. If something is confusing, read
 * `numEpochPerTrain`: For each training, the data in the buffer will be used repeatedly this amount of times.
 * `useHeuristicChance`: See [Training with Heuristics](#training-with-heuristics).
 
-### RLModelPPO.cs
+#### RLModelPPO.cs
 * `checkpointToLoad`: If you assign a model's saved checkpoint file to it, this will be loaded when model is initialized, regardless of the trainer's loading. Might be used when you are not using a trainer.
 * `Network`: You need to assign this field with a scriptable object that implements RLNetworkPPO.cs. 
 * `optimizer`: The time of optimizer to use for this model when training. You can also set its parameters here.
 
-### RLNetworkSimpleAC
+#### RLNetworkSimpleAC
 This is a simple implementation of RLNetworkAC that you can create a plug it in as a neural network definition for any RLModelPPO. PPO uses actor/critic structure(See PPO algorithm).
 - `actorHiddenLayers`/`criticHiddenLayers`: Hidden layers of the network. The array size if the number of hidden layers. In each element, there are for parameters that defines each layer. Those do not have default values, so you have to fill them.
 	- size: Size of this hidden layer. 
@@ -69,6 +69,7 @@ If you already know some policy that is better than random policy, you might giv
 
 Note that your AgentDependentDeicision is only used in training mode. The chance of using it in each step for agent with the script attached depends on `useHeuristicChance`.
 
-
+## Create your own neural network architecture
+If you want to have your own neural network architecture instead of the one provided by [`RLNetworkSimpleAC`](#rlnetworksimpleac), you can inherit `RLNetworkAC` class to build your own neural network. See the [sourcecode](https://github.com/tcmxx/UnityTensorflowKeras/blob/tcmxx/docs/Assets/UnityTensorflow/Learning/PPO/TrainerPPO.cs) of `RLNetworkAC.cs` for documentation.
 
 
diff --git a/Documents/Training-SL.md b/Documents/Training-SL.md
@@ -13,7 +13,7 @@ The example scene `UnityTensorflow/Examples/Pong/PongSL` shows how to use superv
     3. Assign the Trainer to the `Trainer` field of your Brain.
 3. Create a Model
 	1. Attach a `SupervisedLearningModel.cs` to any GameObject.
-    2. Create a `SupervisedLearningNetwork` scriptable object in your project and assign it to the Network field in `SupervisedLearningModel.cs`.
+    2. Create a `SupervisedLearningNetworkSimple` scriptable object in your project and assign it to the Network field in `SupervisedLearningModel.cs`.
     3. Assign the created Model to the `modelRef` field of in `TrainerMimic.cs`
     
 4. Create a Decision
@@ -26,7 +26,7 @@ The example scene `UnityTensorflow/Examples/Pong/PongSL` shows how to use superv
   * The `isCollectinData` field in trainer needs to be true to collect training data. 
   
 ## Explanation of fields in the inspector
-### TrainerMimic.cs
+#### TrainerMimic.cs
 * `isTraining`: Toggle this to switch between training and inference mode. Note that if isTraining if false when the game starts, the training part of the PPO model will not be initialize and you won't be able to train it in this run. Also,
 * `parameters`: You need to assign this field with a TrainerParamsMimic scriptable object. 
 * `continueFromCheckpoint`: If true, when the game starts, the trainer will try to load the saved checkpoint file to resume previous training.
@@ -35,7 +35,7 @@ The example scene `UnityTensorflow/Examples/Pong/PongSL` shows how to use superv
 * 'isCollectingData': If the training is collecting training data from Agents with Decision.
 * `dataBufferCount`: Current collected data count.
 
-### TrainerParamsMimic
+#### TrainerParamsMimic
 * `learningRate`: Learning rate used to train the neural network.
 * `maxTotalSteps`: Max steps the trainer will be training.
 * `saveModelInterval`: The trained model will be saved every this amount of steps.
@@ -44,12 +44,12 @@ The example scene `UnityTensorflow/Examples/Pong/PongSL` shows how to use superv
 * `requiredDataBeforeTraining`: How many collected data count is needed before it start to traing the neural network.
 * `maxBufferSize`: Max buffer size of collected data. If the data buffer count exceeds this number, old data will be overrided. Set this to 0 to remove the limit.
 
-### SupervisedLearningModel.cs
+#### SupervisedLearningModel.cs
 * `checkpointToLoad`: If you assign a model's saved checkpoint file to it, this will be loaded when model is initialized, regardless of the trainer's loading. Might be used when you are not using a trainer.
 * `Network`: You need to assign this field with a scriptable object that implements RLNetworkPPO.cs. 
 * `optimizer`: The optimizer to use for this model when training. You can also set its parameters here.
 
-### SupervisedLearningNetworkSimple
+#### SupervisedLearningNetworkSimple
 This is a simple implementation of SuperviseLearningNetowrk that you can create a plug it in as a neural network definition for any SupervisedLearningModel.
 - `hiddenLayers`: Hidden layers of the network. The array size if the number of hidden layers. In each element, there are for parameters that defines each layer. Those do not have default values, so you have to fill them.
 	- size: Size of this hidden layer. 
@@ -66,15 +66,15 @@ You can also use a [conditional GAN](https://arxiv.org/abs/1411.1784) model inst
 
 Note that currently the GAN network we made does not support visual observation.
 
-### Steps
+#### Steps
 Most the same steps as using regular [supervised learning](Overall Steps) as before, but change step 3 to create a GAN model, and change the `TrainerParamsMimic` in step 2-2 to `TrainerParamsGAN` instead.
 
 - Create a GAN model:
 	1. Attach a `GANModel.cs` to any GameObject.
     2. Create a `GANNetworkDense` scriptable object in your project and assign it to the Network field in `GANModel.cs`.
     3. Assign the created Model to the `modelRef` field of in `TrainerMimic.cs`
     
-### GANModel.cs
+#### GANModel.cs
 * `checkpointToLoad`: If you assign a model's saved checkpoint file to it, this will be loaded when model is initialized, regardless of the trainer's loading. Might be used when you are not using a trainer.
 * `Network`: You need to assign this field with a scriptable object that implements RLNetworkPPO.cs. 
 * `generatorL2LossWeight`: L2 loss weight of the generator. Usually 0 is fine. 
@@ -85,8 +85,11 @@ Most the same steps as using regular [supervised learning](Overall Steps) as bef
 * `discriminatorOptimizer`: The optimizer to use for this model to train discriminator.
 * `initializeOnAwake`: Whether to initialize the GAN model on awake baed on shapes defined above. For ML-Agent environment, set this to false.
 
-### TrainerParamsGAN
+#### TrainerParamsGAN
 See [TrainerParamsMimic](#trainerparamsmimic) for other parameters not listed below.
 * `discriminatorTrainCount`: How many times the discriminator will be trained each training step.
 * `generatorTrainCount`: How many times the generator will be trained each training step.
 * `usePrediction`: Whether use [prediction method](https://www.semanticscholar.org/paper/Stabilizing-Adversarial-Nets-With-Prediction-Yadav-Shah/ec25504486d8751e00e613ca6fa64b256e3581c8) to stablize the training.
+
+## Create your own neural network architecture
+If you want to have your own neural network architecture instead of the one provided by [`SupervisedLearningNetworkSimple`](#supervisedlearningnetworksimple), you can inherit `SupervisedLearningNetwork` class to build your own neural network. See the [sourcecode](https://github.com/tcmxx/UnityTensorflowKeras/blob/tcmxx/docs/Assets/UnityTensorflow/Learning/Mimic/SupervisedLearningNetwork.cs) of `SupervisedLearningNetwork.cs` for documentation.
