Chosing initial state before starting the simulation

[lucianolorenti]

Hi! First of all, thank you for the library.
I am starting to use the library for training some agents based on simulators we have. It would be helpful for our use case to be able to customize the initial state before starting a simulation. I think it is not possible right now.

Probably a workaround would be to return a different condition if the engine time is 0 in the getObservation method.
Maybe some mechanism can be included, by adding an abstract method of the interface.

is there a reason why the Java code is already bundled in a jar instead of the plain source code? (Just to be clear, I am asking just in terms of the ability to modify the Java code, I understand the purpose of the jar 😂 )

[MarcEscandell]

Hi Luciano! Thanks for your comment. I'm glad that you decided to use ALPypeRL library. Let me see if I can help you.

From what I am able to understand, you want to be able to return a custom state before starting the simulation (or at time 0). For your information, if you are implementing the ALPypeRLClientController in your Main "agent", you will be asked to implement double[] getObservation() yourself. If you successfully did, you are in charge of coding up whichever logic you want your getObservation function to follow. ALPypeRL jar file does not have any internal implementation as it is assuming you will do so (via the Java interface).

If I take the CartPole-v0 example, you can modify the getObservation function to handle what to return at time 0. For example:

// Check if at model startup
if (time() == 0) {
	// Return my custom initial state
	return new double[] {0.0, 0.0, 0.0, 0.0};
}

// Else, return the cartPole state
return new double[] {
	cartPole.getXPosition(),
	cartPole.getLinearVelocity(),
	cartPole.getAngle(),
	cartPole.getAngularVelocity()
};

On the other hand, if what you want to do is to be able to change certain parameter values of your model directly from your python script (say your train.py), then you are correct. At the moment it is not possible, but it is a feature that has been considered.
Do let me know if that is what you are looking for and I can consider its future implementation.