No engines found for F22.

[emanuelegreco29]

Hello, I am trying to build a RL agent (with a basic DQN) that controls an aircraft and guides it towards a waypoint. Unfortunately, whenever I start the training, the following error appears, halting the execution:

Throttle 0 does not exist! 0 engines exist, but attempted throttle position setting is for engine 0

I will provide the code I used, to give more context. Whenever train_agent is called, the environment is successfully created and wrapped in a Monitor. However, before training begins, I get the error mentioned. I tried changing aircraft type (F16, 737, A320) and I am never able to run the training. I also noticed that, when calling model.learn(), I get a ton of these:

Failed to tie property propulsion/starter_cmd to object methods
Failed to tie property propulsion/magneto_cmd to object methods
Failed to tie property propulsion/active_engine to object methods
Failed to tie property forces/fbx-prop-lbs to indexed object methods
Failed to tie property forces/fby-prop-lbs to indexed object methods
Failed to tie property forces/fbz-prop-lbs to indexed object methods
Failed to tie property moments/l-prop-lbsft to indexed object methods
Failed to tie property moments/m-prop-lbsft to indexed object methods
Failed to tie property moments/n-prop-lbsft to indexed object methods
Failed to tie property propulsion/total-fuel-lbs to a pointer
Failed to tie property propulsion/total-oxidizer-lbs to a pointer
Failed to tie property propulsion/refuel to a pointer
Failed to tie property propulsion/fuel_dump to a pointer
Failed to tie property propulsion/fuel_freeze to object methods
Failed to tie property fcs/throttle-cmd-norm[0] to indexed object methods
Failed to tie property fcs/throttle-pos-norm[0] to indexed object methods
Failed to tie property fcs/mixture-cmd-norm[0] to indexed object methods
Failed to tie property fcs/mixture-pos-norm[0] to indexed object methods
Failed to tie property fcs/advance-cmd-norm[0] to indexed object methods
Failed to tie property fcs/advance-pos-norm[0] to indexed object methods
Failed to tie property fcs/feather-cmd-norm[0] to indexed object methods
Failed to tie property fcs/feather-pos-norm[0] to indexed object methods

Any help would be deeply appreciated. Thank you.

Code

Environment

class ChaseEnv(gym.Env):
    def __init__(self):
        super(ChaseEnv, self).__init__()
        self.action_space = spaces.Discrete(6)
        # State: [lat, lon, alt, speed, heading] + [target_lat, target_lon, target_alt]
        self.observation_space = spaces.Box(low=-np.inf, high=np.inf, shape=(8,), dtype=np.float32)
        
        self.max_steps = 200
        self.current_step = 0
        
        self.elevator_cmd = 0.0
        self.rudder_cmd = 0.0
        self.throttle_delta = 0.1
        
        self.sim = create_fdm(JSBSIM_ROOT)
        
        self.initial_lat = 37.0  # deg
        self.initial_lon = -122.0  # deg
        self.initial_alt = 1000.0   # ft
        self.initial_speed = 100.0  # ft/s
        self.initial_heading = 0.0  # deg
        
        self.target = np.array([self.initial_lat + 0.05, self.initial_lon + 0.05, self.initial_alt + 50])
        
        self._setup_simulation()

    def _setup_simulation(self):
        self.sim.load_model("f22")
        self.sim["ic/lat-gc-deg"] = self.initial_lat
        self.sim["ic/long-gc-deg"] = self.initial_lon
        self.sim["ic/h-sl-ft"] = self.initial_alt
        self.sim["ic/vt-fps"] = self.initial_speed
        self.sim["ic/psi-true-deg"] = self.initial_heading

        self.sim.run_ic()

        self.sim["fcs/elevator-cmd-norm"] = 0.0
        self.sim["fcs/rudder-cmd-norm"] = 0.0
        self.sim["fcs/throttle-cmd-norm"] = 0.5

    def reset(self, seed=None, options=None):
        if seed is not None:
            np.random.seed(seed)
        self.current_step = 0
        self._setup_simulation()
        return self._get_obs()
    
    def _get_obs(self):
        lat = self.sim["position/lat-gc-deg"]
        lon = self.sim["position/long-gc-deg"]
        alt = self.sim["position/h-sl-ft"]
        speed = self.sim["velocities/vt-fps"]
        heading = self.sim["attitude/psi-true-deg"]
        state = np.array([lat, lon, alt, speed, heading])
        return np.concatenate([state, self.target])
    
    def step(self, action):
        self.elevator_cmd = 0.0
        self.rudder_cmd = 0.0

        current_throttle = self.sim["fcs/throttle-cmd-norm"]
        
        # Action mapping
        if action == 0:      # Pitch Up
            self.elevator_cmd = 0.5
        elif action == 1:    # Pitch Down
            self.elevator_cmd = -0.5
        elif action == 2:    # Go right
            self.rudder_cmd = 0.5
        elif action == 3:    # Go left
            self.rudder_cmd = -0.5
        elif action == 4:    # Thrust Up
            current_throttle = min(current_throttle + self.throttle_delta, 1.0)
        elif action == 5:    # Brake
            current_throttle = max(current_throttle - self.throttle_delta, 0.0)
        
        self.sim["fcs/elevator-cmd-norm"] = self.elevator_cmd
        self.sim["fcs/rudder-cmd-norm"] = self.rudder_cmd
        self.sim["fcs/throttle-cmd-norm"] = current_throttle
        
        self.sim.run()
        
        obs = self._get_obs()
        # Calculate target distance
        agent_pos = obs[:3].astype(np.float32)
        distance = np.linalg.norm(agent_pos - self.target)

        reward = -distance
        if distance < 0.01:
            reward += 100.0
        
        self.current_step += 1
        done = (distance < 0.01) or (self.current_step >= self.max_steps)
        
        return obs, reward, done, {}
    
    def render(self, mode='human'):
        obs = self._get_obs()
        print("State:", obs)

Training

def train_agent(total_timesteps=10000):
    env = ChaseEnv()
    model = DQN("MlpPolicy", env, verbose=1, device="cuda")
    model.learn(total_timesteps=total_timesteps)
    return model, env

def evaluate_agent(model, env, n_eval_episodes=20):
    rewards, lengths = evaluate_policy(model, env, n_eval_episodes=n_eval_episodes, render=False)
    avg_reward = np.mean(rewards)
    avg_length = np.mean(lengths)
    return avg_reward, avg_length, rewards, lengths

[seanmcleod70]

You haven't shown the implementation of create_fdm(). JSBSim looks for engines either in a sub-directory of the aircraft's directory or in the engine path directory that is set.

[emanuelegreco29]

I am sorry, I didn't include it at first because it is very straight forward (and I assume it's not where the problem originates). Here it is:

def create_fdm(jsbsim_root):
    fdm = jsbsim.FGFDMExec(root_dir=jsbsim_root)
    fdm.set_aircraft_path(os.path.join(jsbsim_root, "aircraft"))
    fdm.set_engine_path(os.path.join(jsbsim_root, "engine"))
    fdm.set_systems_path(os.path.join(jsbsim_root, "systems"))
    return fdm

[seanmcleod70]

What happens if you run the python examples in the repo - https://github.com/JSBSim-Team/jsbsim/tree/master/examples/python?

[emanuelegreco29]

For all three examples, I have set:

PATH_TO_JSBSIM_FILES=r"C:\Users\leleg\AppData\Local\JSBSim"

I needed to use r"..." to escape characters, otherwise it would not recognize my path (I have python files stored on a different directory, in another disk, for all that matters) and give me this error:

File "c:\Users\leleg\Downloads\RudderKick.py", line 35
    PATH_TO_JSBSIM_FILES="C:\Users\leleg\AppData\Local\JSBSim"
                         ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
SyntaxError: (unicode error) 'unicodeescape' codec can't decode bytes in position 2-3: truncated \UXXXXXXXX escape

TrimEnvelope

I get from output:

c:\Users\leleg\Downloads\TrimEnvelope.py:94: SyntaxWarning: invalid escape sequence '\g'
ax.set_ylabel('Flight Path Angle $\gamma$ (deg)')

followed by countless Sorry, wdot doesn't appear to be trimmable. At the end, I get this image as output:

PlotAoaVsCAS

Here, I get a tons of errors:

FGPropertyManager::GetNode() No node found for aero/coefficient/CLalpha

In file C:/Users/leleg/AppData/Local/JSBSim/aircraft/global5000/global5000.xml: line 903

  No property by the name aero/coefficient/CLalpha has been defined. This property will 
  not be logged. You should check your configuration file.
  Sorry, wdot doesn't appear to be trimmable

ERROR: unable to open the file C:/Users/leleg/AppData/Local/JSBSim/./global5000.csv     
       => Output to this file is disabled.

  Sorry, wdot doesn't appear to be trimmable

ERROR: unable to open the file C:/Users/leleg/AppData/Local/JSBSim/./global5000.csv
       => Output to this file is disabled.

  Sorry, wdot doesn't appear to be trimmable

ERROR: unable to open the file C:/Users/leleg/AppData/Local/JSBSim/./global5000.csv
       => Output to this file is disabled.

  Sorry, wdot doesn't appear to be trimmable

ERROR: unable to open the file C:/Users/leleg/AppData/Local/JSBSim/./global5000.csv
       => Output to this file is disabled.
       
...

In the end, I receive:

-----------------------------------------
Altitude 30000 - Weight heavy - CG 871.8570000000001
-----------------------------------------
169.99999999999946 12.114399373419582
179.9999999999984 10.870185285962396
190.00000000000014 9.812388342909978
199.9999999999994 8.905997604627204
210.00000000000003 8.123695180516975
219.99999999999957 7.443963917872366
230.0000000000003 6.8496988731448445
240.00000000000014 6.327178023826337
249.99999999999903 5.865304896052884

and the following plot:

RudderKick

This one runs flawlessly and gives me this image as output:

[seanmcleod70]

I needed to use r"..." to escape characters,

Yes, since a single backslash \ is interpreted as being part of an escape sequence, e.g. \t is an escape sequence to insert a tab character. So you will need to either use a raw string like you did with r"..." or replace all the single backslashes with double-backslashes, e.g. c:\\users\\john\\... or replace them with a single forward slash Unix style since Windows will generally also handle a forward slash.

But given that you were able to run all 3 and have plots generated for them means that these example Python scripts were able to find the relevant engine files.

[emanuelegreco29]

That makes me wonder why it cannot find the engines in my script. Maybe I'm missing something?

[seanmcleod70]

Print out all the paths in your create_fdm() function and double-check the engine directory path definitely exists with the relevant engine files in it.

[seanmcleod70]

Also what happens if you move this line of code:

self.sim = create_fdm(JSBSIM_ROOT)

And make it the first line of code inside _setup_simulation()'?

I'm wondering whether maybe the first run succeeds, but subsequent runs call reset() and _setup_simulation() without creating a new FDM and maybe that combination has some sort of issue.

[emanuelegreco29]

output.txt

[seanmcleod70]

Okay, so if you take a look at the output it doesn't look like there is an issue finding the engine file. There appear to be 2 loads of the F-22 model, I presume from 2 runs? And in both cases you can see the engine data output to the console, e.g. listing the thrust tables etc.

The first load of the F-22 looks normal until:

End of vehicle configuration loading.
-------------------------------------------------------------------------------
Environment creato.
Wrapper inserito.
Dummy inserito.
Using cuda device
Modello creato.
Failed to tie property inertial/sea-level-radius_ft to object methods
Failed to tie property simulation/gravity-model to object methods
Failed to tie property velocities/h-dot-fps to object methods
...
...

So something is going wrong inside of or just after Modello creato when you start to see lots of Failed to tie property error messages, the vast majority not being engine related. So there is something else going wrong to cause this.

[emanuelegreco29]

Also what happens if you move this line of code:

self.sim = create_fdm(JSBSIM_ROOT)

And make it the first line of code inside _setup_simulation()'?

This did work, now I just need to prevent the execution to print out the wall of text each time the FDM is created.

[seanmcleod70]

prevent the execution to print out the wall of text each time the FDM is created

jsbsim/tests/JSBSim_utils.py

Lines 30 to 31 in 13008f3

	# Hides startup and debug messages
	jsbsim.FGJSBBase().debug_lvl = 0