continious Active Inference on the RxInfer platform

[Charelvanhoof]

Hi Team,

Thank you for our excellent meeting/discussion yesterday. Looking forward to see how we can make (continuous) active inference work on the RxInfer platform. It looks promising. I guess we might need to add some features to the platform to capture the specifics of continuous active inference.

As requested I opened a topic over here to get things going.

As discussed, as a first step, lets recreate the IWAI 2024 tutorial (Basic active inference capsule) on RxInfer. Here is the link: https://www.kaggle.com/code/charel/active-inference-example-iwai2024 (including working python code, so we can double check we get the same results).

The example is set in a simulation environment where an early evolutionary imaginary aquatic ancestor (Hydra vulgaris) must preserve its physical integrity to survive. For example, it needs to keep in a certain temperature range. It is a basic case of a single active inference capsule with one sensor, one hidden state and one causal state. And Hydar can take action to move up or down.

I propose to build the example step by step:

Step 1.1 Inference without generalized coordinates
Inference of the hidden state, causal state (without generalized coordinates). In the example the function of sensory mapping g is non-linear. If this is an extra challenge we can take a first step with a linear g function
Step 1.2 Add the action
Realtime/online inference including the inference of the reflex action u, which is input for the next timestep (it is not discrete/EFE active inference)
Step 1.3 Add the generalized coordinates
Need to express the states in generalized coordinates e.g [position, speed, acceleration,..] and inference all generalized coordinates. (see the generalized coordinates as a kind of short-term-prediction). Need to be able to generate colored noise and the precision matrix of the colored noise.

Thank for the help because I am new to RxInfer/Julia.

[bvdmitri]

@ThijsvdLaar @ofSingularMind do you want to collaborate on this?

[ThijsvdLaar]

@Charelvanhoof Thanks for opening this discussion and for sharing your notebooks, they are a very nice resource. I like the idea of an education-oriented narrative for explaining continuous-time active inference (CT AIF), with open-source notebooks that exemplify crucial design decisions in the framework.

@Charelvanhoof @ofSingularMind If you're interested, we can start to take CT AIF apart so to speak, and then explain the ingredients that make it tick. I think that alternative design choices for CT AIF can be made (along the way), and RxInfer can play a role here (even without much modification).

For example, I've been trying to implement Friston and Ao's 2012 paper on "Free Energy, Value and Attractors" with RxInfer, and see the following connections:

1. Desired dynamics are described by the generative model, which can be represented by a factor graph;
2. Veridical dynamics (depending on action) are not explicitly represented by a model, but by an equation (Friston & Ao, Eq. H.3) that directly computes the action gradient of the generative process. I see this equation as a result of inference on a (deterministic) "veridical" model that represents the function y(a). Then, this model can also be represented by a factor graph;
3. Generalized coordinates of motion can be included in these models. However, I think this is a design choice (namely, the one that leads to a biologically plausible inference scheme). In contrast, inference from an engineering viewpoint may lead to an (extended) Kalman filter, which (with or without generalized coordinates) can be implemented with RxInfer.

I've been meaning to collect examples of reactive AIF implementations into a public repository, https://github.com/biaslab/RxAIF, so others may learn and take inspiration (but I haven't gotten very far yet). I can already share some materials on CT AIF in this repo (throughout the coming weeks, because I'm currently on leave), and we can collaborate if you like.

[bertdv]

Hi all-- Thanks @Charelvanhoof for opening this discussion. Perhaps I can add my view on this. I think there are two different threads in this issue. One is the simulation of continuous-time dynamical systems, and the other is the simulation of active inference (AIF) processes. I recommend that we consider these issues separately.

In principle, all dynamical systems are simulated using discrete time steps, regardless of whether they are described on paper by differential equations (i.e., continuous-time systems) or by difference equations (i.e., discrete-time systems). The default in RxInfer is to define and simulate systems by difference equations. @Ismail Senoz has worked on the simulation of CT dynamical systems in RxInfer, which essentially comes down to simulating by adaptive time step durations. We can ask @ismailsenoz for more input here.

The simulation of an AIF system (agent + environment) is nothing but the simulation of a dynamical system. By itself, transitioning from discrete-time to continuous-time simulation (or vice versa) is conceptually unrelated to AIF—it's merely a change in the simulation paradigm. So, going forward, I recommend using difference equations for both model specification and for simulation of the inference process, as RxInfer far better supports this. Once that works, we can consider whether the dynamical AIF system can also be specified by differential equations and if the continuous-time inference process can be simulated in RxInfer using the techniques that Ismail has worked on.

As for the second thread, how to simulate (discrete-time) AIF processes in RxInfer, I would consider the following. The expected free energy (EFE) is not a variational free energy (VFE), but rather a cost function for policies. In almost all of the AIF literature, it is suggested to compute EFE for a "smartly selected" set of policy candidates (represented by: u) and then assign a prior "softmax(-G(u))" to those candidates. This is a method that (1) is not consistent with the FEP idea that FE minimization is the only driving process, and (2) does not scale well (since the "smartly selected"-issue leads to tree-search algorithms that explode for large-dimensional state spaces and/or deep horizons). I recommend following Theorem 1 in https://arxiv.org/abs/2504.14898, which reframes EFE-based planning as an emergent property of regular variational free energy (VFE) minimization in a generative model-rollout into the future, terminated by both target priors and epistemic priors. This approach is, in principle, directly realizable in RxInfer.

As an example, consider the slide below, which shows some key code fragments of multiple agents that need to move to known end points without collisions.

The code already includes a generative model and target priors. Inference in this model leads to the agents being moved by a "KL control" driving force. In order to turn this agent into an AIF agent, the code only needs to be extended by adding epistemic priors (eqs 9 in https://arxiv.org/abs/2504.14898). This is not entirely straightforward, but we are working on a nice user interface for RxInfer to support this. @Wouter Nuijten is the most informed person about this work. Going forward, I recommend syncing with @wouterwln on when and how epistemic priors are supported in RxInfer.

In summary, I recommend proceeding as follows:

1. Transfer the example of the Hydra vulgaris as an AIF agent to a discrete-time (DT) model specification (by difference equations). The outcome of this exercise should be a generative model (that specifies how the agent believes its observations will be generated as a function of its actions), and target priors (that specify the preferences over future states as a probability distribution).
2. Implement the DT version of the AIF agent in Python, and check that the behavior is similar to the CT version. (for this check, it is necessary to develop a few benchmark tests).
3. Code the generative model, including the epistemic and target priors, in RxInfer (following the example of the multi-agents above). This should be less than 1 page of code.
4. Run the inference process in RxInfer, and check that the Hydra vulgaris AIF agent in RxInfer behaves similarly (on the benchmark tests) as the AIF agent in Python.

Generally speaking, BIASlab and Lazy Dynamics members can help with tasks 3 and 4 if you encounter problems. In particular, @wouterwln is the expert in helping with task 3. @bvdmitri and other Lazy Dynamics members are experts if task 4 (the inference process) fails. Everybody else in BIASlab can also help, either by themselves or in finding appropriate help.

If the above works, then the same recipe can be used for all other agents. As discussed, the automatic simulation of continuous-time inference processes in RxInfer is a separate issue that needs the involvement of @ismailsenoz. I recommend not considering it until we have many AIF agents working smoothly by discrete-time simulations.