ePIC streaming computing testbed: the DAQ agent simulator

---

Disclaimer

The comms part is currently work in progress, it's a stub not ready for real use.

About

• Work Title: swf-daqsim-agent
• Purpose: To simulate the state machine of the ePIC DAQ, including its components interacting with the data management, prompt processing and ast monitoring systems.
• Tools: We use SimPy which appears to be well suited for the task. It's a discrete event simulation framework written in Python.

In terms of interaction with other components, the two principal modes taking place simultaneously are:

• Mock-up data representing the STF files
• Messages sent to other agents via MQ, to trigger the overall orchestration of data distribution and processing.

The working version of the file names is as follows:

swf.20250625.<integer>.<state>.<substate>.stf

Regarding the Python dependencies, they are captured in the requirements file in this repository.

---

The Simulation

At the time of writing, the prototype aimulation driver script sim_test.py is located in the folder test. It has equipped with a comprehensive set of CLI options. The "--help" option will output the available parameters.

Time handling

We make use of the real time features of SimPy. The default time unit is 1.0s, i.e. that's what will determine the speed of actual execution that includes a variety of delays and timeouts defined in the units of time. There is a way to speed up the simulation or slow it down, by introducing a time scaling factor. In this package, the time "starts" when the main simulation class is instantiated, corresponding to 0.0 on the time axis.

The Schedule

The critical part of the simulation is the process of state transitions in the DAQ. For simulation purposes, we define "schedule" as a list of points on the timeline, with assigned states (and possibly sub-states to be added later). The points are defined as tuples of (weeks, days, hours, minutes, seconds) for ease of human interaction but internally these data are converted to seconds (as floats). A dedicated method in the simulator class keeps watch of the states and actuates transitions.

States and Substates

Please see the README in the daq package folder: https://github.com/BNLNPPS/swf-daqsim-agent/tree/main/daq#states-substates

Communications

This agent is using ActieMQ to send notifications to the swf-data-agent and other elements of the test bed. The Python package stomp-py needs to be installed to support the current version of this interface.

For the ActiveMQ communications to work, the following evironment variables need to be set

• MQ_PASSWD
• MQ_USER
• MQ_CAFILE

There are two types of messages: the run status messages (start/stop), and STF generation messages, notifying the system that a STF has been created.

Run Status Messages

These messages carry the unique run ID and the timestamp. In current design, we opted for using the string representing the start of the run timestamp as its unique ID, for better readbility. This is an acceptable solution because in all realistic scenarios the run manager is always a singleton.

Examples:

{"msg_type": "start_run", "req_id": 1, "run_id": "20250718205015", "ts": "20250718205015"}
{"msg_type": "stop_run",  "req_id": 1, "run_id": "20250718205015", "ts": "20250718205017"}

The timestamp convention is %Y%m%d%H%M%S. This is different from the timestamp format in the STF message (below) which needs more granularity.

STF Generation Message

The STF generation message carries an attribute specifying the run ID, to simplify accounting and adata management procedures. It also contains attributes relevant to data handling downstream, such as the checksum (adler32) and size, in bytes.

The format of the messages sent out to MQ by the simulator is illustrated in the following example:

{
    "run_id":   "20250815161947",
    "state":    "run",
    "substate": "physics",
    "filename": "swf.20250815.161953.306782.run.physics.stf",
    "start":    "20250815161950088492",
    "end":      "20250815161953306782",
    "checksum": "ad:1563439560",
    "size":     189,
    "msg_type": "stf_gen",
    "req_id": 1
}

The last two elements in this dictionary are added on top of the metadata generated for each simulated STF file, so the content above these trailing two is identical between the metadata and the MQ message.

The file metadata is formed using this piece of Python code, presented here to elucidate the metadata format. This is from a method of the generator:

md ={
    'run_id':       self.run_id,
    'state':        self.state,
    'substate':     self.substate,
    'filename':     filename,
    'start':        start.strftime("%Y%m%d%H%M%S%f"),
    'end':          end.strftime("%Y%m%d%H%M%S%f")
}

The start and end attributes relate to the start of the STF generation and its end. Microsecond precision is used to avoid overlaps and clashes.