The Modular Autonomous Discovery for Science (MADSci) Framework

Experiment Control Flow Using MADSci

Overview

MADSci is a modular, autonomous, and scalable framework for scientific discovery and experimentation. It aims to provide:

• Laboratory Instrument Automation and Integration via the MADSci Node standard. Developers can implement device-specific Node modules in any language that can then be integrated into a MADSci system using a common interface standard (currently supports REST-based HTTP communication)
• Workflow Management, allowing users to define and run flexible scientific workflows that can leverage one or more Nodes to complete complex tasks.
• Experiment Management, conducting flexible closed loop autonomous experiments by combining multiple workflow runs, as well as any compute, decision making, data collection, and analysis as needed.
• Resource Management, allowing robust tracking of all the labware, consumables, equipment, samples, and assets used in an autonomous laboratory.
• Event Management, enabling distributed logging and event handling across every part of the autonomous lab.
• Data Management, collecting and storing data created by instruments or analysis as part of an experiment.

Diagram of a MADSci Laboratory's Infrastructure

Notes on Stability

MADSci is currently in beta. Most of the core functionality is working and tested, but there may be bugs or stability issues (if you run into any, please open an issue so we can get it fixed). New releases will likely include breaking changes, so we recommend pinning the version in your dependencies and upgrading only after reviewing the release notes.

Documentation

MADSci is made up of a number of different modular components, each of which can be used independently to fulfill specific needs, or composed to build more complex and capable systems. Below we link to specific documentation for each system component.

• Common: the common types and utilities used across the MADSci toolkit
• Clients: A collection of clients for interacting with different components of MADSci
• Event Manager: handles distributed event logging and querying across a distributed lab.
• Workcell Manager: handles coordinating and scheduling a collection of interoperating instruments, robots, and resources using Workflows.
• Experiment Manager: manages experimental runs and campaigns across a MADSci-powered lab.
• Resource Manager: For tracking labware, assets, samples, and consumables in an automated or autonomous lab.
• Data Manager: handles capturing, storing, and querying data, in either JSON value or file form, created during the course of an experiment (either collected by instruments, or synthesized during anaylsis)
• Squid Lab Manager: a central lab configuration manager and dashboard provider for MADSci-powered labs.

Python

We provide python packages via the Python Package Index for each of the MADSci system components. See the individual components for information about how to install and use their corresponding python package.

Docker

We provide docker images to make containerizing and orchestrating your labs as easy and robust as possible. Currently, we provide:

• ghcr.io/ad-sdl/madsci: base docker image with the full set of MADSci python packages installed and available. You can use this image as a base for your own Manager, Experiment Application, or Node docker containers.
• ghcr.io/ad-sdl/madsci_dashboard: extends the base image to add the web-based dashboard's prebuilt static files. Should be used for running your Squid Lab Manager if you want to host the MADSci Dashboard.

Roadmap

We're working on bringing the following additional components to MADSci:

• Auth Manager: For handling authentication and user and group management for an autonomous lab.
• Transfer Manager: For coordinating resource movement in a lab.

Getting Started with MADSci

To get started with MADSci, we recommend checking out our MADSci Examples Repository. There you'll find Jupyter Notebooks walking through the core concepts and how to leverage MADSci to build your autonomous laboratory and experiments.

Developer Guide

Environment Setup

Some notes for setting up a development environment for the MADSci core packages:

• Clone the Repository
  • If you want to contribute back, or have your own version of the code, consider Forking the Repository first
• Install Python
  • We use PDM to make installing the madsci python packages and managing dependencies easier
  • Alternatively, we recommend using Virtual Environments
• To build and/or use the MADSci docker containers, install Docker
  • You only need the Docker Container Engine, which can be installed standalone on Linux; on Mac or Windows, you'll need to install Docker Desktop or an alternative
  • If your organization doesn't allow Docker Desktop, or if you simply prefer an open source alternative, we recommend Rancher Desktop or Podman
• We use the just command runner for developer helper commands
  • To see all available commands, run just list or look at the justfile (useful even if you don't want to use just)
• If you want to make changes to the Squid Dashboard, install npm
  • Optionally, install yarn for faster package installation
• We use pre-commit to run automated linting, formatting, and other checks for code quality and consistency

If you use an IDE like Visual Studio Code and want a quick way to bootstrap your environment, consider taking advantage of the included Dev Container. See Visual Studio Code Dev Container Documentation or the Development Container Documentation for more information on how to get started with Dev Containers in your IDE of choice.

Running Automated Tests

1. Install the python packages locally using PDM or pip (see above)
2. Ensure you've activated any relevant virtual environments
3. Run pytest in the root directory of the repository
   1. To run only the tests for a specific component, such as the workcell manager, change directories into the relevant subdirectory in src/, or use pytest -k EXPRESSION to filter the tests by test name/parent class
4. Note that many of the pytests depend on docker to start mock database containers (see above for installation notes)