A Modular Machine Learning Framework for Industrial Tasks

[Dashboard] [Examples & Tutorials]

---

🎉 Welcome to Neuralk Foundry

Neuralk Foundry is a lightweight yet powerful framework for building modular machine learning pipelines — particularly well-suited for industrial tasks and representation learning. Whether you're prototyping or scaling up, Foundry helps you build, combine, and orchestrate steps cleanly and efficiently.

Foundry is also the engine behind TabBench, Neuralk's internal benchmark for evaluating ML models on real-world tabular datasets.

Why Foundry?

Most ML frameworks fall into one of two camps:

• Rigid benchmarks and academic pipelines: great for simple supervised learning tasks, but brittle or limited when adapting to more complex use cases.
• Heavyweight MLOps frameworks (e.g., ZenML, Metaflow): offer full orchestration but at the cost of steep setup and reduced flexibility.

Foundry sits in between. It gives you just the right level of structure to scale from prototype to production — without locking you into opinionated tooling.

---

🚀 Key Features

Composable Workflows : Define steps in terms of their inputs and outputs — no black boxes.

Supports Heterogeneous Tasks : Classification, regression, ranking, record linkage, and more.

Customizable & Extensible : Plug in your own logic or replace any step with a variant.

Built-in Caching & Logging : Avoid recomputation and keep track of metrics automatically.

Workflow Explorer UI : Inspect and debug workflows through an interactive, visual interface.

Reproducibility by Design : Strong separation between configuration, code, and data.

---

🧠 How Things Are Organized

Foundry is a modular framework. Its codebase is split into submodules that reflect each phase of the ML pipeline:

neuralk_foundry_ce/
├── datasets/               # Dataset loading utilities
├── sample_selection/
│   ├── splitter/           # Data splitting strategies (e.g., stratified shuffle)
│   └── blocking/           # Candidate pair selection (e.g., for deduplication)
├── feature_engineering/
│   ├── preprocessing/      # Traditional preprocessing for tabular data
│   ├── vectorizer/         # Text and other unstructured data vectorization
│   └── blocking/           # Pair processing modules for matching/merging
├── models/
│   ├── classifier/         # Classification models
│   ├── regressor/          # Regression models
│   ├── embedder/           # Embedding/representation learning
│   └── clustering/         # Clustering and unsupervised methods
├── workflow/               # Core execution engine: Step, Workflow, etc.
└── utils/                  # Helper functions and shared infrastructure

Each component (e.g., a model or preprocessing step) inherits from a base Step class and declares:

• Its expected inputs
• The outputs it produces
• Any configurable parameters

Steps can then be connected into a Workflow, either manually or through a task-specific template (e.g., Classification).

---

⚙️ Quick-Start Installation

Install the package from PyPI:

pip install neuralk_foundry_ce

🔬 Development Installation

Clone the Repository

git clone https://github.com/Neuralk-AI/NeuralkFoundry-CE
cd NeuralkFoundry-CE

Create a Dedicated Environment (recommended)

Neuralk Foundry relies on a variety of external machine learning libraries. As a result, managing package versions can be delicate. To avoid compatibility issues, we strongly recommend installing Foundry in a dedicated virtual environment (e.g., using conda or venv).

conda create -n foundry python=3.11
conda activate foundry

Install the Package

pip install -e .

Examples and tutorials

• Getting Started with Neuralk Foundry A gentle introduction to the framework and how to run your first workflow.

• Three Levels of Workflows Understand how Foundry supports simple pipelines, reusable workflows, and specialized task flows.

• Use a Custom Model Learn how to plug in and use your own ML model within a Foundry pipeline.

Citing Foundry

If you incorporate any part of this repository into your work, please reference it using the following citation:

@article{neuralk2025foundry,
         title={Foundry: A Modular Machine Learning Framework for Industrial Tasks}, 
         author={Neuralk-AI},
         year={2025},
         publisher = {GitHub},
         howpublished = {\url{https://github.com/Neuralk-AI/NeuralkFoundry-CE}},
}

Contact

If you have any questions or wish to propose new features please feel free to open an issue or contact us at alex@neuralk-ai.com.

For collaborations please contact us at antoine@neuralk-ai.com.