Skip to content

Commit 80959de

Browse files
ctuning-adminctuning-admin
authored
V3.5.1 (#1372)
## V3.5.1 - added Collective Mind architecture diagram: https://github.com/mlcommons/ck/tree/master/docs/specs/cm-diagram-v3.5.1.png - added tabulate as requirement based on user feedback - updated white paper: https://arxiv.org/abs/2406.16791 - merged Anandu's docs update - added links to archived repositories to keep development history (ck, cm-mlops, etc)
2 parents d4899a9 + dfc74a6 commit 80959de

File tree

11 files changed

+241
-63
lines changed

11 files changed

+241
-63
lines changed

‎README.md

Lines changed: 95 additions & 43 deletions
Original file line numberDiff line numberDiff line change
@@ -9,76 +9,126 @@
99
[![MLPerf inference resnet50](https://github.com/mlcommons/ck/actions/workflows/test-cm-mlperf-inference-resnet50.yml/badge.svg)](https://github.com/mlcommons/ck/actions/workflows/test-cm-mlperf-inference-resnet50.yml)
1010
[![CMX: image classification with ONNX](https://github.com/mlcommons/ck/actions/workflows/test-cmx-image-classification-onnx.yml/badge.svg)](https://github.com/mlcommons/ck/actions/workflows/test-cmx-image-classification-onnx.yml)
1111

12-
### About
12+
## Collective Knowledge
1313

1414
[Collective Knowledge (CK, CM, CM4MLOps, CM4MLPerf and CMX)](https://cKnowledge.org)
1515
is an educational community project to learn how to run AI, ML and other emerging workloads
1616
in the most efficient and cost-effective way across diverse models, data sets, software and hardware.
17+
It includes the following sub-projects.
1718

18-
CK consists of several sub-projects:
19+
### Collective Mind (CM)
1920

20-
* [Collective Mind framework (CM)](cm) - a very lightweight Python-based framework with minimal dependencies
21-
intended to help researchers and engineers automate their repetitive, tedious and time-consuming tasks
22-
to build, run, benchmark and optimize AI, ML and other applications and systems
23-
across diverse and continuously changing models, data, software and hardware.
21+
[Collective Mind (CM)](https://github.com/mlcommons/ck/tree/master/cm) -
22+
a very lightweight Python-based framework with a unified CLI, Python API and minimal dependencies
23+
intended to help researchers and engineers automate their repetitive, tedious and time-consuming tasks
24+
to build, run, benchmark and optimize AI, ML and other applications and systems
25+
across diverse and continuously changing models, data, software and hardware.
2426

25-
* [CM4MLOPS](https://github.com/mlcommons/cm4mlops) -
26-
a collection of portable, extensible and technology-agnostic automation recipes
27-
with a common CLI and Python API (CM scripts) to unify and automate
28-
all the manual steps required to compose, run, benchmark and optimize complex ML/AI applications
29-
on diverse platforms with any software and hardware: see [online catalog at CK playground](https://access.cknowledge.org/playground/?action=scripts),
30-
[online MLCommons catalog](https://docs.mlcommons.org/cm4mlops/scripts)
27+
Collective Mind is continuously enhanced through public and private CM4* Git repositories,
28+
which serve as the unified interface for various collections of reusable automations and artifacts.
3129

32-
* [CM interface to run MLPerf inference benchmarks](https://docs.mlcommons.org/inference)
30+
The CM architecture diagram is available for viewing
31+
[here](https://github.com/mlcommons/ck/tree/master/docs/specs/cm-diagram-v3.5.1.png).
3332

34-
* [CM4ABTF](https://github.com/mlcommons/cm4abtf) - a unified CM interface and automation recipes
35-
to run automotive benchmark across different models, data sets, software and hardware from different vendors.
33+
### Collective Mind repositories
3634

37-
* [CMX (the next generation of CM, CM4MLOps and CM4MLPerf)](cm/docs/cmx) -
38-
we are developing the next generation of CM
39-
to make it simpler and more flexible based on user feedback. Please follow
40-
this project [here]( https://github.com/orgs/mlcommons/projects/46 ).
35+
#### CM4MLOps
4136

37+
[CM4MLOPS repository powered by CM](https://github.com/mlcommons/cm4mlops) -
38+
a collection of portable, extensible and technology-agnostic automation recipes
39+
with a common CLI and Python API (CM scripts) to unify and automate
40+
all the manual steps required to compose, run, benchmark and optimize complex ML/AI applications
41+
on diverse platforms with any software and hardware.
4242

43-
* [Collective Knowledge Playground](https://access.cKnowledge.org) - a unified platform
44-
to list CM scripts similar to PYPI, aggregate AI/ML Systems benchmarking results in a reproducible format with CM workflows,
45-
and organize [public optimization challenges and reproducibility initiatives](https://access.cknowledge.org/playground/?action=challenges)
46-
to co-design more efficient and cost-effiective software and hardware for emerging workloads.
43+
The two key automations are *script" and *cache*:
44+
see [online catalog at CK playground](https://access.cknowledge.org/playground/?action=scripts),
45+
[online MLCommons catalog](https://docs.mlcommons.org/cm4mlops/scripts).
4746

48-
* [CM4MLPerf-results](https://github.com/mlcommons/cm4mlperf-results) -
49-
a simplified and unified representation of the past MLPerf results
50-
for further visualization and analysis using [CK graphs](https://access.cknowledge.org/playground/?action=experiments)
51-
(*the new version is coming soon*).
47+
CM scripts extend the concept of `cmake` with simple Python automations, native scripts
48+
and JSON/YAML meta descriptions. They require Python 3.7+ with minimal dependencies and are
49+
[continuously extended by the community and MLCommons members](https://github.com/mlcommons/ck/blob/master/CONTRIBUTING.md)
50+
to run natively on Ubuntu, MacOS, Windows, RHEL, Debian, Amazon Linux
51+
and any other operating system, in a cloud or inside automatically generated containers
52+
while keeping backward compatibility.
5253

54+
See the [online documentation](https://docs.mlcommons.org/inference)
55+
at MLCommons to run MLPerf inference benchmarks across diverse systems using CM.
5356

54-
* [Artifact Evaluation](https://cTuning.org/ae) - automating artifact evaluation and reproducibility initiatives at ML and systems conferences.
57+
#### CM4ABTF
5558

56-
### Deprecated and archived projects
59+
[CM4ABTF repository powered by CM](https://github.com/mlcommons/cm4abtf) -
60+
a collection of portable automations and CM scripts to run the upcoming
61+
automotive MLPerf benchmark across different models, data sets, software
62+
and hardware from different vendors.
5763

58-
* [CM-MLOps](cm-mlops) - now [CM4MLOps](cm4mlops)
59-
* [CK automation framework v1 and v2](ck) - now [CM](cm)
64+
#### CM4MLPerf-results
6065

66+
[CM4MLPerf-results powered by CM](https://github.com/mlcommons/cm4mlperf-results) -
67+
a simplified and unified representation of the past MLPerf results
68+
in the CM format for further visualization and analysis using [CK graphs](https://access.cknowledge.org/playground/?action=experiments).
6169

62-
### License
70+
#### CM4Research
71+
72+
[CM4Research repository powered by CM](https://github.com/ctuning/cm4research) -
73+
a unified interface designed to streamline the preparation, execution, and reproduction of experiments in research projects.
74+
75+
76+
### Projects powered by Collective Mind
77+
78+
#### Collective Knowledge Playground
79+
80+
[Collective Knowledge Playground](https://access.cKnowledge.org) -
81+
a unified and open-source platform designed to [index all CM scripts](https://access.cknowledge.org/playground/?action=scripts) similar to PYPI,
82+
assist users in preparing CM commands to:
83+
84+
* [run MLPerf benchmarks](https://access.cknowledge.org/playground/?action=howtorun)
85+
* aggregate, process, visualize, and compare [benchmarking results](https://access.cknowledge.org/playground/?action=experiments) for AI and ML systems
86+
* organize [open, reproducible optimization challenges and tournaments](https://access.cknowledge.org/playground/?action=challenges).
87+
88+
These initiatives aim to help academia and industry
89+
collaboratively enhance the efficiency and cost-effectiveness of AI systems.
90+
91+
#### Artifact Evaluation
92+
93+
[Artifact Evaluation automation](https://cTuning.org/ae) - a community-driven initiative
94+
leveraging the Collective Mind framework to automate artifact evaluation
95+
and support reproducibility efforts at ML and systems conferences.
96+
97+
98+
## Incubator
99+
100+
[CMX](https://github.com/mlcommons/ck/tree/master/cmx) - the next evolution of the Collective Mind framework,
101+
designed to enhance simplicity, flexibility, and extensibility of automations
102+
based on user feedback. Follow the project's progress [here]( https://github.com/orgs/mlcommons/projects/46 ).
103+
104+
105+
## Archived projects
106+
107+
* [CM-MLOps](https://github.com/mlcommons/ck/tree/master/cm-mlops) - now [CM4MLOps](https://github.com/mlcommons/ck/tree/master/cm4mlops)
108+
* [CK automation framework v1 and v2](https://github.com/mlcommons/ck/tree/master/ck) - now [CM](https://github.com/mlcommons/ck/tree/master/cm)
109+
110+
111+
## License
63112

64113
[Apache 2.0](LICENSE.md)
65114

66-
### Copyright
115+
## Copyright
67116

68117
* Copyright (c) 2021-2024 MLCommons
69118
* Copyright (c) 2014-2021 cTuning foundation
70119

71-
### Author
120+
## Author
72121

73122
* [Grigori Fursin](https://cKnowledge.org/gfursin) (FlexAI, cTuning)
74123

75-
### Maintainers
124+
## Maintainers
125+
126+
* Collective Mind (CM): [Grigori Fursin](https://cKnowledge.org/gfursin)
127+
* CM4MLOps repository: [Arjun Suresh](https://github.com/arjunsuresh) and [Anandhu Sooraj](https://github.com/anandhu-eng)
128+
* CMX (the next generation of CM and CM4MLOps): [Grigori Fursin](https://cKnowledge.org/gfursin)
76129

77-
* [Collective Mind (CM)](cm): [Grigori Fursin](https://cKnowledge.org/gfursin)
78-
* CM4MLOps (CM automation recipes): [Arjun Suresh](https://github.com/arjunsuresh) and [Anandhu Sooraj](https://github.com/anandhu-eng)
79-
* CMX (the next generation of CM, CM4MLOps and CM4MLPerf): [Grigori Fursin](https://cKnowledge.org/gfursin)
80130

81-
### Citing our project
131+
## Citing Collective Mind and Collective Knowledge
82132

83133
If you found the CM automation framework helpful, kindly reference this article:
84134
[ [ArXiv](https://arxiv.org/abs/2406.16791) ], [ [BibTex](https://github.com/mlcommons/ck/blob/master/citation.bib) ].
@@ -91,17 +141,19 @@ To learn more about the motivation behind CK and CM technology, please explore t
91141
* Journal of Royal Society'20: [ [paper](https://royalsocietypublishing.org/doi/10.1098/rsta.2020.0211) ]
92142

93143

94-
### CM Documentation
144+
## CM Documentation
95145

96-
* [CM installation GUI](https://access.cknowledge.org/playground/?action=install)
146+
* [Collective Mind white paper](https://arxiv.org/abs/2406.16791)
147+
* [CM/CMX architecture](https://github.com/mlcommons/ck/tree/master/docs/specs/cm-diagram-v3.5.1.png)
148+
* [CM/CMX installation GUI](https://access.cknowledge.org/playground/?action=install)
97149
* [CM Getting Started Guide and FAQ](https://github.com/mlcommons/ck/tree/master/docs/getting-started.md)
98150
* [Common CM interface to run MLPerf inference benchmarks](https://github.com/mlcommons/ck/tree/master/docs/mlperf/inference)
99151
* [Common CM interface to re-run experiments from ML and Systems papers including MICRO'23 and the Student Cluster Competition @ SuperComputing'23](https://github.com/mlcommons/ck/tree/master/docs/tutorials/common-interface-to-reproduce-research-projects.md)
100-
* [CM automation recipes for MLOps and DevOps](https://access.cknowledge.org/playground/?action=scripts)
152+
* [CM4MLOps automation recipes for MLOps and DevOps](https://access.cknowledge.org/playground/?action=scripts)
101153
* [Other CM tutorials](https://github.com/mlcommons/ck/tree/master/docs/tutorials)
102154
* [Full documentation](https://github.com/mlcommons/ck/tree/master/docs/README.md)
103155
* [CM taskforce](https://github.com/mlcommons/ck/tree/master/docs/taskforce.md)
104-
* [CMX, CM and CK history](https://github.com/mlcommons/ck/tree/master/docs/history.md)
156+
* [CK, CM and CMX history](https://github.com/mlcommons/ck/tree/master/docs/history.md)
105157

106158

107159
### Acknowledgments

‎cm/CHANGES.md

Lines changed: 8 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -1,5 +1,11 @@
1-
## V3.4.4.1
2-
- added tabulate as requirement
1+
## V3.5.1
2+
- added Collective Mind architecture diagram:
3+
https://github.com/mlcommons/ck/tree/master/docs/specs/cm-diagram-v3.5.1.png
4+
- added tabulate as requirement based on user feedback
5+
- updated white paper: https://arxiv.org/abs/2406.16791
6+
- merged Anandu's docs update
7+
- added links to archived repositories to keep development history
8+
(ck, cm-mlops, etc)
39

410
## V3.4.4
511
- improved error reporting in utils.load_json and utils.load_yaml

0 commit comments

Comments
 (0)