You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Grigori Fursin, the cTuning foundation and OctoML donated this project to MLCommons to benefit everyone.
3
+
Grigori Fursin, the cTuning foundation and OctoML donated the CK and CM projects to MLCommons to benefit everyone and continue development as a community effort.
Prepared MLCommons proposal for the creation of the [MLCommons Task Force on Automation and Reproducibility](https://github.com/mlcommons/ck/blob/master/docs/taskforce.md),
27
+
aimed at fostering community-driven support for CK and CM developments to benefit everyone.
28
+
29
+
* Jan, 2022: hired Arjun Suresh at OctoML to support and maintain CK1 framework and help prepare OctoML's MLPerf submissions using CK1.
30
+
Meanwhile, transitioned to focusing on CM and CM-MLOps development, building upon the prototypes created in 2021.
31
+
26
32
* Mar 1, 2022: started developing cm-mlops: [ref](https://github.com/octoml/cm-mlops/commit/0ae94736a420dfa84f7417fc62d323303b8760c6).
27
33
28
34
* Mar 24, 2022: after successfully stabilizing the initial prototype of CM, donated it to MLCommons to benefit the entire community:
@@ -55,7 +61,7 @@ aimed at benefiting the broader community and fostering its ongoing development
Welcomed Arjun Suresh as a contributor to CM: [ref](https://github.com/mlcommons/ck/blob/2c5d5c5c944ae5f252113c62af457c7a4c5e877a/CONTRIBUTING.md#contributors-in-alphabetical-order).
84
+
Welcomed Arjun Suresh as a contributor to CM automations for MLPerf: [ref](https://github.com/mlcommons/ck/blob/2c5d5c5c944ae5f252113c62af457c7a4c5e877a/CONTRIBUTING.md#contributors-in-alphabetical-order).
79
85
80
86
* From September 2022: coordinated community development of CM and CM4MLOps
81
-
to [modularize and automate MLPerf](https://docs.mlcommons.org/inference)
87
+
to [modularize and automate MLPerf benchmarks](https://docs.mlcommons.org/inference)
82
88
and support [reproducibility initiatives at ML and Systems conferences](https://cTuning.or/ae)
83
-
through the MLCommons Task Force on Automation and Reproducibility.
89
+
through the [MLCommons Task Force on Automation and Reproducibility](https://github.com/mlcommons/ck/blob/master/docs/taskforce.md).
90
+
91
+
* Directed and financed the creation of (CM) automations to streamline the MLPerf power measurement processes.
92
+
93
+
* Proposed to use MLPerf benchmarks for the Student Cluster Competition, led the developments
94
+
and prepared a tutorial to run MLPerf inference at SCC'22 via CM: [ref](https://github.com/mlcommons/ck/blob/master/docs/tutorials/sc22-scc-mlperf.md)
95
+
96
+
* April 2023: departed OctoML to focus on the development of the [CK playground](https://access.cKnowledge.org) and CM automations
97
+
to make Mlperf accessible to everyone. Hired Arjun Suresh to help with developments.
98
+
99
+
* Initiated and funded development of the [MLPerf explorer](https://github.com/ctuning/q2a-mlperf-visualizer)
100
+
to improve visualization of results
101
+
102
+
* August 2023: organized the 1st mass-scale MLPerf community submission of 12217 inference benchmark v3.1 results
103
+
out of total 13351 results (including 90% of all power results) across diverse models, software and hardware
104
+
from different vendors via [open challenges](https://access.cknowledge.org/playground/?action=challenges) funded by cTuning.org :
with results visualized by the [MLPerf explorer](https://github.com/ctuning/q2a-mlperf-visualizer),
107
+
[CM4MLOps challenges at GitHub](https://github.com/mlcommons/cm4mlops/tree/main/challenge).
108
+
109
+
* February, 2024: proposed to use CM to automate [MLPerf automotive benchmark (ABTF)](https://mlcommons.org/working-groups/benchmarks/automotive/).
110
+
111
+
* moved my prototypes of the CM automation for ABTF to cm4abtf repo: [ref](https://github.com/mlcommons/cm4abtf/commit/f92b9f464de89a38a4bde149290dede2d94c8631)
112
+
* led further CM4ABTF developments funded by cTuning.org.
84
113
85
114
* Starting in April 2024, began the gradual transfer of ongoing maintenance and enhancement
86
115
responsibilities for CM and CM4MLOps, including MLPerf automations, to MLCommons.
87
116
Welcomed Anandhu Sooraj as a maintainer and contributor to CM4MLOps with MLPerf automations.
88
117
89
-
For more details, please refer to this [white paper](https://arxiv.org/abs/2406.16791)
90
-
and [ACM REP'23 keynote](https://doi.org/10.5281/zenodo.8105339).
118
+
* Took a break from all development activities.
119
+
120
+
* July 2024: started prototyping the next generation of CM (CMX and CMX4MLOps) with simpler interfaces
121
+
based on user feedback while maintaining backward compatibility.
122
+
123
+
* 2025: continue developing CMX and CMX4MLOPs to make it easier to run and customize MLPerf inference, training
124
+
and other benchmarks across diverse models, datasets, software and hardware.
91
125
126
+
For more details, please refer to the [white paper](https://arxiv.org/abs/2406.16791)
127
+
and the [ACM REP'23 keynote](https://doi.org/10.5281/zenodo.8105339).
*[CK, CM and CMX history](https://github.com/mlcommons/ck/tree/master/docs/history.md)
164
+
*History: [CK](https://github.com/mlcommons/ck/tree/master/docs/history.md), [CM and CM automations for MLOps and MLPerf](https://github.com/mlcommons/ck/blob/master/HISTORY.CM.md)
153
165
154
166
155
167
### Acknowledgments
156
168
157
-
The open-source Collective Knowledge project (CK, CM, CM4MLOps/CM4MLPerf,
158
-
CM4Research and CMX) was created by [Grigori Fursin](https://cKnowledge.org/gfursin)
169
+
This open-source project was created by [Grigori Fursin](https://cKnowledge.org/gfursin)
159
170
and sponsored by cTuning.org, OctoAI and HiPEAC.
160
171
Grigori donated CK to MLCommons to benefit the community
161
172
and to advance its development as a collaborative, community-driven effort.
@@ -164,3 +175,6 @@ We thank [MLCommons](https://mlcommons.org), [FlexAI](https://flex.ai)
164
175
and [cTuning](https://cTuning.org) for supporting this project,
165
176
as well as our dedicated [volunteers and collaborators](https://github.com/mlcommons/ck/blob/master/CONTRIBUTING.md)
166
177
for their feedback and contributions!
178
+
179
+
If you found the CM automations helpful, kindly reference this article:
Grigori Fursin, the cTuning foundation and OctoML donated the CK and CM projects to MLCommons to benefit everyone and continue development as a community effort.
0 commit comments