Open tools and methodologies for collaborative, reproducible, and efficient R&D
cTuning.org is a non-profit educational organization, a founding member of MLCommons, and a long-standing ACM collaborator. Founded by Grigori Fursin in 2008 during the MILEPOST project and later registered as the cTuning Foundation, cTuning.org has helped pioneer community-driven approaches to reproducible research, artifact evaluation, collaborative benchmarking, workflow and agent-based automation, knowledge sharing, and efficient software–hardware co-design to evaluate, run, optimize, and co-design AI, ML, computing systems, and other emerging workloads more efficiently and cost-effectively.
Our mission is to make R&D more collaborative, reproducible, reusable, and trustworthy across computing systems, AI, machine learning, and emerging workloads. We develop open methodologies, automation tools, and community practices that help students, researchers, engineers, and organizations accelerate scientific discovery and innovation, reduce cost and complexity, and avoid common technical, methodological, and organizational pitfalls.
You can learn more about our history, vision, community initiatives, open-source developments, and related projects through our ACM TechTalk'21, keynote at ACM REP'23, joint Nature article'23, journal article in Philosophical Transactions of the Royal Society'21 and white paper'24. Follow cTuning Foundation updates on LinkedIn and related updates on X.
Current community activities
- Maintaining our Collective Knowledge Playground (a new version powered by cMeta is being developed by Grigori Fursin in collaboration with cTuning Labs) - an educational community project to help students, researchers, and engineers learn how to run, benchmark, optimize, and co-design AI, ML, and other emerging workloads in a more efficient, energy-aware, and cost-effective way across diverse models, datasets, software, and hardware.
- Supporting MLCommons and MLPerf automation initiatives to make AI systems benchmarking easier to run, reproduce, customize, and optimize across diverse models, datasets, software, and hardware.
- Helping ACM and IEEE conferences improve Artifact Evaluation and make it easier to validate results from published papers in the real world across continuously changing models, data, software, and hardware.
- Unifying and automating the Artifact Appendix and reproducibility checklist across AI, ML, and computing systems conferences.
- Organizing educational challenges, reproducibility initiatives, and community experiments to help students, researchers, and engineers learn how to co-design more efficient, energy-aware, and cost-effective software and hardware for emerging workloads.
Selected impact
- Helped ACM develop a common methodology for artifact review and badging and set up the Emerging Interest Group on Reproducibility and Replicability.
- Helped ACM and IEEE conferences organize 20+ reproducibility challenges and artifact evaluations.
- Helped MLCommons establish the Task Force on Automation and Reproducibility and extend Collective Knowledge technology to run MLPerf benchmarks out of the box across diverse software and hardware, from cloud to edge.
- Helped the community submit 10,000+ MLPerf inference results.
- Helped students, researchers, and practitioners learn best practices for collaborative and reproducible research through the ACM TechTalk'21, keynote at ACM REP'23, and white paper'24.
Legacy cTuning/MLCommons Collective Knowledge and Collective Mind technology
An overview of our legacy open-source Collective Knowledge (CK) and Collective Mind (CM) technology, developed with the community and MLCommons to run, reproduce, benchmark, and co-design AI, ML, and computing systems across diverse models, datasets, software, and hardware.
Sponsors
