Public repositories of knowledge developed by the cTuning foundation

In 2015, we released the 4th version of our cTuning repository - a brand new, open-source, customizable Collective Knowledge Repository. It aggregates all our past developments, ideas and techniques, and allows users to share, cross-link and reference any object and knowledge (workloads, data sets, tools, optimization results, predictive models, etc.) as a reusable component with a unified JSON API via GitHub or BitBucket!

You can easily check out, reuse and improve the following workloads/artifacts and modules shared by the community! You can also find various results from experiment crowdsourcing in the CK live repository and even participate in crowd-tuning yourself.


With our background in physics and AI, and a practical need to get efficient computer systems to solve our tasks, we are shocked by the current state of computer engineering. It is rare to find publications with reproducible and statistically meaningful results, systematized knowledge, open-source data and tools. One of the main reason is that sharing of data and tools simply doesn't pay off while the main focus is to publish as many papers as possible and to share as little data or tools as possible to avoid competition. However, since our aim to build efficient computer systems in terms of performance, power and reliability is to continue innovation in science, since 1996 we started a long-term and painful process of systematization of knowledge about design and optimization of computer systems through collaborative open-source tools and repositories. The first public repository have been used to enable machine learning self-tuning compiler - users ('crowd') have been searching effective predictive models to explain the behavior of computer systems preserved in the cTuning database. These models have been integrated with the MILEPOST GCC compiler or could be used as plugins for dynamic program adaptation (UNIDAPT framework). Eventually, we initiated new research methodology and publication model where experimental results (data, tools, models) are continuously shared and validated by the community. More information is available here.

Public and in-house repositories of knowledge

# Year CK Description
[R1] 2017-01 - cur. Open repository of reusable, customizable, portable and optimized AI artifacts to accelerate AI research and boost innovation
  • Developed by the non-profit cTuning foundation and dividiti
  • Opened to public in 2017
  • Supports our 1st open, reproducible and Pareto efficient SW/HW co-design competition for deep learning (speed, accuracy, costs) at ACM ASPLOS [E3]
  • Powered by Collective Knowledge Framework [S2]
[ cKnowledge.org/repo ][ Portable, customizable and reusable AI CK components ][ Locator of CK-based AI artifacts (beta) ][ CK repositories ]
[R2] 2014-11 - cur. Collective Knowledge public repository (CK aka cTuning4) to continue improving whole experimental setup sharing (code, data, dependencies, experimental results, models) along with interactive articles
  • Developed by the non-profit cTuning foundation
  • Opened to public in 2015
  • Received award [A3]
  • Included all past and current semantically connected research artifacts from Grigori's research, development and experimentation (hundreds of codelets and benchmarks; thousands of datasets; GCC, LLVM, Open64, PathScale, ICC compiler optimization description; tools and scripts; online tuning plugins; machine learning plugins; adaptive exploration pluigns; graph plotting plugins; data mining plugins; machine learning based meta compiler; MILEPOST GCC, etc
  • Supports our initiatives on Artifact Evaluation and new publication models where results and papers are validated and improved by the community [M3]
  • Partially funded by [F3]
  • Powered by Collective Knowledge Framework [S2]
[ Collective Knowledge live repository ][ Examples of interactive graphs ][ Examples of interactive reports ]
[R3] 2011-09 - cur. Collective Mind public repository (cM aka cTuning3) to start collaborative systematization of analysis, design and optimization of computer systems based on extensible public repositories of knowledge, crowdsourcing, online tuning and machine learning, and to initiate new publication model where all research artifacts are continuously shared, validated and extended by the community
  • Opened to public in 2013
  • Included all past and current semantically connected research artifacts from Grigori's research, development and experimentation (hundreds of codelets and benchmarks; thousands of datasets; GCC, LLVM, Open64, PathScale, ICC compiler optimization description; tools and scripts; online tuning plugins; machine learning plugins; adaptive exploration pluigns; graph plotting plugins; data mining plugins; machine learning based meta compiler; MILEPOST GCC, etc
  • Connected with Android Collective Mind Node [S6] to crowdsource program and architecture characterization and multi-objective autotuning (execution time, code size, compilation time, power consumption) using any available Android-based mobile phone, tablet or laptop
  • Used for the new publication model [E25]
  • Funded by [A4, F4]
  • Powered by Collective Mind Framework [S3]
[ Collective Mind live repository ][ Online advice web service to predict optimizations based on features ][ Universal autotuning and learning pipeline for top-down multi-objective optimization ]
[R4] 2010-03 - 2011-08 In-house Codelet Tuning Repository for Intel Exascale Lab (aka cTuning2) to decompose large applications into codelets for continuous characterization and tuning
  • Developed with Grigori's team [Q6, Q4, Q3] as customizable repository for Intel Exascale Lab, CEA, GENCI and UVSQ (France)
  • Funded by [F5]
  • Powered by Codelet Tuning Infrastructure [S7]
  • Discontinued for [R3]
[R5] 2006-01 - cur. cTuning.org public repository (aka cTuning1) to start collaborative systematization of analysis, design and optimization of computer systems based on extensible public repositories of knowledge, crowdsourcing, online tuning and machine learning
  • Opened to public in 2008
  • Included past research experimentatal results on program and architecture multi-objective tuning (execution time, code size, compilation time, power consumption) for reproducibility and collaborative extensions
  • Connected with MILEPOST GCC [S10] for continuous and online training and improvement of the prediction models
  • Funded by [F9, F8, F7, F6, F4]
  • Powered by cTuning Framework [S11]
  • Gradually being discontinued for [R3]
[ cTuning live repository ][ Online advice web service to predict optimizations based on features ]
[R6] 2004-06 - 2006-06 In-house collaborative optimization repository for research on multi-objective program and architecture autotuning and co-design combined with machine learning
  • Powered by FCO framework [S15]
  • Funded by [A8, F8]
  • Discontinued for [R5]
[R7] 1999-02 - 2006-06 In-house collaborative optimization repository for research on multi-objective program and architecture characterization, optimization and co-design with first experiments on predictive modeling
  • Powered by EOS framework [S19]
  • Developed and used in the EU FP5 MHAOTEU project [J17]
  • Funded by [F10, A8]
  • Discontinued for [R6]
[R8] 1993-03 - 1999-02 In-house Experimental Repository for research, development and experimentation on novel, semiconductor neural networks, and on providing unified access to HPC resources as a web service
  • Powered by SCS framework [S20]
  • Partially funded by [A11, J23]
  • Discontinued for [R7]