cTuning.org - free, open source, collaborative repository and tools for program and architecture characterization and optimization
cTuning is based on the following reference publications: CPE'04, GCC Summit'09, HiPEAC'05, PhD thesis, PLDI'10, HiPEAC'09, IJPP'11, ACM TACO'10
It may not always be visible to the IT users, but developing and optimizing current and emerging computing systems using available technology is excessively inefficient, time consuming and costly. During past decades, many research papers have been published about how to design and optimize computer systems. However, most of the time, we struggle to reproduce those results in realistic environments or find common open-source tools based on these research papers. Therefore, we created this collaborative tuning center with a public repository to share data about behavior of computer systems ("big data"), share free open-source tools, benchmarks, data sets, and let community build predictive models that correlate behavior of computer systems with various properties and characteristics of computer systems. cTuning technology is used to help academic and industry to collaboratively improve execution time, code size, power consumption, reliability and other important characteristics of various computing systems ranging from supercomputers to mobile systems.
Current design of our Collective Optimization Framework:
We developed Collective Optimization Database to continuously collect a large number of optimization cases from the community to learn how to correlate program features, program and system behavior and good optimizations between multiple programs, datasets, compilers, operating systems and architectures. This repository is also intended to improve the quality of academic research by avoiding costly duplicate experiments and providing reproducible results.
cTuning open-source infrastructure is still far from solving all optimization problems but we hope that it already opens up some interesting collaborative R&D opportunities to the community to develop intelligent self-tuning adaptive computing systems. We hope that cTuning-like technology will one day eventually improve production compilers that we use including GCC, LLVM, Rose source-to-source tool, Open64, IBM XL and Testarossa, HMPP directive based compiler for hybrid multicore systems, Intel compiler suites, and operating systems including Moblin, Android, standard desktop/server Linux distributions, Windows, cloud/distributed operating systems and so on. We would like to thank all cTuning colleagues and users who are or have been helping with this project.
Note: cTuning is an ongoing evolving project - please be patient and tolerant to the community. You are warmly welcome to join cTuning community to help us parametrize and automate code, compiler and architecture design and optimization!
We are participating in the following collaborative activities:
- Develop common open-source tools with unified APIs (universal compilers adaptable to any heterogeneous multi-core architecture, computer architecture simulators, adaptive run-time systems) to optimize programs and architectures collectively using iterative compilation, statistical and machine learning techniques. More information is available at cTools page.
- Share interesting optimization cases from the community for programs/libraries/OS (compiler optimizations/architecture configurations to improve execution time, code size, architecture size, power consumption, etc) in the Collective Optimization Database to help users optimize their systems, enable replicable collaborative research and enable further analysis using statistical and machine learning techniques.More information is available at cDatabase page.
- Enable collaborative research using cTools to automate and simplify the process of developing and optimizing new computer architectures, compilers, operating systems and programming environments using statistical analysis, machine learning, dynamic adaptation and bio-inspired techniques. We believe that our adaptive approaches are critical to overcome the complexity of computing systems and improve their performance, power consumption, system size and fault-tolerance automatically while reducing their cost and time to market. More information is available at cResearch page.
Example of complex optimization search spaces for susan_c (including optimization space frontier for multi-objective optimizations) from Collective Benchmark after using CCC framework (that has some similarities with the useful ACOVEA tool but also allows automatic sharing of optimization knowledge with the community in the Collective Optimization Database and uses plugins to implement various search techniques besides genetic algorithms) and cTuning CC/MILEPOST GCC 4.4.x:
Example of program similarities using static program features and based on best found program optimizations continuously collected in the cTuning optimization repository that improve execution time (as well as code size, compilation time, etc):
cTuning CC/MILEPOST GCC uses online machine learning and prediction plugins to correlate program features and optimizations on the fly and quickly predict profitable optimizations for a previously unseen program and for multiple user objectives (balancing execution time, code size, compilation time, etc).
Collective Mind framework and repository is the latest development by Grigori Fursin that aggregates all his past R&D to deliver new universal plugin-based framework to systematize and automate computer engineering using crowdsourcing and machine learning.