CTools:CTuningCC

Web shortcut: http://cTuning.org/ctuning-cc

Navigation: cTuning.org > CTools 

cTuning CC is a free, open source compiler collection that combines multiple tools and techniques including MILEPOST GCC, ICI, CCC framework, cTuning web-services and Collective Optimization Database and cBench as the first practical step toward self-tuning, adaptive computing systems based on industrial tools, empirical techniques, transparent collective optimization, statistical analysis and machine learning. cTuning CC is a wrapper around any compiler such as GCC, LLVM, Open64, Path64, etc that can transparently invoke machine learning mode to correlate program features of a compiled program with the ones stored in the Collective Optimization Database and suggest better optimizations for multi-objective criteria such as improving execution time, compilation time, code size, etc (using optimization space frontier detection).

It may not always be visible to the IT users, but developing and optimizing computing systems using available over-complicated technology is too time consuming and costly often resulting in underperforming, power-hungry and inefficient computers and programs. Novel cTuning technology attempts to overcome the complexity of computing system by automating architecture, code and dataset analysis, characterization and multi- objective optimization (currently execution time, code size and compilation time) and enabling portable optimization using

• continuous parameterization of all components of a computing system (from architecture to operating system, compiler and code),
• continuous empirical collective optimization space exploration distributed among multiple users,
• continuous profiling and characterization of applications (extraction of program and architecture features), run-time behavior and resources,
• continuous sharing of analysis and optimization information in the Collective Optimization Database
• continuous refining and adaptation of performance models and optimization prediction based on standard statistical and machine learning techniques.

cTuning CC includes:

• New cTuning compiler wrapper to transparently extract program structure and features (using MILEPOST GCC), communicate with cTuning web services to share optimization data and predict optimizations, and invoke any other user compiler (GCC, LLVM, Open64, ICC, XL, ROSE, etc)
• MILEPOST GCC 4.4.x (self-tuning, adaptive, machine-learning based compiler) with ICI (plugin framework) v2.05 and MILEPOST feature extractor V2.1
• New Continuous Collective Compilation framework
• Collective Benchmark
• New plugins and web-services for multi-objective optimizations(balancing execution time, code size, compilation time)

We are developing cTuning infrastructure as a very simple, modular and portable tool so that users could easily download, install and use it to compile, execute, characterize and optimize their programs or share optimization knowledge. Our users managed to optimize some large industrial applications such as BerkeleyDB (1.4 speedup over GCC 4.4.0 -O3 on several Intel Xeon machines), some audio and video codecs, multiple standard benchmarks, Linux kernel, etc.

Please, note that this is an on-going, evolving project driven by the cTuning community, so please be patient or join the project and help to improve cTuning infrastructure.

• Background:
  • Motivation and history
  • Usage scenarios / stories
  • Resources (links, publications, etc)

• Downloads

• Documentation

• User suggestions and feedback

• Bug reports

• FAQs - Does cTuning CC/MILEPOST GCC solve all optimization problems? How similar is it ACOVEA tool? etc...

• Related:
  • MILEPOST GCC - machine learning enabled, self-tuning, adaptive compiler
  • Online predictor of optimizations based on program features.
  • Collective optimization repository to continuously collect profitable optimization cases from the community that improve program execution time, code size, compilation time, etc...
  • CCC framework - Continuous Collective Compilation Framework to automate search of profitable optimization cases to improve program execution time, code size, compilation time, etc. It is used to train MILEPOST GCC and has some similarities with the useful ACOVEA tool but also allows automatic sharing of optimization knowledge with the community in the Collective Optimization Database and uses plugins to implement various search techniques besides genetic algorithms.
  • Archive website of the MILEPOST project (2006-2009).

Motivation example:

Example of complex optimization search spaces for susan_c (including optimization space frontier for multi-objective optimizations) from Collective Benchmark after using CCC framework (that has some similarities with the useful ACOVEA tool but also allows automatic sharing of optimization knowledge with the community in the Collective Optimization Database and uses plugins to implement various search techniques besides genetic algorithms) and cTuning CC/MILEPOST GCC 4.4.x:

Example of program similarities using static program features and based on best found program optimizations continuously collected in the cTuning optimization repository that improve execution time (as well as code size, compilation time, etc):

cTuning CC/MILEPOST GCC 4.4.x helps to correlate program features and optimizations using various machine learning techniques to quickly predict good optimizations for a previously unseen program.