From cTuning.org
Current revision (15:49, 12 September 2015) (view source) |
|||
| (36 intermediate revisions not shown.) | |||
| Line 1: | Line 1: | ||
| - | + | '''''NEWS: Since 2015, we moved all related developments to our new [http://github.com/ctuning/ck/wiki Collective Knowledge Framework]! Demo of a new live repository is available [http://cknowledge.org/repo here].''''' | |
| - | {| | + | |
| - | | | + | {{CTitle|http://ctuning.org/wiki/images/milepost_image.jpg|cTuning Compiler Collection|machine-learning enabled, self-tuning, adaptive compilers}} |
{{CShortCut|ctuning-cc}} | {{CShortCut|ctuning-cc}} | ||
| Line 8: | Line 8: | ||
{{CMenu:CTools|}} | {{CMenu:CTools|}} | ||
| - | cTuning CC is a free, open source compiler collection that combines multiple tools and techniques | + | {{NEWS}} |
| + | {{Reference}} | ||
| + | |||
| + | cTuning CC is a free, open source compiler collection that combines multiple tools and techniques including [[CTools:MilepostGCC|MILEPOST GCC]], [[CTools:ICI|ICI]], [[CTools:CCC|CCC framework]], [[CDatabase|cTuning web-services and Collective Optimization Database]] and [[CTools:CBench|cBench]] as the first practical step toward self-tuning, adaptive computing systems based on industrial tools, empirical techniques, transparent collective optimization, statistical analysis and machine learning. cTuning CC is a wrapper around any compiler such as [http://gcc.gnu.org GCC], [http://llvm.org LLVM], [http://www.open64.net Open64], [http://www.pathscale.com Path64], etc that can transparently invoke machine learning mode to correlate program features of a compiled program with the ones stored in the [http://cTuning.org/cdatabase Collective Optimization Database] and suggest better optimizations for multi-objective criteria such as improving execution time, compilation time, code size, etc (using optimization space frontier detection). | ||
It may not always be visible to the IT users, but developing and optimizing computing systems using available over-complicated technology is too time consuming and costly often resulting in underperforming, power-hungry and inefficient computers and programs. Novel cTuning technology attempts to overcome the complexity of computing system by automating architecture, code and dataset analysis, characterization and multi- objective optimization (currently execution time, code size and compilation time) and enabling portable optimization using | It may not always be visible to the IT users, but developing and optimizing computing systems using available over-complicated technology is too time consuming and costly often resulting in underperforming, power-hungry and inefficient computers and programs. Novel cTuning technology attempts to overcome the complexity of computing system by automating architecture, code and dataset analysis, characterization and multi- objective optimization (currently execution time, code size and compilation time) and enabling portable optimization using | ||
| Line 30: | Line 33: | ||
<div align="center">http://ctuning.org/wiki/images/img-ctuning-cc.gif</div> | <div align="center">http://ctuning.org/wiki/images/img-ctuning-cc.gif</div> | ||
| + | <div align="center">http://ctuning.org/wiki/images/img-milepost-gcc-structure.gif</div> | ||
* '''Background:''' | * '''Background:''' | ||
| Line 54: | Line 58: | ||
| - | + | '''Motivation example:''' | |
| - | + | ||
| - | + | ||
| - | + | ||
| - | + | ||
| - | + | ||
| - | + | ||
| - | + | ||
| - | + | ||
| - | + | ||
| - | + | ||
| - | + | ||
| - | + | ||
| - | + | ||
| - | + | ||
| - | + | ||
| - | + | ||
| - | + | ||
| - | + | ||
| - | + | ||
| - | + | Example of complex optimization search spaces for susan_c (including optimization space frontier for multi-objective optimizations) from [[CTools:CBench|Collective Benchmark]] after using [[CTools:CCC|CCC framework]] (that has some similarities with the useful [http://www.coyotegulch.com/products/acovea ACOVEA tool] but also allows automatic sharing of optimization knowledge with the community in the [[CDatabase|Collective Optimization Database]] and uses plugins to implement various search techniques besides genetic algorithms) and [[CTools:CTuningCC|cTuning CC]]/[[CTools:MilepostGCC|MILEPOST GCC 4.4.x]]: | |
| + | <div align="center">http://cTuning.org/wiki/images/fig_opt_case_susan_c_1.gif http://ctuning.org/wiki/images/fig_opt_case_susan_c_2.gif</div> | ||
| - | | | + | Example of program similarities using [[CTools:MilepostGCC:StaticFeatures|static program features]] and based on best found program optimizations continuously collected in the [[CDatabase|cTuning optimization repository]] that improve execution time (as well as code size, compilation time, etc): |
| + | <div align="center">http://cTuning.org/wiki/images/img_influence_features.gif http://cTuning.org/wiki/images/img_influence_optimizations.gif</div>. | ||
| + | [[CTools:CTuningCC|cTuning CC]]/[[CTools:MilepostGCC|MILEPOST GCC 4.4.x]] helps to correlate program features and optimizations using various machine learning techniques to quickly predict good optimizations for a previously unseen program. | ||
---- | ---- | ||
Current revision
NEWS: Since 2015, we moved all related developments to our new Collective Knowledge Framework! Demo of a new live repository is available here.
 |
cTuning Compiler Collection |
| machine-learning enabled, self-tuning, adaptive compilers |
Web shortcut: http://cTuning.org/ctuning-cc
Navigation: cTuning.org > CTools
NOTES:
- Reference publications about cTuning.org long-term vision: GCC Summit'09, ACM TACO'10 journal and IJPP'11 journal.
cTuning.org is based on the following reference publications: GCC Summit'09, CPE'04, HiPEAC'05, PhD thesis'04, SMART'09, PLDI'10, HiPEAC'09, IJPP'11
cTuning CC is a free, open source compiler collection that combines multiple tools and techniques including MILEPOST GCC, ICI, CCC framework, cTuning web-services and Collective Optimization Database and cBench as the first practical step toward self-tuning, adaptive computing systems based on industrial tools, empirical techniques, transparent collective optimization, statistical analysis and machine learning. cTuning CC is a wrapper around any compiler such as GCC, LLVM, Open64, Path64, etc that can transparently invoke machine learning mode to correlate program features of a compiled program with the ones stored in the Collective Optimization Database and suggest better optimizations for multi-objective criteria such as improving execution time, compilation time, code size, etc (using optimization space frontier detection).
It may not always be visible to the IT users, but developing and optimizing computing systems using available over-complicated technology is too time consuming and costly often resulting in underperforming, power-hungry and inefficient computers and programs. Novel cTuning technology attempts to overcome the complexity of computing system by automating architecture, code and dataset analysis, characterization and multi- objective optimization (currently execution time, code size and compilation time) and enabling portable optimization using
- continuous parameterization of all components of a computing system (from architecture to operating system, compiler and code),
- continuous empirical collective optimization space exploration distributed among multiple users,
- continuous profiling and characterization of applications (extraction of program and architecture features), run-time behavior and resources,
- continuous sharing of analysis and optimization information in the Collective Optimization Database
- continuous refining and adaptation of performance models and optimization prediction based on standard statistical and machine learning techniques.
cTuning CC includes:
- New cTuning compiler wrapper to transparently extract program structure and features (using MILEPOST GCC), communicate with cTuning web services to share optimization data and predict optimizations, and invoke any other user compiler (GCC, LLVM, Open64, ICC, XL, ROSE, etc)
- MILEPOST GCC 4.4.x (self-tuning, adaptive, machine-learning based compiler) with ICI (plugin framework) v2.05 and MILEPOST feature extractor V2.1
- New Continuous Collective Compilation framework
- Collective Benchmark
- New plugins and web-services for multi-objective optimizations(balancing execution time, code size, compilation time)
We are developing cTuning infrastructure as a very simple, modular and portable tool so that users could easily download, install and use it to compile, execute, characterize and optimize their programs or share optimization knowledge. Our users managed to optimize some large industrial applications such as BerkeleyDB (1.4 speedup over GCC 4.4.0 -O3 on several Intel Xeon machines), some audio and video codecs, multiple standard benchmarks, Linux kernel, etc.
Please, note that this is an on-going, evolving project driven by the cTuning community, so please be patient or join the project and help to improve cTuning infrastructure.
- FAQs - Does cTuning CC/MILEPOST GCC solve all optimization problems? How similar is it ACOVEA tool? etc...
- Related:
- MILEPOST GCC - machine learning enabled, self-tuning, adaptive compiler
- Online predictor of optimizations based on program features.
- Collective optimization repository to continuously collect profitable optimization cases from the community that improve program execution time, code size, compilation time, etc...
- CCC framework - Continuous Collective Compilation Framework to automate search of profitable optimization cases to improve program execution time, code size, compilation time, etc. It is used to train MILEPOST GCC and has some similarities with the useful ACOVEA tool but also allows automatic sharing of optimization knowledge with the community in the Collective Optimization Database and uses plugins to implement various search techniques besides genetic algorithms.
- Archive website of the MILEPOST project (2006-2009).
Motivation example:
Example of complex optimization search spaces for susan_c (including optimization space frontier for multi-objective optimizations) from Collective Benchmark after using CCC framework (that has some similarities with the useful ACOVEA tool but also allows automatic sharing of optimization knowledge with the community in the Collective Optimization Database and uses plugins to implement various search techniques besides genetic algorithms) and cTuning CC/MILEPOST GCC 4.4.x:
Example of program similarities using static program features and based on best found program optimizations continuously collected in the cTuning optimization repository that improve execution time (as well as code size, compilation time, etc):
cTuning CC/MILEPOST GCC 4.4.x helps to correlate program features and optimizations using various machine learning techniques to quickly predict good optimizations for a previously unseen program.
| cTuning CC friends: |
      
|