From cTuning.org

Revision as of 12:31, 23 May 2011 by Gfursin (Talk | contribs)
Jump to: navigation, search

cTuning.org - free, open source, collaborative repository and tools for program and architecture characterization and optimization.

It may not always be visible to the IT users, but developing and optimizing current and emerging computing systems using available technology is excessively inefficient, time consuming and costly. During last several decades, many research papers have been published with suggestions about how to easily solve all these problems! Unfortunately, we often struggled replicating those results in realistic environments or finding stable practical tools based on these research papers. Therefore, Dr. Grigori Fursin created this collaborative Collective Tuning wiki-based portal based on his past R&D to continue developing free common open-source extensible collaborative infrastructure, benchmarks and collective optimization repository based on multiple research techniques and production tools to parametrize, abstract, automate, simplify and systematize program, compiler and architecture design and optimization using collective tuning, run-time adaptation, statistical and machine learning techniques. It enables sharing of benchmarks, datasets, optimization cases through unified web-services and tools with common APIs to be able to predict better optimizations or architecture designs provided there is enough information collected in the repository from multiple users. This technology minimizes repetitive time consuming tasks and human intervention: even though there is still a lot to be done, we are glad to see it helping several companies, end users and researchers to improve execution time, code size, power consumption, reliability and other important characteristics of the available computing systems ranging from supercomputers to mobile systems automatically.

cTuning web-site and infrastructure is totally free and is now maintained and extended collaboratively and voluntarily by its users (basically by you). It is an on-going project, so please be patient or join the effort by sharing optimization data, extending tools, exchanging ideas, referencing this work, etc. We created cTuning.org because we believe in collaborative and public nature of R&D and because we would like to change trends in academic IT research intended to publish myriads of similar papers often non-reproducible results, without releasing open source tools and data.

Current design of our Collective Optimization Framework:

ctuning.gif

We developed Collective Optimization Database to continuously collect a large number of optimization cases from the community to learn how to correlate program features, program and system behavior and good optimizations between multiple programs, datasets, compilers, operating systems and architectures. This repository is also intended to improve the quality of academic research by avoiding costly duplicate experiments and providing reproducible results.

cTuning open-source infrastructure is still far from solving all optimization problems but we hope that it already opens up some interesting collaborative R&D opportunities to the community to develop intelligent self-tuning adaptive computing systems. We hope that cTuning-like technology will one day eventually improve production compilers that we use including GCC, LLVM, Rose source-to-source tool, Open64, IBM XL and Testarossa, HMPP directive based compiler for hybrid multicore systems, Intel compiler suites, and operating systems including Moblin, Android, standard desktop/server Linux distributions, Windows, cloud/distributed operating systems and so on. We would like to thank all cTuning colleagues and users who are or have been helping with this project.

Note: cTuning is an ongoing evolving project - please be patient and tolerant to the community. You are warmly welcome to join cTuning community to help us parametrize and automate code, compiler and architecture design and optimization!

We are participating in the following collaborative activities:

  • Develop common open-source tools with unified APIs (universal compilers adaptable to any heterogeneous multi-core architecture, computer architecture simulators, adaptive run-time systems) to optimize programs and architectures collectively using iterative compilation, statistical and machine learning techniques.
    More information is available at cTools page.
  • Share interesting optimization cases from the community for programs/libraries/OS (compiler optimizations/architecture configurations to improve execution time, code size, architecture size, power consumption, etc) in the Collective Optimization Database to help users optimize their systems, enable replicable collaborative research and enable further analysis using statistical and machine learning techniques.
    More information is available at cDatabase page.
  • Enable collaborative research using cTools to automate and simplify the process of developing and optimizing new computer architectures, compilers, operating systems and programming environments using statistical analysis, machine learning, dynamic adaptation and bio-inspired techniques. We believe that our adaptive approaches are critical to overcome the complexity of computing systems and improve their performance, power consumption, system size and fault-tolerance automatically while reducing their cost and time to market.
    More information is available at cResearch page.

Motivation example:

Example of complex optimization search spaces for susan_c (including optimization space frontier for multi-objective optimizations) from Collective Benchmark after using CCC framework (that has some similarities with the useful ACOVEA tool but also allows automatic sharing of optimization knowledge with the community in the Collective Optimization Database and uses plugins to implement various search techniques besides genetic algorithms) and cTuning CC/MILEPOST GCC 4.4.x:

fig_opt_case_susan_c_1.gif fig_opt_case_susan_c_2.gif

Example of program similarities using static program features and based on best found program optimizations continuously collected in the cTuning optimization repository that improve execution time (as well as code size, compilation time, etc):

img_influence_features.gif img_influence_optimizations.gif
.

cTuning CC/MILEPOST GCC 4.4.x helps to correlate program features and optimizations using various machine learning techniques to quickly predict good optimizations for a previously unseen program.

News
  • 2011.April.7 - All papers and presentations from SMART 2011 are now available online. Thanks to all the participants for very interesting and lively discussions!
  • 2011.April.6 - CGO 2012 will be in San Jose, California. Abstract deadline is set to September 13, 2011. More info will be available soon at the official website.
  • 2011.April.6 - Please, check out new publication paradigm for HiPEAC 2012 that will take place in Paris in January 23-25, 2012.
  • 2011.April.6 - ASPLOS 2012 abstract deadline: Monday, July 18, 2011
  • 2011.February.25 - Call for papers: EXADAPT 2011 (co-located with PLDI/FCRC 2011): 1st International Workshop on Adaptive Self-Tuning Computing Systems for the Exaflop Era. Paper submission deadline: March 27, 2011.
  • 2010.December.31 - Dear all, we wish you very nice and relaxing holidays and super-exciting, productive and successful New Year ;) !..
  • 2010.December.25 - The website for SMART'2011 workshop (co-located with CGO'2011) is now finalized and the submission website is open! Please, follow this link, submit your best papers ;) and spread the word!
  • 2010.December.20 - Extended variant of our paper on "Collective Optimization" will appear in December issue of the ACM Transactions on Architecture and Code Optimization (TACO). PDF and BIB are now available here: FT2010.
  • 2010.October.31 - Paper about practical aggregation of semantical program properties for machine learning based optimization by M.Namolaru et al from CASES'10 is now available on-line here. It describes mechanisms of feature extraction inside MILEPOST GCC/cTuning CC.
  • 2010.October.26 - SMART'11 will be co-located with the CGO'11 conference. More information will be following soon!
  • 2010.August.16 - Submissions are now open for CGO 2011.
  • 2010.June.30 - Call for papers: CGO'11.
  • 2010.June.9 - Finally, we recovered all cTuning website and services after physical hard drive failure. However, if you still experience some problems or abnormal behavior, please report that to the cTuning discussions mailing list! Thanks and sorry for any inconvenience!
  • 2010.May.22 - Pre-release of cTuning CC V2.5 is now available. cTuning CC is a free, open source compiler collection that combines multiple tools and techniques including MILEPOST GCC, ICI, CCC framework, cTuning web-services and Collective Optimization Database and cBench as the first practical step toward self-tuning, adaptive computing systems based on industrial tools, empirical techniques, transparent collective optimization, statistical analysis and machine learning. cTuning CC is a wrapper around any compiler such as GCC, LLVM, Open64, Path64, etc that can transparently invoke machine learning mode to correlate program features of a compiled program with the ones stored in the Collective Optimization Database and suggest better optimizations for multi-objective criteria such as improving execution time, compilation time, code size, etc (using optimization space frontier detection).
  • 2010.April.28 - List of all projects accepted for Google Summer of Code 2010 is now available on-line. There are many projects related to GCC, LLVM, MONO, etc.
    Discussion page about GCC as a research compiler has been updated.
    Diego Novillo started a proposal to modularize GCC.
  • 2010.April.14 - Long awaited GCC 4.5 has been released! It features new plugin framework with some parts of ICI to continue GCC modularization and parametrization, simplify pass manipulation and reordering, and enable better integration with cTuning/MILEPOST tools to automate optimization space exploration and prediction of profitable combinations of program transformations during multi-objective optimizations (balancing execution time, code size, compilation time, etc) for a given program/dataset/architecture.
  • 2010.March.17 - We pre-released all tools including MILEPOST GCC, CCC framework, cBench and cDatabase. It's a major update of cTuning tools including changes behind to support transparent optimizations of programs and libraries, better multi-objective optimization (including balancing of execution time, code size and compilation time), bug fixes in averaging multiple optimization cases, C++ support in MILEPOST GCC, support of all version of GCC 4.4, new static features in MILEPOST GCC, extended documentation, etc. Feedback and comments are welcome here.
  • 2010.March.01 - CGO'10 program is available on-line.
  • 2010.February.10 - Accepted papers for PLDI'10 are now available on-line.
  • 2010.January.28 - Proceedings and slides from GROW'10 and SMART'10 are now available online.
  • 2010.January.4 - Call for participation: GROW'10 and SMART'10 workshops will be held on the 23rd and 24th of January in Pisa, Italy co-located with the HiPEAC conference. Preliminary programs are available: SMART'10 program and GROW'10 program.


News archive



cTuning hosts:
cTuning friends:
logo_unidapt1.gif
logo-inria-scientifique-couleur.jpg logo_ibm.jpg logo_caps.gif logo_ue.gif logo_arc.gif logo_ict.gif logo_hipeac.jpg
Locations of visitors to this page