Main Page

From cTuning.org

(Difference between revisions)

Revision as of 11:54, 7 October 2009

It may not always be visible to the IT users, but developing and optimizing current and emerging computing systems using available technology is too time consuming and costly. cTuning.org is a community-driven collaborative wiki-based portal that brings together academia, industry and end-users to develop intelligent collective tuning technology that automates and simplifies compiler, program and architecture design and optimization. This technology minimizes repetitive time consuming tasks and human intervention using collective optimization, run-time adaptation, statistical and machine learning techniques. It can already help to improve performance, power consumption, reliability and other important characteristics of the available computing systems automatically (ranging from supercomputers to embedded systems) and should eventually enable development of the emerging intelligent self-tuning adaptive computing systems.

cTuning concept, collaborative R&D tools, cTuning website and optimization prediction services have been originally developed and coordinated by Dr. Grigori Fursin (UNIDAPT Group, INRIA, France) and extended during the MILEPOST project. Since 2009, cTuning.org is a open community-driven project to develop self-tuning computing systems. If you have any questions about cTuning.org, don't hesitate to contact us or use cTuning mailing lists. If you need to reference cTuning project, you can reference cTuning website or the following papers FT2009,Fur2009,FMTP2008.

We are very grateful to all our colleagues and users for providing valuable feedback or contributing to the project.

You are warmly welcome to join this community-driven collaborative effort to automate code, compiler and architecture design and optimization and boost innovation and research to enable future self-tuning adaptive computing systems based on heterogeneous, reconfigurable multi-core architectures.

Note: cTuning is an ongoing evolving project - please be patient and tolerant to the community and help us with this collaborative effort!

Typical non-trivial distribution of optimization points in the 2D space of speedups vs code size of a susan_corners program on AMD Athlon64 3700+ architecture with GCC 4.2.2 during automatic program optimization using ccc-run-glob-flags-rnd-uniform plugin from CCC framework with uniform random combinations of more than 100 global compiler flags (each flag has 50% probability to be selected for a given combination of optimizations). Similar data for other benchmark, datasets and architectures is available in the Collective Optimization Database. cTuning technology helps users find or predict optimial optimization points in such complex spaces using "one-button" approach.

We are participating in the following collaborative activities:

Develop common open-source tools with unified APIs (universal compilers adaptable to any heterogeneous multi-core architecture, computer architecture simulators, adaptive run-time systems) to optimize programs and architectures collectively using iterative compilation, statistical and machine learning techniques.
More information is available at cTools page.
Share interesting optimization cases from the community for programs/libraries/OS (compiler optimizations/architecture configurations to improve execution time, code size, architecture size, power consumption, etc) in the Collective Optimization Database to help users optimize their systems, enable replicable collaborative research and enable further analysis using statistical and machine learning techniques.
More information is available at cDatabase page.
Enable collaborative research using cTools to automate and simplify the process of developing and optimizing new computer architectures, compilers, operating systems and programming environments using statistical analysis, machine learning, dynamic adaptation and bio-inspired techniques. We believe that our adaptive approaches are critical to overcome the complexity of computing systems and improve their performance, power consumption, system size and fault-tolerance automatically while reducing their cost and time to market.
More information is available at cResearch page.

News

2009.October.2 - We successfully passed the final MILEPOST review and the project is officially over. We would like to thank all the partners from the University of Edinburgh, IBM Haifa, CAPS and ARC for a great collaborative work during last 3 years and cTuning community for a very interesting feedback and extensions! We released all the tools from the project and hope to continue extending them within community-driven cTuning.org. This infrastructure should open up many interesting research opportunities for performance auto-tuning based on statistical and machine learning techniques so we hope to see many more interesting extensions to the MILEPOST/cTuning technology soon ;) !..

2009.September.25 - New CFP for SMART'10 workshop co-located with HiPEAC'10 conference in Pisa, Italy is now available. Prof. Keith Cooper from Rice University kindly agreed to give a keynote talk.

2009.September.22 - Congratulations to Dr.Christophe Dubach who has won a presitigious BCS/CPHC Distinguished Dissertation Award for his thesis "Using Machine-Learning to Efficiently Explore the Architecture/Compiler Co-Design Space" supervised by Prof. Michael O'Boyle. This topic is related to cTuning technology!

2009.September.1 - The documentation of MILEPOST GCC/GCC ICI extensions by Yuanjie and Liang during GSOC'09 program is now available: Function cloning and program instrumentation and Fine grain program tuning. We would like to fully test and sync these developments with mainline GCC within next month or two.

2009.August.05 - The colleagues from the UNIDAPT Group started investigating the use of cTuning/MILEPOST technology and the UNIDAPT framework to predict good optimization and parallelization schemes for hybrid heterogeneous CPU/GPU-like architectures together with CAPS Entreprise based on run-time adaptation and profiling, empirical iterative compilation, statistical analysis, machine learning, program and dataset features and run-time decision trees (FT2009, LCWP2009, Fur2009, JGVP2009, TWFP2009, FMTP2008, LFF2007, FCOP2005). They plan to add new optimization cases to the Collective Optimization Database in Autumn, 2009.

2009.July.27 - The paper "Portable Compiler Optimization Across Embedded Programs and Microarchitectures using Machine Learning" (DJBP2009) has been accepted for the 42nd IEEE/ACM International Symposium on Microarchitecture (MICRO). The research has been led by the colleagues from the University of Edinburgh - congratulations!

2009.July.22 - cTuning-related CFP: SMART'10 and GROW'10 workshops will be co-located with HiPEAC'10 conference in Pisa, Italy at the end of January, 2009. You are warmly invited to submit your novel research results and developments to our workshops! At the websites of these workshops you can find all information about topics, PC, deadlines and submission procedures.

2009.July.15 - New optimization cases to improve execution time, code size and compilation time are now available in Collective Optimization Database for cBench, EEMBC, SPEC and other applications with multiple datasets using MILEPOST GCC 4.4.0 and 2 architectures (Intel and AMD) from GRID5000 (infrastructure for large scale parallel and distributed computing research).

2009.July.08 - The preprint of a paper "Collective Tuning Initiative: automating and accelerating development and optimization of computing systems" describing cTuning infrastructure is now available online (Fur2009).

2009.June.29 - Stable MILEPOST GCC 4.4.0 has been released. Follow further community developments at cTuning GCC ICI page and cTuning development mailing list.
Next, we plan to use MILEPOST/cTuning technology to enable realistic adaptive parallelization, data partitioning and scheduling for heterogeneous multi-core systems using statistical and machine learning techniques.

2009.June.26 - The pdf of the paper that describes Collective Tuning Infrastructure and cTuning concept (presented at the GCC Summit'09) will be available in a few weeks here.

2009.June.17 - We participated in discussions to include plugin system similar to ICI to mainline GCC for a long time and finally GCC 4.5 will feature a low-level plugin system. We are now synchronizing high-level ICI/MILEPOST with the mainline to be able to reuse all our available plugins. We also develop several new plugins within Google Summer of Code'2009 to enable XML representation of the compilation flow, fine-grain program optimizations and instrumentation, automatic tuning of optimization heuristic based on machine learning, and function-level run-time adaptation. Comparison of GCC low-level and high-level ICI plugins is available here. The ICI development and discussions mailing list is available here.

2009.June.10 - Extended version of the "Collective Optimization" paper (FT2009) describing collective tuning concept has been accepted for ACM Transactions on Architecture and Code Optimization (TACO).

2009.June.03-10 - We gave several talks/demos/tutorials about cTuning at the HiPEAC Computing week (Infineon, Munich, Germany) and GCC Summit (Montreal, Canada).

2009.June.01 - After nearly 1 year of developments we released/updated all our open-source collaborative R&D tools:
- fully redesigned and documented Interactive Compilation Interface v2.0 for GCC 4.4.0 synchronized with the official plugin GCC branch - transforming compilers into plugin-enabled research toolsets
- MILEPOST GCC 4.4.0 pre-release version at SVN - automating program optimization and compiler optimization heuristic tuning using machine learning
- Continuous Collective Compilation Framework v2.0 - enabling automatic collaborative program optimization based on statistical and machine learning techniques
- Collective Benchmark/MiDataSets v1.0 - enabling realistic program optimization research and benchmarking using multiple open-source programs/datasets.
  
  We also updated Collective Optimization Database with various optimization cases for Intel and AMD processors and comparison of different compilers including GCC, LLVM, Open64, Intel, etc - enabling sharing and reuse of optimization knowledge.
  
  We would like to thank cTuning community for feedback, help and support! You are welcome to join this community effort to automate program optimization and compiler/architecture design.

2009.April.27 - We gave a talk at the University of Illinois at Urbana-Champaign about Collective Tuning Initiative and tools and MILEPOST project ("Collective Optimization, run-time adaptation and machine learning"). Presentation is available here. We would like to thank all the UIUC colleagues for a very interesting and useful feedback.

2009.April.23 - Preview version of the optimization predictor based on static program features and machine learning (to improve program execution time, code size, etc) is now available on-line. It is an on-going project, so please be patient. Comments are welcome!

cTuning concept:		cTuning friends:
		You are welcome to register your interest at this page.

@@ Line 26: / Line 26: @@
 | valign="top" |
 {{News1|
+*'''2009.October.2''' - We successfully passed the final [http://www.milepost.eu MILEPOST] review and the project is officially over. We would like to thank all the partners from the University of Edinburgh, IBM Haifa, CAPS and ARC for a great collaborative work during last 3 years and [http://cTuning.org/community cTuning community] for a very interesting feedback and extensions! We released all the tools from the project and hope to continue extending them within community-driven [http://cTuning.org cTuning.org]. This infrastructure should open up many interesting research opportunities for performance auto-tuning based on statistical and machine learning techniques so we hope to see many more interesting extensions to the MILEPOST/cTuning technology soon ;) !..
 *'''2009.September.25''' - New CFP for [http://ctuning.org/workshop-smart10 SMART'10 workshop] co-located with HiPEAC'10 conference in Pisa, Italy is now available. [http://www.cs.rice.edu/~keith Prof. Keith Cooper] from Rice University kindly agreed to give a keynote talk.
 *'''2009.September.22''' - Congratulations to Dr.Christophe Dubach who has won a presitigious BCS/CPHC Distinguished Dissertation Award for his thesis "Using Machine-Learning to Efficiently Explore the Architecture/Compiler Co-Design Space" supervised by Prof. Michael O'Boyle. This topic is related to cTuning technology!
 *'''2009.September.1''' - The documentation of [http://ctuning.org/milepost-gcc MILEPOST GCC]/[http://ctuning.org/ici GCC ICI] extensions by Yuanjie and Liang during GSOC'09 program is now available: [http://ctuning.org/wiki/index.php/CTools:ICI:Projects:GSOC09:Function_cloning_and_program_instrumentation Function cloning and program instrumentation] and [http://ctuning.org/wiki/index.php/CTools:ICI:Projects:GSOC09:Fine_grain_tuning Fine grain program tuning]. We would like to fully test and sync these developments with mainline GCC within next month or two.
 *'''2009.August.05''' - The colleagues from the [http://unidapt.org UNIDAPT Group] started investigating the use of [http://ctuning.org cTuning]/[http://www.milepost.eu MILEPOST] technology and the [http://ctuning.org/unidapt UNIDAPT framework] to predict good optimization and parallelization schemes for hybrid heterogeneous CPU/GPU-like architectures together with [http://www.caps-entreprise.com CAPS Entreprise] based on run-time adaptation and profiling, empirical iterative compilation, statistical analysis, machine learning, program and dataset features and run-time decision trees ({{Ref|FT2009}}, {{Ref|LCWP2009}}, {{Ref|Fur2009}}, {{Ref|JGVP2009}}, {{Ref1|TWFP2009}}, {{Ref|FMTP2008}}, {{Ref|LFF2007}}, {{Ref|FCOP2005}}). They plan to add new optimization cases to the [http://ctuning.org/cdatabase Collective Optimization Database] in Autumn, 2009.
 *'''2009.July.27''' - The paper "Portable Compiler Optimization Across Embedded Programs and Microarchitectures using Machine Learning" ({{Ref|DJBP2009}}) has been accepted for the [http://www.microarch.org/micro42 42nd IEEE/ACM International Symposium on Microarchitecture (MICRO)]. The research has been led by the colleagues from the University of Edinburgh - congratulations!
 *'''2009.July.22''' - cTuning-related CFP: [http://ctuning.org/workshop-smart10 SMART'10] and [http://ctuning.org/workshop-grow10 GROW'10] workshops will be co-located with [http://www.hipeac.net/conference HiPEAC'10 conference] in Pisa, Italy at the end of January, 2009. You are warmly invited to submit your novel research results and developments to our workshops! At the websites of these workshops you can find all information about topics, PC, deadlines and submission procedures.
 *'''2009.July.15''' - New optimization cases to improve execution time, code size and compilation time are now available in [http://ctuning.org/cdatabase Collective Optimization Database] for [http://ctuning.org/cbench cBench], [http://www.eembc.org EEMBC], [http://www.spec.org SPEC] and other applications with multiple datasets using [http://ctuning.org/milepost-gcc MILEPOST GCC 4.4.0] and 2 architectures ([http://www.intel.com Intel] and [http://www.amd.com AMD]) from [http://www.grid5000.fr GRID5000] (infrastructure for large scale parallel and distributed computing research).
 *'''2009.July.08''' - The preprint of a paper "Collective Tuning Initiative: automating and accelerating development and optimization of computing systems" describing cTuning infrastructure is now available online ({{Ref|Fur2009}}).
 *'''2009.June.29''' - Stable MILEPOST GCC 4.4.0 has been [[CTools:MilepostGCC|released]]. Follow further community developments at [http://ctuning.org/ici cTuning GCC ICI page] and [http://groups.google.com/group/ctuning-discussions cTuning development mailing list].<BR>Next, we plan to use MILEPOST/cTuning technology to enable realistic adaptive parallelization, data partitioning and scheduling for heterogeneous multi-core systems using statistical and machine learning techniques.
 *'''2009.June.26''' - The pdf of the paper that describes Collective Tuning Infrastructure and cTuning concept (presented at the GCC Summit'09) will be available in a few weeks [http://unidapt.org/index.php/Dissemination#Fur2009 here].
 *'''2009.June.17''' - We participated in discussions to include plugin system similar to [http://ctuning.org/ici ICI] to mainline GCC for a long time and finally GCC 4.5 will feature a low-level plugin system. We are now synchronizing high-level ICI/MILEPOST with the mainline to be able to reuse all our available plugins. We also develop several new plugins within [http://socghop.appspot.com/org/home/google/gsoc2009/gcc Google Summer of Code'2009] to enable XML representation of the compilation flow, fine-grain program optimizations and instrumentation, automatic tuning of optimization heuristic based on machine learning, and function-level run-time adaptation. Comparison of GCC low-level and high-level ICI plugins is available [http://gcc.gnu.org/wiki/GCC_PluginComparison here]. The ICI development and discussions mailing list is available [http://groups.google.com/group/ctuning-discussions here].
 *'''2009.June.10''' - Extended version of the "Collective Optimization" paper ({{Ref|FT2009|}}) describing collective tuning concept has been accepted for ACM Transactions on Architecture and Code Optimization (TACO).
 *'''2009.June.03-10''' - We gave several talks/demos/tutorials about cTuning at the [http://www.hipeac.net/industry_workshop7 HiPEAC Computing week] (Infineon, Munich, Germany) and [http://www.gccsummit.org/2009 GCC Summit] (Montreal, Canada).
 * '''2009.June.01''' - After nearly 1 year of developments we released/updated all our open-source collaborative [[CTools|R&D tools]]:
 ** fully redesigned and documented [[CTools:ICI|Interactive Compilation Interface]] v2.0 for GCC 4.4.0 synchronized with the official plugin GCC branch - transforming compilers into plugin-enabled research toolsets