CTools

From cTuning.org

Current revision

We moved all developments to our new Collective Knowledge Framework in 2015!

Web shortcut: http://cTuning.org/community

cTuning.org is based on the following reference publications: GCC Summit'09, CPE'04, HiPEAC'05, PhD thesis'04, SMART'09, PLDI'10, HiPEAC'09, IJPP'11

cTuning community develops common open-source tools with unified APIs (compilers, computer architecture simulators, run-time systems, benchmarks) to enable collaborative research and development of the future intelligent adaptive self-tuning computing systems based on iterative compilation, statistical machine learning and run-time adaptation. We believe that current way of disseminating novel techniques only through publications without providing tools to verify and replicate results or with incompatible unstable prototypes often do not help the community to use these techniques immediately or continue collaborative research. We hope that cTools will help to avoid costly duplicate developments, will boost innovation and enable reproducible research while automating and accelerating design, verification, optimization and benchmarking of the architectures and programs. You are welcome to use cTools, provide feedback and help to extend them. You can find more information about potential and on-going extension projects or submit your own projects, feedback and suggestions at individual tool pages below. Alternatively, you are welcome to leave general feedback and suggestions at the cTools feedback page.

The list of current collaborative and relatively stable R&D tools:

• Collective Mind - completely new plugin-based infrastructure and repository for collaborative R&D in computer engineering that unifies all techniques, tools, benchmarks, data sets and models from below.

• cTuning CC - free, open source compiler collection that combines multiple tools and techniques including MILEPOST GCC, ICI, CCC framework, cTuning web-services and Collective Optimization Database and cBench as the first practical step toward self-tuning, adaptive computing systems based on industrial tools, empirical techniques, transparent collective optimization, statistical analysis and machine learning.

• MILEPOST GCC - machine learning-enabled self-tuning research compiler that correlates program features and optimizations during empirical learning to predict good optimization for unseen programs.

• Interactive Compilation Interface (ICI) - a plugin system with high-level API to transform current compilers into collaborative modular interactive R&D toolsets. GCC is the main supported platform, however there are plans to add ICI to LLVM and other compilers. It is currently used in the MILEPOST GCC. During Google Summer of Code'09, GCC ICI has been extended to support generic function cloning, fine-grain program optimizations (such as selecting unrolling factors, tiling parameters, inlining, etc) and basic program instrumentation. Currently, we are working to move ICI to the mainline GCC to use GCC as a production-quality open-source research compiler.

• Continuous Collective Compilation Framework (CCC) - high-level collaborative plugin-enabled framework to perform iterative compilation (i.e. search for profitable combinations of program optimizations) and gather various static and dynamic optimization profile data in a Collective Optimization Database. It is used to help end-users optimize their programs automatically (improve execution time/code size/compilation time, etc) using direct search as well as various collective, statistical and machine learning techniques. It supports multiple compilers such as GCC, LLVM, Open64, ICC, etc and support both global optimization (compiler flags tuning) and finer-grain optimizations such as function level, loop level, instruction level (collaborative work is still in progress with prototypes available for download) whenever a compiler support Interactive Compilation Interface (ICI) such as MILEPOST GCC. Collective Benchmarks and Datasets (cBench/MiDataSets) can be immediately used with the CCC framework for iterative program optimization on a given architecture. We strongly advice you to use cDatabase when you install CCC framework to be able to use predefined configurations from the community and to use machine learning and collective optimization techniques to predict program optimizations.

• Collective Benchmarks and Datasets (cBench/MiDataSets) - collection of open-source programs and multiple datasets to enable realistic benchmarking and research on program and architecture optimization. All programs support CCC framework to enable collective optimization.

List of collaborative tools in an early development/brainstorming stage:

• Universal Adaptation Framework (UNIDAPT) - hybrid static/dynamic framework to enable run-time adaptation, optimization and scheduling with low overhead for unicore and heterogeneous multi-core architectures (GPGPU, CELL, etc).

• Collaborative Architecture Simulation Platform - we would like to add common API to current simulation platforms such as UNISIM and SimpleScalar, for example. This project has not started yet and any help will be appreciated.

• Collaborative Virtualization Platform - we would like to add common API to current virtualization platforms such as LLVM,GCC4NET,MONO for example. This project is in infancy so any help will be appreciated

• Development Methodology

Archive:

• MILEPOST Framework - infrastructure that combines MILEPOST GCC, CCC Framework, Collective Optimization Database and UNIDAPT Framework to find "good" program optimizations or architectural configurations for reconfigurable processors entirely automatically using statistical and machine learning techniques. After the end of the MILEPOST project in October, 2009, the MILEPOST framework has been fully integrated with cTools. Note: this framework is now fully integrated with the cTuning infrastructure, tools and repository so it is not used/extended anymore on its own.

Note: cTuning is an ongoing evolving project - please be patient and tolerant to the community or join us to help with this collaborative effort!