|
|
| (9 intermediate revisions not shown.) |
| Line 1: |
Line 1: |
| - | Continuing innovation in science and industry requires ever-increasing computing resources while placing strict requirements on system performance, power consumption, size, response, reliability, portability and design time. Both embedded and large-scale systems evolve toward complex heterogeneous multi-chip systems with dramatically increased design, test and optimization time. Optimizing compilers play a key role in producing executable codes quickly and automatically while satisfying all the above requirements for a broad range of programs and architectures. However, for many years, state-of-the-art compilers fail to deliver portable performance due to necessarily simplistic hardware models, fixed and black-box optimization heuristics, inability to tune application at fine-grain level, large optimization spaces, highly dynamic behavior of the system and inability to adapt to varying program and system behavior at run time with low overhead. Possibility for architecture reconfiguration makes system optimization problem even worse. This suggests that current compiler and architecture design and program optimization technologies are reaching their limits and should be revisited to keep pace with rapidly evolving hardware. We believe that compiler and architecture design should be simplified and automated using iterative feedback-directed compilation techniques, collective optimization, run-time adaptation, statistical analysis and machine learning.
| + | Moved [http://cTuning.org/about.html here] |
| - | | + | |
| - | The roots of the cTuning project go back to the [http://cordis.europa.eu/esprit/src/22729.htm Oceans project] started in 1996 to investigate empirical iterative compilation for embedded applications {{Ref|BBBP1998}}. It was followed by the [http://cordis.europa.eu/esprit/src/24942.htm MHAOTEU project] where memory hierarchy analysis and optimisation tools for large applications had to be developed. Grigori Fursin joined this project in 1999 as a PhD student and had to automate program optimizations using iterative compilation. It was difficult to continue this research since there was a lack of stable, powerful, extensible, open-source compiler and optimization tools. After having a disappointing experience using multiple incompatible unstable research tools and unsupported third-party software, Grigori decided to develop open-source research tools with common APIs based on production-quality compilers and Collective Optimization Database to enable collaborative and replicable R&D and avoid duplicate developments. During development collaborative tools have been tested, used and extended to investigate practical iterative compilation techniques to optimize programs (on a program, function, loop or instruction level) or architectures automatically, speed them up using statistical and machine learning techniques, automate design space exploration and enable statically compiled adaptive applications within [http://www.icsa.informatics.ed.ac.uk/compilers CARD Group], [https://alchemy.saclay.inria.fr/site Alchemy Group], [http://unidapt.org UNIDAPT Group]. Here is the [[Community:People|list]] of all colleagues contributing to cTools and cResearch.
| + | |
| - | | + | |
| - | We believe that current way of disseminating novel techniques only through publications without providing tools to verify and replicate results or with incompatible unstable prototypes is intended mainly to claim the authorship of the idea in a niche research area, but often do not help the community to use these techniques immediately in a collaborative research to continue addressing global challenges. Hence, we decided to disseminate our open-source research tools with common APIs to help the community with the collaborative and replicable R&D while avoiding costly, time-consuming duplicate developments. Using Collective Optimization Database to share and reuse interesting optimization cases also intended to improve the quality of academic research since it enables collaborative, replicable and referenced experiments and results.
| + | |
| - | | + | |
| - | Using collaborative R&D tools also accelerated our research and resulted in a [http://www.milepost.eu MILEPOST project] where we developed collaborative [http://unidapt.org/index.php/Developments:MILEPOST_Framework MILEPOST Framework] that includes machine learning enabled self-tuning compiler, Collective Optimization Database and Continuous Collective Compilation Framework to enable learning and predicting of good compiler optimizations or architecture configurations {{Ref|FMTP2008}}, {{Ref|FT2009}}. In this project, we obtained promising results optimizing realistic codes (programs/libraries/OS modules) for performance/code size for IBM servers and different configurations of ARC processors. We now believe that the Collective Optimization concept can be useful to the IT community and we created cTuning.org project to disseminate this technology and extend it collaboratively. <br>
| + | |
| - | <br>
| + | |
| - | | + | |
| - | Nowadays, cTuning.org is a community-driven collaborative Mediawiki-based project that brings together industry, academia and IT users to optimize programs and architectures collectively and build intelligent self-tuning adaptive computing systems based on statistical and machine learning techniques. We are participating in the following collaborative activities: <br>
| + | |
| - | | + | |
| - | *Develop common open-source tools with unified APIs (universal compiler for heterogeneous architectures, computer architecture simulators, run-time systems) to optimize programs and architectures collectively using iterative compilation, statistical and machine learning techniques.<br> <div align="right">''More information is available at [[CTools|cTools page]].''</div>
| + | |
| - | *Share interesting optimization cases from the community for programs/libraries/OS (compiler optimizations/architecture configurations to improve execution time, code size, architecture size, power consumption, etc) in the Collective Optimization Database to help users optimize their systems, enable replicability of the results and improve the quality of academic research. <br><div align="right">''More information is available at [[CDatabase|cDatabase page]].''</div>
| + | |
| - | *Enable collaborative research using cTools to automate and simplify the process of developing and optimizing new computer architectures, compilers, operating systems and programming environments using statistical analysis, machine learning, dynamic adaptation and bio-inspired techniques. We believe that our adaptive approaches are critical to overcome the complexity of computing systems and improve their performance, power consumption, system size and fault-tolerance automatically while reducing their cost and time to market. <br><div align="right">''More information is available at [[CResearch|cResearch page]].''</div>
| + | |
| - | | + | |
| - | Everyone is welcome to use cTools for their R&D, browse and update cDatabase, provide feedback or [[Join|join]] cTuning initiative to actively participate in this collaborative effort, extend cTools, cResearch and propose your new collaborative R&D projects for the cTuning community. We hope that this community effort will boost innovation in compiler, OS and architecture technology, provide novel adaptive mechanisms for heterogeneous, reconfigurable multi-core systems and emerging technologies such as cloud computing, will simplify program and architecture design and optimization, reduce system cost and and will eventually have a positive effect on science and industries that demand ever-increasing computing resources while placing strict requirements on systems.
| + | |