|
|
| Line 1: |
Line 1: |
| | + | {{CTitle|http://ctuning.org/wiki/images/logo_milepost_gcc.gif|MILEPOST GCC|machine-learning enabled self-tuning compiler}} |
| | + | |
| | + | {|border="0" cellpadding="15" cellspacing="0" |
| | + | | valign="top" | |
| | + | |
| | {{CMenu:CTools|}} | | {{CMenu:CTools|}} |
| | + | |
| | + | MILEPOST GCC is an attept to build the first machine learning enabled open-source self-tuning research and production compiler that can adapt to any architecture using iterative feedback-directed compilation, machine learning and collective optimization. It is based on production quality [http://gcc.gnu.org GCC] that supports more than 30 families of architectures and can compile real, large applications including Linux, and on [[CTools:ICI|Interactive Compilation Interface]] that provides plugin system to access internals of compilers. MILEPOST GCC attempts to correlate program features and program optimizations during empirical iterative compilation to predict good optimizations for unseen programs based on prior learning. MILEPOST and cTuning infrastructure automates code and architecture optimization to improve execution time, code size, compilation time and other characteristics at the same time. This technology is not GCC-dependent and can be used in any compiler using common [[CTools:ICI|Interactive Compilation Interface]] and compiler independent plugins. More information can be found in the following paper about [http://unidapt.org/index.php/Dissemination#FMTP2008 MILEPOST GCC]. |
| | | | |
| | <div align="center" style="margin: 8px; float: right;"> | | <div align="center" style="margin: 8px; float: right;"> |
| Line 5: |
Line 12: |
| | </div> | | </div> |
| | | | |
| - | * '''[[CTools:ICI:Downloads|Download MILEPOST GCC 4.4.0]]''' | + | * '''Background:''' |
| | + | **[[CTools:MilepostGCC:Motivation|Motivation and history]] |
| | + | **[[CTools:MilepostGCC:Usage|Usage scenarios / stories]] |
| | + | **[[CTools:MilepostGCC:Resources|Resources (links, publications, etc)]] |
| | | | |
| - | * '''[http://ctuning.org/wiki/index.php/Special:CPredict Access online optimization predictor based on program features] | + | * [[CTools:MilepostGCC:Downloads|'''Downloads''']] |
| - | * '''[[CDatabase|Access collective optimization repository]]
| + | |
| | | | |
| - | * '''[[CTools:MilepostGCC:StaticFeatures|List]] of static program features available in MILEPOST GCC | + | * [[CTools:MilepostGCC:Documentation|'''Documentation''']] |
| | | | |
| - | * '''[[CTools:ICI|Further developments/documentation (within GCC ICI)]] | + | * '''[[CTools:MilepostGCC:Feedback|User suggestions and feedback]]''' |
| - | * '''[http://groups.google.com/group/ctuning-discussions Development mailing list]
| + | |
| | | | |
| - | * [http://socghop.appspot.com/org/home/google/gsoc2009/gcc '''Google Summer of Code'09 MILEPOST GCC extensions and plugins''' (XML representation of the compilation flow, fine-grain optimizations and instrumentation, polyhedral transformations, run-time adaptation)] | + | * '''[[CTools:MilepostGCC:Projects|Projects to extend CCC Framework (development plan)]]''' |
| - | ** [[CTools:ICI:Projects:Fine_Grain_Optimizations|Development page for fine-grain tuning from GSOC'09]]
| + | |
| - | *** [http://ctuning.org/wiki/index.php/CTools:ICI:Projects:GSOC09:Fine_grain_tuning documentation]
| + | |
| - | ** [[CTools:ICI:Projects:Cloning|Development page for function cloning from GSOC'09]]
| + | |
| - | *** [http://ctuning.org/wiki/index.php/CTools:ICI:Projects:GSOC09:Function_cloning_and_program_instrumentation documentation]
| + | |
| | | | |
| - | The development of the MILEPOST GCC and MILEPOST Framework has been coordinated by [http://fursin.net/research Dr. Grigori Fursin] ([http://unidapt.org UNIDAPT group], [http://www.inria.fr/saclay INRIA, France]) within [http://www.milepost.eu EU FP6 MILEPOST project] (2006-2009). MILEPOST consortium includes INRIA, IBM Haifa, University of Edinburgh (project coordinator - Prof. Michael O'Boyle), ARC International Ltd. and CAPS Entreprise. Please, use our [http://groups.google.com/group/ctuning-discussions cTuning development mailing list] if you have have any technical questions or comments about MILEPOST framework or cTuning initiative. You are welcome to contact Grigori if you have general questions about MILEPOST and cTuning research directions.
| + | * '''Current development:''' |
| | + | **[[CTools:MilepostGCC:Dev:Source|Getting started / Source code]] |
| | + | **[[CTools:MilepostGCC:Dev:Discussions|Discussion forum]] |
| | + | **[[CTools:MilepostGCC:Dev:Bugs|Bug reports]] |
| | | | |
| - | MILEPOST GCC is the first machine learning enabled open-source self-tuning research compiler that can adapt to any architecture using iterative feedback-directed compilation, machine learning and collective optimization. It combines the strength of the production quality [http://gcc.gnu.org GCC] that supports more than 30 families of architectures and can compile real, large applications including Linux, and the flexibility of the [[CTools:ICI|Interactive Compilation Interface]] that transforms GCC into a research compiler. It is currently based on predictive modeling using program and machine-specific features, execution time, hardware counters and off-line training. MILEPOST GCC includes [[CTools:MilepostGCC:StaticFeatures|static program feature extractor]] developed by IBM Haifa. MILEPOST/cTuning technology is orthogonal to GCC and can be used in any future adaptive self-tuning compiler using common [[CTools:ICI|Interactive Compilation Interface]]. MILEPOST and cTuning infrastructure automates code and architecture optimization to improve execution time, code size, compilation time and other characteristics at the same time.
| + | | |
| | | | |
| - | ''Unlike other tools and projects that are either commercial, non open-source, exist only in publications or as unstable prototypes, cTuning/MILEPOST is the first open-source practical community-driven attempt to apply machine learning, statistical collective optimization and run-time adaptation inside a stable, production-quality compiler in order to simplify and automate the development of compilers, architectures, run-time systems and programs, and enable future self-optimizing smart computing systems''. | + | {{News| |
| | + | <BR> |
| | + | :::::: ''[http://groups.google.com/group/ctuning-announce News archive]'' |
| | | | |
| - | In June, 2009, MILEPOST GCC [[CTools:ICI:Downloads|has been released]] and all further developments have been integrated with the [http://ctuning.org/ctools cTuning tools]: [http://ctuning.org/cdatabase Collective Optimization Database], [http://ctuning.org/cpredict cTuning optimization prediction web-services], [http://ctuning.org/ici Interactive Compilation Interface for GCC], [http://ctuning.org/ccc Continuous Collective Compilation Framework] to enable collaborative community-driven developments after the end of the [http://www.milepost.eu MILEPOST project] (August 2009). You are warmly welcome to join [http://ctuning.org/wiki/index.php/Community cTuning community] and follow/participate in developments and discussions using cTuning Wiki-based portal and 2 mailing lists: [http://groups.google.com/group/ctuning-discussions high volume development list] and [http://groups.google.com/group/ctuning-announce low volume announcement list].
| + | }} |
| | | | |
| - | We don't claim that MILEPOST GCC, MILEPOST Framework and cTuning tools can solve all optimization problems ;) but we believe that having an open research-friendly extensible compiler with machine learning and adaptive plugins based on production quality GCC that supports multiple languages and architectures opens up many research opportunities for the community and is the first practical step towards our long-term objective to enable adaptive self-tuning computing systems. With the help of the community, we hope to provide better validation of code correctness when applying complex combinations of optimizations, provide plugins for XML representation of the compilation flow, tuning of fine-grain optimizations/polyhderal GRAPHITE transformations/link-time optimizations, code instrumentation and run-time adaptation capabilities for statically compiled programs ([http://socghop.appspot.com/org/home/google/gsoc2009/gcc see Google Summer of Code'09 program]). We would also like to add support to MILEPOST GCC/cTuning tools to be able to optimize whole Linux ([http://gentoo.org Gentoo]-like) or optimize programs for mobile systems on the fly (for example, using [http://www.android.com/ Android], [http://moblin.org Moblin], etc) and extend this technology to enable realistic adaptive parallelization, data partitioning and scheduling for heterogeneous multi-core systems using statistical and machine learning techniques.
| + | |} |
| | | | |
| - | We are very grateful to [[Community:People|all our colleagues and users]] for providing valuable feedback or contributing to the cTuning/MILEPOST projects.
| + | ---- |
| | | | |
| - | '''''Note:''' cTuning is an ongoing evolving project - please be patient and tolerant to the community and help us with this collaborative effort!'' | + | ''You are welcome to [[Join|join]] us and participate in CCC developments, discussions, provide feedback or suggestions to extend CCC or add new functionality. The framework currently supports multiple compilers including GCC, Open64, PathScale, Intel ICC, IBM XLC and a large number of server, desktop and embedded architectures.'' |
| | | | |
| - | '''Some MILEPOST/cTuning related publications:'''<BR>
| + | <div align="center"> |
| - | * [http://unidapt.org/index.php/Dissemination#FMTP2008 "MILEPOST GCC: machine learning based research compiler"]
| + | {| cellspacing="0" cellpadding="3" border="0" width="100%" style="color: black;" |
| - | * [http://unidapt.org/index.php/Dissemination#FT2009 "Collective Optimization"]
| + | |- |
| - | * [http://unidapt.org/index.php/Dissemination#Fur2009 "Collective Tuning Initiative: automating and accelerating development and optimization of computing systems"]
| + | | align="center" style="background: rgb(0, 100, 159) none repeat scroll 0% 0%; -moz-background-clip: -moz-initial; -moz-background-origin: -moz-initial; -moz-background-inline-policy: -moz-initial; color: white;" | '''CCC supporters:''' |
| - | | + | |- |
| - | '''Brief motivation:'''<BR>
| + | | align="center" valign="top" | {{CTools:CCC:Logo_Friends}} |
| - | It may not always be visible to the IT users, but developing and optimizing current and emerging computing systems using available technology is too time consuming and costly.Tuning hardwired compiler optimizations for rapidly evolving hardware makes porting an optimizing compiler for each new platform extremely challenging. Our radical approach is to develop a modular, extensible, self-tuning intelligent compiler that automatically learns the best optimization heuristics based on combining feedback-directed iterative compilation and machine learning. MILEPOST GCC is a machine learning based compiler that automatically adjusts its optimization heuristics to improve execution time, code size, or compilation time of specific programs on multiple architectures.
| + | <div align="center"><br>''You are welcome to register your interest at this [[Community:Friends|page]]''.</div> |
| - | | + | |} |
| - | Currently, we use several iterative search strategies within CCC framework to find combinations of good optimization flags to substitute GCC default optimization levels for a particular architecture (such as -O0,-O1,-O2,-O3,-Os which we will not need in the future adaptive compilers) or tune optimization passes on a function-level for a particular program. Our preliminary experimental results (some are now available in the [[CDatabase|Collective Optimization Database]]) show that it is possible to considerably reduce execution time and code size of various benchmarks (MiBench, MediaBench, EEMBC, SPEC) on a range of platforms (x86, x8664, IA64, ARC, Loongson/Godson, etc) entirely automatically.
| + | </div> |
| - | | + | |
| - | Typical non-trivial distribution of optimization points in the 2D space of speedups vs code size of a ''susan_corners'' program on AMD Athlon64 3700+ architecture with GCC 4.2.2 during automatic program optimization using ''ccc-run-glob-flags-rnd-uniform'' plugin from [[CTools:CCC|CCC framework]] with uniform random combinations of more than 100 global compiler flags (each flag has 50% probability to be selected for a given combination of optimizations). Similar data for other benchmark, datasets and architectures is available in the [[CDatabase|Collective Optimization Database]]. cTuning technology helps users find or predict optimial optimization points in such complex spaces using "one-button" approach.<BR>
| + | |
| - | <div align="center">http://ctuning.org/wiki/images/example_speedups_susan_c_amd64.gif</div> | + | |
| - | | + | |
| - | MILEPOST GCC can be used interactively in research on adaptive computing through the [[CTools:ICI|Interactive Compilation Interface]] (we currently support optimization pass selection and reordering, plugins and event mechanism to invoke any passes on demand (such as program feature extraction pass) or tune cost-models of specific transformations within passes). [[CTools:CCC|CCC framework]] provides a milepost-gcc wrapper that detects -ml flag, extracts program features, queries [[CDatabase|optimization prediction web-service connected to optimization repository]] and substitutes default optimizations with the suggested ones based on program similarities and machine learning to improve execution time or code size on the fly.
| + | |
| - | | + | |
| - | We hope that this technology will considerably minimize repetitive time consuming tasks and human intervention when optimizing computing systems and will reduce time to market for new products thus boosting innovation and novel research.
| + | |
