CDatabase

From cTuning.org

(Difference between revisions)

Current revision

We moved all developments to our new Collective Knowledge Framework in 2015!

cTuning.org is based on the following reference publications: GCC Summit'09, CPE'04, HiPEAC'05, PhD thesis'04, SMART'09, PLDI'10, HiPEAC'09, IJPP'11

cTuning.org - free, open source, collaborative repository and tools for program and architecture characterization and optimization.

We provide public database with statistics about behavior of computer systems, so that community could develop various predictive models explaining this behavior, correlate it with program and system properties and characteristics, build advice systems about how to optimize programs and enable efficient run-time adaptation and architecture reconfiguration

[ View, rank and reference optimization cases from the community ]
[ Share your optimization case with the community ]
[ Find referenced optimization case ]

[ Predict optimizations using ML ]

(Full access to the cDatabase including web-services)

Collective Optimization Database (cDatabase or COD) provides a common global repository with data analysis plugins to share, analyze and reference useful program/architecture optimization cases. It has been developed to help users optimize programs, libraries, kernels and the whole systems (compiler optimizations/architecture configurations to improve execution time, code size, architecture size, power consumption, etc). It is intended to simplify and automate the design and optimization of programs, compilers, run-time systems and architectures based on recent statistical and machine learning techniques (FT2010, FT2009, FMTP2008, ABCP2006, MILEPOST, UNIDAPT). We also hope that it will also be useful for adaptive parallelization and scheduling for the emerging and future heterogeneous multi-core systems including current CPU/GPU and CELL architectures (LCWP2009). Finally, it is intended to improve the quality of academic research by avoiding costly duplicate experiments and providing replicable referable results. It can provide detailed performance analysis and comparison of different programs, datasets, compilers and architectures. It can be used to optimize programs on-the-fly when using cloud computing services.

We currently keep information about architectures and their configurations, software environments, programs, datasets, compilers (such as GCC, Open64, ICC, PathScale and plan to add dynamic compilers such as LLVM, IBM Testerossa, etc), compiler flags for program and architecture optimizations, compiler ICI optimization passes, fine-grain program optimizations, execution time, code size, profiling statistics, program static and dynamic features (hardware counters), parallelization schemes, etc.

cDatabase is an evolving project driven by the community and industry demands - you are welcome to join the project, extend the database and data analysis plugins, provide feedback and add your optimization data to help the community. If you want to use database directly, you can find more info about cDatabase API and web-services in cDatabase documentation. You can also communicate with cTuning community through our mailing lists.

Background:

Documentation: cDatabase API and web-services

User suggestions and feedback

Projects to extend Collective Optimization Database

Copyright and license note

cDatabase friends:

@@ Line 1: / Line 1: @@
-{{CTitle|http://ctuning.org/wiki/images/logo_cdb.gif|Collective Optimization Database|Sharing and reusing optimization knowledge to help tuning computing systems collectively}}
+'''''We moved all developments to our new [http://cKnowledge.org Collective Knowledge Framework] in 2015!'''''
-{|border="0" cellpadding="15" cellspacing="0"
-| valign="top" |
-''It may not always be visible to the IT users, but developing and optimizing current and emerging computing systems using available technology is too time consuming and costly. Collective Optimization Database stores useful optimization cases from the community to help optimize available computing systems (ranging from supercomputers to embedded systems) and enable development of the emerging intelligent self-tuning adaptive computing systems based on collective optimization, run-time adaptation, statistical and machine learning techniques.''
+{{Reference}}
-<div align="center" style="font-size:larger">
+'''cTuning.org - free, open source, collaborative repository and tools for program and architecture characterization and optimization.'''
+''We provide public database with statistics about behavior of computer systems, so that community could develop various predictive models explaining this behavior, correlate it with program and system properties and characteristics, build advice systems about how to optimize programs and enable efficient run-time adaptation and architecture reconfiguration''
+<div align="left" style="font-size:larger">
 {| width="100%" border="0" cellpadding="10"
-| <div align="center" style="font-size:larger">
+| <div align="left" style="font-size:larger">
 [&nbsp;[http://ctuning.org/wiki/index.php/Special:CDatabase?request=view_opt_cases View, rank and reference optimization cases from the community]&nbsp;]<BR>
 [&nbsp;[http://ctuning.org/wiki/index.php/Special:CDatabase?request=add_opt_cases Share your optimization case with the community]&nbsp;]<BR>
 [&nbsp;[http://ctuning.org/wiki/index.php/Special:CDatabase?request=find_opt_case Find referenced optimization case]&nbsp;]<BR><BR>
 [&nbsp;[http://ctuning.org/wiki/index.php/Special:CPredict Predict optimizations using ML]&nbsp;]<BR><BR>
-</div><div align="center" style="font-size:larger">
+</div><div align="left" style="font-size:larger">
-<div align="center" style="font-size:smaller">[[CDatabase:Access|(Full access to the cDatabase including web-services)]] </div>
+<div align="left" style="font-size:smaller">[[CDatabase:Access|(Full access to the cDatabase including web-services)]] </div>
 |}
 </div>
-Collective Optimization Database (cDatabase or COD) provides a common global repository with data analysis plugins to share, analyze and reference useful program/architecture optimization cases. It has been developed to help users optimize programs, libraries, kernels and the whole systems (compiler optimizations/architecture configurations to improve execution time, code size, architecture size, power consumption, etc). It is intended to simplify and automate the design and optimization of programs, compilers, run-time systems and architectures based on recent statistical and machine learning techniques ({{Ref|FT2009}}, {{Ref|FMTP2008}}, {{Ref|ABCP2006}}, [http://www.milepost.eu MILEPOST], [http://unidapt.org UNIDAPT]). We also hope that it will also be useful for adaptive parallelization and scheduling for the emerging and future heterogeneous multi-core systems including current CPU/GPU and CELL architectures ([http://fursin.net/wiki/index.php5?title=Research:Dissemination#LCWP2009 LCWP2009]). Finally, it is  intended to improve the quality of academic research by avoiding costly duplicate experiments and providing replicable referable results. It can provide detailed performance analysis and comparison of different programs, datasets, compilers and architectures. It can be used to optimize programs on-the-fly when using cloud computing services.
+<div align="center">http://ctuning.org/wiki/images/ctuning_services.gif</div>
+Collective Optimization Database (cDatabase or COD) provides a common global repository with data analysis plugins to share, analyze and reference useful program/architecture optimization cases. It has been developed to help users optimize programs, libraries, kernels and the whole systems (compiler optimizations/architecture configurations to improve execution time, code size, architecture size, power consumption, etc). It is intended to simplify and automate the design and optimization of programs, compilers, run-time systems and architectures based on recent statistical and machine learning techniques ({{Ref|FT2010}}, {{Ref|FT2009}}, {{Ref|FMTP2008}}, {{Ref|ABCP2006}}, [http://cTuning.org/project-milepost MILEPOST], [http://unidapt.org UNIDAPT]). We also hope that it will also be useful for adaptive parallelization and scheduling for the emerging and future heterogeneous multi-core systems including current CPU/GPU and CELL architectures ([http://fursin.net/wiki/index.php5?title=Research:Dissemination#LCWP2009 LCWP2009]). Finally, it is  intended to improve the quality of academic research by avoiding costly duplicate experiments and providing replicable referable results. It can provide detailed performance analysis and comparison of different programs, datasets, compilers and architectures. It can be used to optimize programs on-the-fly when using cloud computing services.
 We currently keep information about architectures and their configurations, software environments, programs, datasets, compilers (such as GCC, Open64, ICC, PathScale and plan to add dynamic compilers such as LLVM, IBM Testerossa, etc), compiler flags for program and architecture optimizations, compiler [[CTools:ICI|ICI]] optimization passes, fine-grain program optimizations, execution time, code size, profiling statistics, program static and dynamic features (hardware counters), parallelization schemes, etc.
 cDatabase is an evolving project driven by the community and industry demands - you are welcome to [[Join|join]] the project, [[CDatabase:Projects|extend]] the database and data analysis plugins, provide [[CDatabase:Feedback|feedback]] and add your optimization data to help the community. If you want to use database directly, you can find more info about cDatabase API and web-services in [[CDatabase:Documentation:API|cDatabase documentation]]. You can also communicate with cTuning community through our [[Community|mailing lists]].
-| valign="top" |
-{{News1|
-*'''2009.June.15''' - COD is back online - there were a few configuration problems during migration to a new host within INRIA.
-*'''2009.June.12''' - We are currently experiencing network problems when accessing COD - we will try to sort it out on Monday. Sorry for any inconvenience caused by that.
-*'''2009.June.10''' - Extended version of the "Collective Optimization" paper ({{Ref|FT2009|}}) describing collective tuning concept has been accepted for ACM Transactions on Architecture and Code Optimization (TACO).
-*'''2009.June.03-10''' - We gave several talks/demos/tutorials about cTuning at the [http://www.hipeac.net/industry_workshop7 HiPEAC Computing week] (Infineon, Munich, Germany) and [http://www.gccsummit.org/2009 GCC Summit] (Montreal, Canada).
-* '''2009.June.01''' - After nearly 1 year of developments we released/updated all our open-source collaborative [[CTools|R&D tools]]:
-** fully redesigned and documented [[CTools:ICI|Interactive Compilation Interface]] v2.0 for GCC 4.4.0 synchronized with the official plugin GCC branch - transforming compilers into plugin-enabled research toolsets
-** [[cTools:MilepostGCC|MILEPOST GCC 4.4.0]] pre-release version at SVN - automating program optimization and compiler optimization heuristic tuning using machine learning
-** [[CTools:CCC|Continuous Collective Compilation Framework v2.0]] - enabling automatic collaborative program optimization based on statistical and machine learning techniques
-** [[CTools:CBench|Collective Benchmark/MiDataSets v1.0]] - enabling realistic program optimization research and benchmarking using multiple open-source programs/datasets.<BR><BR>We also updated [[CDatabase|Collective Optimization Database]] with various optimization cases for Intel and AMD processors and comparison of different compilers including GCC, LLVM, Open64, Intel, etc - enabling sharing and reuse of optimization knowledge.<BR><BR>We would like to thank [[Community:People|cTuning community]] for feedback, help and support! You are welcome to join this [[Community|community effort]] to automate program optimization and compiler/architecture design.
-* '''2009.April.27''' - We gave a talk at the University of Illinois at Urbana-Champaign about [http://ctuning.org Collective Tuning Initiative and tools] and [http://www.milepost.eu MILEPOST project] ("Collective Optimization, run-time adaptation and machine learning"). Presentation is available [http://unidapt.org/presentations/presentation_fursin_uiuc2009.pdf here]. We would like to thank all the UIUC&nbsp;colleagues for a very interesting and useful feedback.<br>
-* '''2009.April.23''' - First version of optimization predictor based on static program features is now available [http://ctuning.org/wiki/index.php/Special:CPredict on-line]. It is an on-going project, so please be patient. Comments are welcome!
-* '''2009.April.21''' - [http://socghop.appspot.com/org/home/google/gsoc2009/gcc Several projects] to enable automatic fine-grain program optimization and run-time adaptation in GCC using iterative compilation and machine learning (based on our cTuning/UNIDAPT/ICI/MILEPOST technology) have been accepted by the Google Summer of Code. You are welcome to [[Join|join cTuning community]] and follow or participate in the developments using our dedicated [[Community|mailing lists]].
-* '''2009.March.16''' - You are welcome to check our new [[CDatabase:Projects|cDatabase projects page]] if you would like to help us extend the database and community services or you can provide your  [[CDatabase:Feedback|cDatabase feedback and suggestions]]!<br><br>
-* '''2009.January.20''' - We went off-line to move to a new hosting and a new Mediawiki platform.<br><br>
-* '''2008.October.17''' - First public version of cDatabase is ready.<br><br>
-* '''2007.March.03''' - cDatabase has been extended for the MILEPOST project.<br><br>
-}}
-|}
 ----
@@ Line 75: / Line 49: @@
 |-
 | align="center" valign="top" | {{CDatabase:Logo_Friends}}
-<div align="center"><br>''You are welcome to register your interest at this [[Community:Friends|page]]''.</div>
 |}
 </div>