CTools:CBench

From cTuning.org

(Difference between revisions)

Revision as of 10:39, 7 August 2009

	Collective Benchmark
	*Enabling realistic benchmarking and optimization*

Navigation: cTuning.org > CTools

Collective Benchmark (cBench) is a collection of open-source sequential and parallel programs with multiple datasets assembled by the community to enable realistic benchmarking and research on program and architecture optimization. The source code of individual programs is simplified to ease portability. All the benchmarks now include scripts to be used with Continuous Collective Compilation Framework to perform automatic optimizations (iterative compilation) using GCC, LLVM, GCC4CIL, Open64, PathScale, Intel and other compilers and a wide range of architectures. This collection of programs can later be used to create specialized benchmarks.

cBench is an evolving project driven by the community demands - you are welcome to join the project, extend or add benchmarks and datasets and provide performance analysis, leave feedback and add your optimization data to the Collective Optimization Database to help the community optimize their programs. You can also communicate with cTuning community through our mailing lists.

Background:
- Motivation and history
- Resources (links, publications, tutorials)

Releases:

User suggestions and feedback

Projects to extend cBench (development plan and suggestions) - > 5 projects

Current development:

News

2009.August.05 - The colleagues from the UNIDAPT Group started investigating the use of cTuning/MILEPOST technology and the UNIDAPT framework to predict good optimization and parallelization schemes for hybrid heterogeneous CPU/GPU-like architectures based on statistical analysis, machine learning, program and dataset features and run-time decision trees together with CAPS Entreprise. They plan to add new optimization cases to the Collective Optimization Database in Autumn, 2009.

2009.July.27 - The paper "Portable Compiler Optimization Across Embedded Programs and Microarchitectures using Machine Learning" (DJBP2009) has been accepted for the 42nd IEEE/ACM International Symposium on Microarchitecture (MICRO). The research has been led by the colleagues from the University of Edinburgh - congratulations!

2009.July.22 - Grigori Fursin is a co-organizer of SMART'10 and GROW'10 workshops that will be co-located with HiPEAC'10 conference in Pisa, Italy at the end of January, 2009. You are warmly invited to submit your novel research results and developments to our workshops! At the websites of these workshops you can find all information about topics, PC, deadlines and submission procedures.

2009.July.15 - New optimization cases to improve execution time, code size and compilation time are now available in Collective Optimization Database for cBench, EEMBC, SPEC and other applications with multiple datasets using MILEPOST GCC 4.4.0 and 2 architectures (Intel and AMD) from GRID5000 (infrastructure for large scale parallel and distributed computing research).

2009.July.08 - The preprint of a paper "Collective Tuning Initiative: automating and accelerating development and optimization of computing systems" describing cTuning infrastructure is now available online (Fur2009).

2009.June.29 - Stable MILEPOST GCC 4.4.0 has been released. Follow further community developments at cTuning GCC ICI page and cTuning development mailing list.
Next, we plan to use MILEPOST/cTuning technology to enable realistic adaptive parallelization, data partitioning and scheduling for heterogeneous multi-core systems using statistical and machine learning techniques.

2009.June.26 - The pdf of the paper that describes Collective Tuning Infrastructure and cTuning concept (presented at the GCC Summit'09) will be available in a few weeks here.

2009.June.10 - Extended version of the "Collective Optimization" paper (FT2009) describing collective tuning concept has been accepted for ACM Transactions on Architecture and Code Optimization (TACO).

2009.June.01 - After nearly 1 year of developments we released/updated all our open-source collaborative R&D tools:
- fully redesigned and documented Interactive Compilation Interface v2.0 for GCC 4.4.0 synchronized with the official plugin GCC branch - transforming compilers into plugin-enabled research toolsets
- MILEPOST GCC 4.4.0 pre-release version at SVN - automating program optimization and compiler optimization heuristic tuning using machine learning
- Continuous Collective Compilation Framework v2.0 - enabling automatic collaborative program optimization based on statistical and machine learning techniques
- Collective Benchmark/MiDataSets v1.0 - enabling realistic program optimization research and benchmarking using multiple open-source programs/datasets.
  
  We also updated Collective Optimization Database with various optimization cases for Intel and AMD processors and comparison of different compilers including GCC, LLVM, Open64, Intel, etc - enabling sharing and reuse of optimization knowledge.
  
  We would like to thank cTuning community for feedback, help and support! You are welcome to join this community effort to automate program optimization and compiler/architecture design.

2009.April.27 - We gave a talk at the University of Illinois at Urbana-Champaign about Collective Tuning Initiative and tools and MILEPOST project ("Collective Optimization, run-time adaptation and machine learning"). Presentation is available here. We would like to thank all the UIUC colleagues for a very interesting and useful feedback.

2009.April.21 - Several projects to enable automatic fine-grain program optimization and run-time adaptation in GCC using iterative compilation and machine learning (based on our cTuning/UNIDAPT/ICI/MILEPOST technology) have been accepted by the Google Summer of Code. You are welcome to join cTuning community and follow or participate in the developments using our dedicated mailing lists.

2009.March.07 - cBench v1.0 (MiDataSets v1.4) is in the final verification stage before the release.

2009.March.02 - Google Summer of Code'2009 will be accepting development applications soon - you are welcome to check cBench extension projects and cBench feedback/suggestion pages if you would like to help us to extend our collaborative tools within this program!

Old news archive (2006-2008)

cBench is an open collaborative community-driven project. You are welcome to join us and participate in discussions, developments or provide feedback and suggestions to extend cBench or add new benchmarks/datasets.

@@ Line 35: / Line 35: @@
 {{News|
+*'''2009.August.05''' - The colleagues from the [http://unidapt.org UNIDAPT Group] started investigating the use of [http://ctuning.org cTuning]/[http://www.milepost.eu MILEPOST] technology and the [http://ctuning.org/unidapt UNIDAPT framework] to predict good optimization and parallelization schemes for hybrid heterogeneous CPU/GPU-like architectures based on statistical analysis, machine learning, program and dataset features and run-time decision trees together with [http://www.caps-entreprise.com CAPS Entreprise]. They plan to add new optimization cases to the [http://ctuning.org/cdatabase Collective Optimization Database] in Autumn, 2009.
+*'''2009.July.27''' - The paper "Portable Compiler Optimization Across Embedded Programs and Microarchitectures using Machine Learning" ({{Ref|DJBP2009}}) has been accepted for the [http://www.microarch.org/micro42 42nd IEEE/ACM International Symposium on Microarchitecture (MICRO)]. The research has been led by the colleagues from the University of Edinburgh - congratulations!
+*'''2009.July.22''' - [http://fursin.net/research Grigori Fursin] is a co-organizer of [http://ctuning.org/workshop-smart10 SMART'10] and [http://ctuning.org/workshop-grow10 GROW'10] workshops that will be co-located with [http://www.hipeac.net/conference HiPEAC'10 conference] in Pisa, Italy at the end of January, 2009. You are warmly invited to submit your novel research results and developments to our workshops! At the websites of these workshops you can find all information about topics, PC, deadlines and submission procedures.
+*'''2009.July.15''' - New optimization cases to improve execution time, code size and compilation time are now available in [http://ctuning.org/cdatabase Collective Optimization Database] for [http://ctuning.org/cbench cBench], [http://www.eembc.org EEMBC], [http://www.spec.org SPEC] and other applications with multiple datasets using [http://ctuning.org/milepost-gcc MILEPOST GCC 4.4.0] and 2 architectures ([http://www.intel.com Intel] and [http://www.amd.com AMD]) from [http://www.grid5000.fr GRID5000] (infrastructure for large scale parallel and distributed computing research).
 *'''2009.July.08''' - The preprint of a paper "Collective Tuning Initiative: automating and accelerating development and optimization of computing systems" describing cTuning infrastructure is now available online ({{Ref|Fur2009}}).