(53 intermediate revisions by the same user not shown)
Line 3: Line 3:
 
|-
 
|-
 
| style="text-align: right width: 200px" | [[File:Ae-stamp-cgo.png]]<br/>
 
| style="text-align: right width: 200px" | [[File:Ae-stamp-cgo.png]]<br/>
−
| <p style="text-align: center"><span style="font-size:xx-large">'''<span style="font-family: tahoma,geneva,sans-serif">Artifact Evaluation for CGO'15</span>'''</span></p><p style="text-align: center"><span style="font-size:small">[ [http://cgo.org/cgo2015 Back to CGO'15 conference website] ] [ [[Reproducibility|Back to reproducibility wiki]] ]</span></p>
+
| <p style="text-align: center"><span style="font-size:xx-large">'''<span style="font-family: tahoma,geneva,sans-serif">Artifact Evaluation for CGO'15</span>'''</span></p><p style="text-align: center"><span style="font-size:small">[ [http://cgo.org/cgo2015 Back to CGO'15 conference website] ]</span></p>
 
|}
 
|}
  
−
{| style="width: 95%" border="1" cellpadding="15" cellspacing="1"
+
----
 +
'''<span style="font-size:large;  font-family: tahoma,geneva,sans-serif">Prizes for 2 highest ranked artifacts</span>'''
 +
 
 +
{| border="0" cellpadding="5" cellspacing="1"
 
|-
 
|-
−
| valign="top" | <div class="span4" >
+
| style="text-align: center;  width:450px" valign="top" |
−
<span style="font-size:large">'''<span style="font-family: tahoma,geneva,sans-serif">Important dates</span>'''</span>
+
<span style="color:#ff0000">'''<span style="font-size: larger">1st place</span>'''</span> sponsored by<br/>[[File:NVLogo 2D resize.jpg|NVLogo 2D resize.jpg|link=http://nvidia.com]]
  
 +
| style="text-align: center;  width:450px" valign="top" |
 +
<span style="color:#ff0000"><span style="font-size: larger">'''2nd place'''</span></span> sponsored by<br/>[[File:CTuning foundation logo1a.png|CTuning foundation logo1a.png|link=http://cTuning.org]]
  
 +
|-
 +
| style="text-align: center" |
 +
"Nvidia Quadro K6000" for<br/>'''"The SprayList: A scalable relaxed priority queue"'''<br/>Justin Kopinsky, Dan Alistarh, Jerry Li&nbsp; and Nir Shavit
 +
 +
| style="text-align: center" |
 +
"Acer C720P" for<br/>'''"A graph-based higher-order intermediate representation"'''<br/>Roland Leißa, Marcel Köster and Sebastian Hack&nbsp;
 +
 +
|}
 +
----
 +
 +
{| style="width: 97%" border="0" cellpadding="20" cellspacing="1"
 +
|-
 +
| style="width: 25%" valign="top" | <div class="span4">
 +
<span style="font-size:large">'''<span style="font-family: tahoma,geneva,sans-serif">Important dates</span>'''</span>
 +
 +
*CGO paper decision:<br/><strike>'''10 Nov 2014'''</strike>
 +
*Artifact submission:<br/><strike>'''18 Nov 2014 AoE'''</strike>
 +
*Technical clarification:<br/><strike>'''10-12 Dec 2014'''</strike>
 +
*Decision announced:<br/><strike>'''16 Dec 2014'''</strike>
 +
*Public discussion:<br/><strike>'''9 Feb 2015'''</strike> [ [http://www.slideshare.net/GrigoriFursin/presentation-fursin-aecgoppopp2015 Slides] ]
  
 
<span style="font-size:large">'''<span style="font-family: tahoma,geneva,sans-serif">Packaging guidelines</span>'''</span>
 
<span style="font-size:large">'''<span style="font-family: tahoma,geneva,sans-serif">Packaging guidelines</span>'''</span>
Line 17: Line 42:
 
We use the following [http://www.artifact-eval.org/guidelines.html guidelines for artifacts]. Because we want to encourage as many of you as possible to submit, these are only guidelines, not hard rules. Of course, if the material you want us to evaluate is fairly standard, following these suggestions will greatly simplify our lives. After all, remember: ''You '''want''' us to be able to evaluate your artifact, don't you?''
 
We use the following [http://www.artifact-eval.org/guidelines.html guidelines for artifacts]. Because we want to encourage as many of you as possible to submit, these are only guidelines, not hard rules. Of course, if the material you want us to evaluate is fairly standard, following these suggestions will greatly simplify our lives. After all, remember: ''You '''want''' us to be able to evaluate your artifact, don't you?''
  
−
<span style="font-size:large">'''<span style="font-family: tahoma,geneva,sans-serif">How to submit</span>'''</span>  
+
<span style="font-size:large">'''<span style="font-family: tahoma,geneva,sans-serif">How to submit</span>'''</span>
  
−
Please read the [http://www.artifact-eval.org/guidelines.html guidelines] on ''what'' to submit. Please upload your submission to [https://easychair.org/conferences/?conf=aecgo2015 EasyChair].<br/><span style="font-size:large">'''<span style="font-family: tahoma,geneva,sans-serif">CGO'15 AEC</span>'''</span>
+
Please read the [http://www.artifact-eval.org/guidelines.html guidelines] on ''what'' to submit. Please upload your submission to [https://easychair.org/conferences/?conf=aecgoppopp2015 EasyChair] (common with PPoPP).
  
−
The Artifact Evaluation committe committee consists of several up-and-coming researchers with [http://www.cs.utah.edu/%7Eeeide/ Eric Eide], [http://www.cs.brown.edu/%7Esk/ Shriram Krishnamurthi], and [http://www.cs.purdue.edu/homes/jv/ Jan Vitek] heading the process.
+
<span style="font-size:large">'''<span style="font-family: tahoma,geneva,sans-serif">Review process</span>'''</span>
  
−
See the [http://pldi14-aec.cs.brown.edu/committee/index.html committee members]!
+
The review process is described in detail [http://www.artifact-eval.org/review-process.html here].
 +
</div>
 +
| style="width: 35%" valign="top" |
 +
<span style="font-size:large">'''<span style="font-family: tahoma,geneva,sans-serif">Artifact Evaluation Committee (AEC)</span>'''</span>
  
−
<span style="font-size:large">'''<span style="font-family: tahoma,geneva,sans-serif">Prior Artifact Evaluation</span>'''</span>
+
'''Chairs:'''
−
Here's a [http://cs.brown.edu/%7Esk/Memos/Conference-Artifact-Evaluation/ report] on the AEC process for ESEC/FSE 2011. You can also see the AEC sites for [http://ecoop13-aec.cs.brown.edu/ ECOOP 2013] and [http://splashcon.org/2013/cfp/665 OOPSLA 2013].<br/></div>
+
−
| valign="top" | 
+
−
<span style="font-size:large">'''<span style="font-family: tahoma,geneva,sans-serif">Process</span>'''</span>
+
  
−
Artifact evaluation is open ''only'' to accepted papers. This is intentional: it ensures that the AEC cannot influence whether or not a paper is accepted. This measure was put in place to reassure authors who felt this would be too radical a change to the process of evaluating conference paper submissions.
+
*[http://cTuning.org/lab/people/gfursin Grigori Fursin], cTuning foundation, France
 +
*[http://people.cs.pitt.edu/%7Echilders Bruce R. Childers], University of Pittsburgh, USA
  
−
Of course, this doesn't mean you can't start getting ready! We have published the packaging guidelines, so you can begin to prepare your artifacts for submission. Even if your paper doesn't get accepted, this should still be a useful exercise for general dissemination. In addition, we hope the guidelines will give you ideas about how your material can be disseminated, either by suggesting methods you hadn't considered or pointing you to resources you weren't aware of.
+
'''Advisory board:'''
  
−
As soon as papers are chosen for the conference, you will be invited to submit. We will then have a few weeks in which to perform the evaluation. We may contact you if we have a problem with the submission, but there is no formal “response” period.
+
*&nbsp;[http://www.cs.brown.edu/%7Esk/ Shriram Krishnamurthi], Brown University, USA
  
−
Thus, even though there is nothing formally to do now, you would do well to have your artifact packaged and ready to go. The moment you receive the good news, you should be ready to upload!
+
'''Committee members:'''
  
−
| valign="top" |
+
*David Boehme, LLNL, USA
−
== Papers with artifacts above threshold<br/> ==
+
*Santiago Bock, University of Pittsburgh, USA
 +
*Lingda Li, Rutgers University, USA
 +
*Lin Ma, Huawei America Research Lab, USA
 +
*Yiannis Nikolakopulos, Chalmers University, Sweden
 +
*Jeeva Paudel, University of Alberta, Canada
 +
*Paul Thomson, Imperial College London, UK
 +
*Peter Libic, Charles University in Prague, Czech Republic
 +
*Dave Wilkinson, University of Pittsburgh, USA
 +
*Weiwei Chen, Qualcomm, USA
 +
*Riyadh Baghdadi, ENS and University Pierre et Marie Curie, Paris, France
 +
*Na Meng, University of Texas Austin, USA
 +
*Arun Raman, Qualcomm, USA
 +
*Bapi Chatterjee, Chalmers University, Sweden
 +
*Martin Maas, University of California Berkeley, USA
 +
*Vojtech Horky, Charles University in Prague, Czech Republic
 +
*Vasileios Trigonakis, EPFL, Switzerland
 +
*Mahdi Eslamimehr, SAP Labs, USA
 +
*Yuhao Zhu, University of Texas Austin, USA
 +
*Melanie Kambadur, Columbia University, USA
 +
*Michael Laurenzano, UMich, USA
 +
 
 +
| style="width: 37%" valign="top" |  
 +
<span style="font-size:large">'''<span style="font-family: tahoma,geneva,sans-serif">Above threshold</span>'''</span>
 +
 
 +
List of papers with artifacts above threshold:
 +
 
 +
*'''''Locality-Centric Thread Scheduling for Bulk-synchronous Programming Models on CPU Architectures'''''<br/>Hee-Seok Kim, Izzat El Hajj, John Stratton, Steven Lumetta and Wen-Mei Hwu
 +
*<span>'''''MemorySanitizer: fast detector of uninitialized memory use in C++'''''<br/>Evgeniy Stepanov and Konstantin Serebryany</span>
 +
*'''''A Parallel Abstract Interpreter for JavaScript'''''<br/>Kyle Dewey, Vineeth Kashyap and Ben Hardekopf
 +
*'''''A Graph-Based Higher-Order Intermediate Representation'''''<br/>Roland Leißa, Marcel Köster and Sebastian Hack
 +
*'''''Optimizing the flash-RAM energy trade-off in deeply embedded systems'''''<br/>James Pallister, Kerstin Eder and Simon J. Hollis
 +
*'''''Scalable Conditional Induction Variable (CIV) Analysis'''''<br/>Cosmin E. Oancea and Lawrence Rauchwerger
 +
<p style="text-align: center"></p>
 +
<span style="font-size:large">'''<span style="font-family: tahoma,geneva,sans-serif">Discussions</span>'''</span>
 +
 
 +
We consider AE as a continuous learning curve. If you have questions or comments and suggestions on how to improve packaging and reviewing process, please get in touch or use our public [https://groups.google.com/forum/#%21forum/collective-mind Google] or [http://www.linkedin.com/groups/Reproducible-research-experimentation-in-computer-7433414 LinkedIn] groups. We will hold a joint public CGO/PPoPP AE discussion session on February 9th, 2015.
 +
 
 +
|-
 +
| colspan="3" style="width: 25%" valign="top" |
 +
<span style="font-size:large">'''<span style="font-family: tahoma,geneva,sans-serif">Prior Artifact Evaluation</span>'''</span>
 +
 
 +
Artifacts have been already evaluated at several conferences and workshops including recent [http://pldi14-aec.cs.brown.edu PLDI 2014], [http://2014.splashcon.org/track/splash2014-artifacts OOPSLA 2014] and [http://adapt-workshop.org/2014 ADAPT 2014]. Our eventual goal is to develop common evaluation methodology gradually and collaboratively. See [http://artifact-eval.org artifact-eval.org], [http://cknowledge.org/reproducibility cknowledge.org/reproducibility] and [https://groups.google.com/forum/#!forum/collective-mind mailing list] for more details.
  
 
|}
 
|}
 +
 +
----
 +
 +
<br/>'''Sponsors and supporters'''
 +
 +
[[File:NVLogo 2D resize.jpg|NVLogo 2D resize.jpg|link=http://nvidia.com]][[File:CTuning foundation logo1a.png|CTuning foundation logo1a.png|link=http://cTuning.org]]
 
</center>
 
</center>

Latest revision as of 19:50, 12 February 2015

Ae-stamp-cgo.png

Artifact Evaluation for CGO'15

[ Back to CGO'15 conference website ]

Prizes for 2 highest ranked artifacts

1st place sponsored by
NVLogo 2D resize.jpg

2nd place sponsored by
CTuning foundation logo1a.png

"Nvidia Quadro K6000" for
"The SprayList: A scalable relaxed priority queue"
Justin Kopinsky, Dan Alistarh, Jerry Li  and Nir Shavit

"Acer C720P" for
"A graph-based higher-order intermediate representation"
Roland Leißa, Marcel Köster and Sebastian Hack 

Important dates

  • CGO paper decision:
    10 Nov 2014
  • Artifact submission:
    18 Nov 2014 AoE
  • Technical clarification:
    10-12 Dec 2014
  • Decision announced:
    16 Dec 2014
  • Public discussion:
    9 Feb 2015 [ Slides ]

Packaging guidelines

We use the following guidelines for artifacts. Because we want to encourage as many of you as possible to submit, these are only guidelines, not hard rules. Of course, if the material you want us to evaluate is fairly standard, following these suggestions will greatly simplify our lives. After all, remember: You want us to be able to evaluate your artifact, don't you?

How to submit

Please read the guidelines on what to submit. Please upload your submission to EasyChair (common with PPoPP).

Review process

The review process is described in detail here.

Artifact Evaluation Committee (AEC)

Chairs:

Advisory board:

Committee members:

  • David Boehme, LLNL, USA
  • Santiago Bock, University of Pittsburgh, USA
  • Lingda Li, Rutgers University, USA
  • Lin Ma, Huawei America Research Lab, USA
  • Yiannis Nikolakopulos, Chalmers University, Sweden
  • Jeeva Paudel, University of Alberta, Canada
  • Paul Thomson, Imperial College London, UK
  • Peter Libic, Charles University in Prague, Czech Republic
  • Dave Wilkinson, University of Pittsburgh, USA
  • Weiwei Chen, Qualcomm, USA
  • Riyadh Baghdadi, ENS and University Pierre et Marie Curie, Paris, France
  • Na Meng, University of Texas Austin, USA
  • Arun Raman, Qualcomm, USA
  • Bapi Chatterjee, Chalmers University, Sweden
  • Martin Maas, University of California Berkeley, USA
  • Vojtech Horky, Charles University in Prague, Czech Republic
  • Vasileios Trigonakis, EPFL, Switzerland
  • Mahdi Eslamimehr, SAP Labs, USA
  • Yuhao Zhu, University of Texas Austin, USA
  • Melanie Kambadur, Columbia University, USA
  • Michael Laurenzano, UMich, USA

Above threshold

List of papers with artifacts above threshold:

  • Locality-Centric Thread Scheduling for Bulk-synchronous Programming Models on CPU Architectures
    Hee-Seok Kim, Izzat El Hajj, John Stratton, Steven Lumetta and Wen-Mei Hwu
  • MemorySanitizer: fast detector of uninitialized memory use in C++
    Evgeniy Stepanov and Konstantin Serebryany
  • A Parallel Abstract Interpreter for JavaScript
    Kyle Dewey, Vineeth Kashyap and Ben Hardekopf
  • A Graph-Based Higher-Order Intermediate Representation
    Roland Leißa, Marcel Köster and Sebastian Hack
  • Optimizing the flash-RAM energy trade-off in deeply embedded systems
    James Pallister, Kerstin Eder and Simon J. Hollis
  • Scalable Conditional Induction Variable (CIV) Analysis
    Cosmin E. Oancea and Lawrence Rauchwerger

Discussions

We consider AE as a continuous learning curve. If you have questions or comments and suggestions on how to improve packaging and reviewing process, please get in touch or use our public Google or LinkedIn groups. We will hold a joint public CGO/PPoPP AE discussion session on February 9th, 2015.

Prior Artifact Evaluation

Artifacts have been already evaluated at several conferences and workshops including recent PLDI 2014, OOPSLA 2014 and ADAPT 2014. Our eventual goal is to develop common evaluation methodology gradually and collaboratively. See artifact-eval.org, cknowledge.org/reproducibility and mailing list for more details.


Sponsors and supporters

NVLogo 2D resize.jpgCTuning foundation logo1a.png

Retrieved from "http://ctuning.org/cm/wiki/index.php?title=Reproducibility:AE:CGO2015&oldid=753"
Share

(C) 2011-2014 cTuning foundation