Sponsors and supporters
           
If you would like to sponsor this community service including prizes for highest ranked artifacts and supporting open-source technology for collaborative, customizable and reproducible experimentation, please get in touch with the AE steering committee!
Important dates:
Paper decision: 10 Nov 2015
Artifact submission: 20 Nov 2015
Technical clarification: 14-20 Dec 2015
Decision announced: 22 Dec 2015
Public discussion: 14 March 2016

AE Chairs:
Grigori Fursin (dividiti/cTuning)
Bruce Childers (U. of Pittsburgh)

Artifact Evaluation for PPoPP 2016

[ Back to PPoPP 2016 conference website ]

Artifact evaluation is finished - see accepted artifacts (with awards) here !

News:
  • Highest ranked artifact: "A Wait-free Queue as Fast as Fetch-and-Add", Chaoran Yang and John Mellor-Crummey
    (award by dividiti)
  • 14 March 2016 (Monday, 18:00-18:30) - public AE dicussion (results, distinguished artifact award, issues, future work).
  • 14-16 March 2016 - informal Collective Knowledge demos to solve various encountered issues during Artifact Evaluation. CK enables collaborative and reproducible experimentation using shared and realistic workloads combined with customizable autotuning and predictive analytics plugins.

Authors of accepted PPoPP 2016 papers will be invited to formally submit their supporting materials to the Artifact Evaluation process. The Artifact Evaluation process is run by a separate committee whose task is to assess how the artifacts support the work described in the papers. This submission is voluntary and will not influence the final decision regarding the papers. Papers that successfully go through the Artifact Evaluation process will receive a seal of approval printed on the papers themselves. Authors of accepted papers are then encouraged to make these materials publicly available upon publication of the proceedings, by including them as "source materials" in the Digital Library.

How to submit

Please prepare your artifacts for submission using the following guide. Then, register your submission at the joint PPoPP/CGO EasyChair website - you will be asked to submit your paper title (do not forget to add PPoPP 2016 as prefix), author list, artifact abstract, pdf of your paper with an appendix describing how to access and validate your artifacts, and possible conflicts of interests with PPoPP PC members ( main, external), existing AE members or anyone else.

Reviewing process

Your artifacts will be reviewed based on the following guidelines. Artifacts receiving "met expectations" or above score will pass evaluation and will receive a stamp of approval. We are also arranging a prize for the highest ranked artifact.

Feedback

We consider Artifact Evaluation as a continuous learning curve - our eventual goal is to collaboratively develop common experimental and artifact sharing methodology for computer system's research. After running AE at PPoPP 2015 for the first time, we received an overall positive feedback from the community and, therefore, continue this experiment. However, we also experienced various issues which you may check out in our online presentation. Based on this experience and in collaboration with ACM, we are preparing revised submission and reviewing guidelines for PPoPP 2016. If you have questions, comments and suggestions on how to improve packaging, submission and reviewing process, please do not hesitate to get in touch!