Artifact Evaluation for Systems Conferences and Journals
We work with the community and ACM to improve methodology and tools for reproducible experimentation, artifact submission / reviewing and open challenges!
About Artifacts Committee Submission Reviewing FAQ Prior AE

List of accepted artifacts


Accepted artifacts for PPoPP 2018

[
PPoPP'18 AE website ]

This year we successfully validated the new ACM Artifact Review and Badging policy which we co-authored last year. However, we also noticed that "reusability/customization" criteria in the new guidelines are quite vague and caused ambiguity in evaluation of several complex artifacts. We would like to discuss this issue further with the community and develop more precise guidelines for next artifact evaluation. In the mean time, since our philosophy of artifact evaluation is that it is a cooperative process between authors and reviewers to overcome technical issues together, we helped all authors improve artifacts and pass evaluation.

Paper Artifact available Artifact functional Artifact reusable Results replicated
Optimizing N-Dimensional, Winograd-Based Convolution for Manycore CPUs
Zhen Jia, Aleksandar Zlateski, Frédo Durand and Kai Li

Artifact
(10.6084/m9.figshare.5873868)
PAM: Parallel Augmented Maps
Yihan Sun, Daniel Ferizovic and Guy Blelloch

Artifact
(10.5281/zenodo.1168703)
An Effective Fusion and Tile Size Model for Optimizing Image Processing Pipelines
Abhinav Jangda and Uday Bondhugula

Artifact
(10.5281/zenodo.1168539)
Cache-Tries: Concurrent Lock-Free Hash Tries with Constant-Time Operations
Aleksandar Prokopec

Artifact
(10.5281/zenodo.1168402)
swSpTRSV: a Fast Sparse Triangular Solve with Sparse Level Tile Layout on Sunway Architectures
Xinliang Wang, Weifeng Liu, Wei Xue and Li Wu

Artifact
(10.5281/zenodo.1168762)
VerifiedFT: A Verified, High-Performance Dynamic Race Detector
James Wilcox, Cormac Flanagan and Stephen Freund

Artifact
(10.5281/zenodo.1171046)
Juggler: A Dependency-Aware Task Based Execution Framework for GPUs
Mehmet Belviranli, Seyong Lee, Jeff Vetter and Laxmi Bhuyan

Artifact
(10.5281/zenodo.1168558)
Bridging the Gap between Deep Learning and Sparse Matrix Format Selection
Yue Zhao, Jiajia Li, Chunhua Liao and Xipeng Shen

Artifact
(10.6084/m9.figshare.5873868)
Harnessing Epoch-based Reclamation for Efficient Range Queries
Maya Arbel and Trevor Brown

Artifact
(10.5281/zenodo.1168726)
Register Optimizations for Stencils on GPUs
Prashant Rawat, Aravind Sukumaran-Rajam, Atanas Rountev, Fabrice Rastello, Louis-Noel Pouchet and P. Sadayappan

Artifact
(10.5281/zenodo.1168498)
Making Pull-Based Graph Processing Performant
Samuel Grossman, Heiner Litz and Christos Kozyrakis

Artifact
(10.5281/zenodo.1169388)
Interval-Based Memory Reclamation
Haosen Wen, Joseph Izraelevitz, Wentao Cai, H. Alan Beadle and Michael L. Scott

Artifact
(10.5281/zenodo.1168572)
Efficient Parallel Race Detection for Two-Dimensional Dags
Yifan Xu, Kunal Agrawal and I-Ting Angelina Lee

Artifact
(10.5281/zenodo.1169390)
Featherlight On-the-fly False-sharing Detection
Milind Chabbi, Shasha Wen and Xu Liu

Artifacts:
1, 2, 3, 4, 5
(10.5281/zenodo.1168535)
(10.5281/zenodo.1168520)
(10.5281/zenodo.1168529)
(10.5281/zenodo.1168533)
(10.5281/zenodo.1168526)
Communication-Avoiding Minimum Cuts and Connected Components
Pavel Kalvoda, Lukas Gianinazzi, Alessandro De Palma, Maciej Besta and Torsten Hoefler

Artifact
(10.5281/zenodo.1169439)

Accepted artifacts for CGO 2018

[
CGO'18 AE website ]

This year we successfully validated the new ACM Artifact Review and Badging policy which we co-authored last year.

Paper Artifact available Artifact functional Artifact reusable Results replicated
Poker: Permutation-based SIMD Execution of Intensive Tree Search by Path Encoding
F. Zhang, J. Xue
Optimal DNN Primitive Selection with Partitioned Boolean Quadratic Programming
A. Anderson, D. Gregg
May-Happen-in-Parallel Analysis with Static Vector Clocks
Q. Zhou, L. Li, L. Wang, J. Xue, X. Feng
SIMD Intrinsics on Managed Language Runtimes
Alen Stojanov, Ivaylo Toskov, Tiark Rompf, and Markus Püschel
DeLICM: Scalar Memory Dependence Removal at Zero Memory Cost
M. Kruse, T. Grosser
CVR: Efficient SpMV Vectorization on X86 Processors
B. Xie, J. Zhan, X. Liu, Z. Jia, W. Gao, L. Zhang
Lightweight Detection of Cache Conflicts
P. Roy, S. Song, S. Krishnamoorthy, X. Liu
Synthesizing Instruction Selection
S. Buchwald, A. Fried, S. Hack
Qubit Allocation
M. Siraichi, V. Santos, S. Collange, F. Pereira
High Performance Stencil Code Generation with LIFT
B. Hagedorn, L. Stoltzfus, M. Steuwer, S. Gorlatch, C. Dubach
Conflict-free Vectorization of Associative Irregular Applications with Recent SIMD Architectural Advances
P. Jiang, G. Agrawal
nAdroid: Statically Detecting Ordering Violations in Android Applications
X. Fu, D. Lee, C. Jung
CUDAAdvisor: LLVM-based Runtime Profiling for Modern GPUs
D. Shen, S. Song, A. Li, X. Liu
Enabling Enclave Code Secrecy via Self-Modification
E. Bauman, H. Wang, M. Zhang, Z. Lin
A Compiler for Cyber-Physical Digital Microfluidic Biochips
Christopher Curtis, Daniel Grissom, Philip Brisk

We also encourage sharing of artifacts via ACM DL even if they were not submitted for evaluation due to lack of time, etc.
However, we now collaborate with ACM to develop common formats and API for such artifacts.

Accepted artifacts for IA3 at SuperComputing 2017


[
Event website ]

Accepted artifacts for CGO 2017


[
Event website ]

This year we successfully tried an "open reviewing model" with a few artifacts, when we asked the community to publicly evaluate several artifacts already available at GitHub, GitLab and other project hosting services. This allowed us to find external reviewers who had access to very rare HPC servers or proprietary benchmarks and tools. We also allowed authors of the accepted artifacts to add up to 2 pages of Artifact Appendix to their camera-ready paper and let readers better understand what was evaluated and how.

Accepted artifacts for PACT 2017


[
Event website ]

Accepted artifacts for PPoPP 2017


[
Event website ]

This year we successfully tried an "open reviewing model" with a few artifacts, when we asked the community to publicly evaluate several artifacts already available at GitHub, GitLab and other project hosting services. This allowed us to find external reviewers who had access to very rare HPC servers or proprietary benchmarks and tools. We also allowed authors of the accepted artifacts to add up to 2 pages of Artifact Appendix to their camera-ready paper and let readers better understand what was evaluated and how.

Accepted artifacts for ADAPT 2016

Note that all artifacts have been openly validated and discussed by the community via
Reddit and shared in the unified and reusable CK format!


[ Event website ]

Accepted artifacts for PACT 2016


[
Event website ]

Accepted artifacts for PPoPP 2016


[
Event website ]

Accepted artifacts for CGO 2016


[
Event website ]

Accepted artifacts for PPoPP 2015


[
Сonference website ]

Accepted artifacts for CGO 2015


[
Сonference website ]
Maintained by
cTuning foundation (non-profit R&D organization),
Grigori Fursin and volunteers!
          
Powered by Collective Knowledge
                     
  
  
  
           Locations of visitors to this page