|
|
| Line 1: |
Line 1: |
| − | [ '''''[[Reproducibility|Back to main page]]''''' ]
| + | Moved to [https://github.com/ctuning/ck/wiki/Enabling-open-science-articles GitHub]. |
| − | | + | |
| − | === '''Assorted articles and presentations'''<br/> ===
| + | |
| − | | + | |
| − | *[http://www.pl-enthusiast.net/2015/09/01/pl-conference-papers-to-get-a-journal PL conference papers to get a journal] - discussing ACM proposal to formally recognize conference publications as equal in quality as journal publications - controversial
| + | |
| − | *Our experience report with encountered problems in computer systems' research and new proposal for ''community-driven reviewing and validation of publications'' [[http://arxiv.org/abs/1406.4020 arXiv] , [http://dl.acm.org/citation.cfm?id=2618142 ACM DL]]
| + | |
| − | *[https://hal.inria.fr/inria-00436029 Collective Tuning Initiative: automating and accelerating development and optimization of computing systems]
| + | |
| − | *[http://www.the-scientist.com/?articles.view/articleNo/33719/title/Science-s-Reproducibility-Problem Science's Reproducibility Problem], Bob Grant, The Scientist
| + | |
| − | *[http://www.cs.huji.ac.il/~feit/papers/Repeat15SIGOPS.pdf From Repeatability to Reproducibility and Corroboration], Dror G. Feitelson, SIGOPS Operating Syst. Rev. 49(1), pp. 3-11, Jan 2015
| + | |
| − | *[http://cacm.acm.org/blogs/blog-cacm/98560 Reviewing peer review], Jeannette M. Wing, ACM Blog
| + | |
| − | *[http://cacm.acm.org/blogs/blog-cacm/100284 How Should Peer Review Evolve?], Ed H. Chi, ACM Blog
| + | |
| − | *[http://crd.lbl.gov/~dhbailey/dhbpapers/twelve-ways.pdf Twelve Ways to Fool the Masses When Giving Performance Results on Parallel Computers], David H. Bailey, Supercomputing Review, June 11, 1991
| + | |
| − | *[Ten Ways to Fool the Masses When Giving Performance Results on GPUs], Scott Pakin, HPCWire, December 13, 2011
| + | |
| − | *[http://www.cam.ac.uk/research/news/new-gold-standard-established-for-open-and-reproducible-research Cambridge University news about open and reproducible research]
| + | |
| − | *[http://reproduciblescience.blogspot.fr/2015/06/interesting-replicable-badge-for.html Interesting Replicable "Badge" for journal articles], Daniel S. Katz
| + | |
| − | *[http://thomasleeper.com/2015/05/open-science-language What's in a Name? The Concepts and Language of Replication and Reproducibility]
| + | |
| − | *[http://web.eece.maine.edu/~vweaver/papers/wddd08/wddd08_workshop.pdf Are Cycle Accurate Simulations a Waste of Time?], Vincent M. Weaver and Sally A. McKee
| + | |
| − | *[http://ehp.niehs.nih.gov/122-a188 Research Wranglers: Initiatives to Improve Reproducibility of Study Findings] (Environmental Health Perspectives)
| + | |
| − | *[https://web.stanford.edu/~vcs/talks/VictoriaStoddenICML2010.pdf Reproducible Research in Computational Science: Problems and Solutions For Data and Code Sharing], Victoria Stodden, ICML workshop 2010
| + | |
| − | *[http://www.the-scientist.com/?articles.view/articleNo/32426/title/Predatory-Publishing Predatory publishing]
| + | |
| − | *[http://michaelnielsen.org/blog/three-myths-about-scientific-peer-review Three myths about scientific peer review]
| + | |
| − | *[http://spyros.zoupanos.net/papers/p39.open-repeatability.pdf The repeatability experiment of SIGMOD 2008], SIGMOD Record, vol. 37, no. 1 (March 2008), 39-45.
| + | |
| − | *[http://dl.acm.org/citation.cfm?id=1536616.1536619 "Begin with an Author's response to Reviews"] - proposal to submit past reviews along with articles
| + | |
| − | *[https://www.dropbox.com/s/67n49i9k3cb28mo/GOBLE-CW2014.ppt Presentation by Professor Carole Goble]
| + | |
| − | *Dennis McCafferty, [http://dl.acm.org/citation.cfm?id=1831415 "Should Code be Released?"]
| + | |
| − | *[http://www.elsevier.com/connect/can-data-be-peer-reviewed Elsevier:Can data be peer-reviewed?]
| + | |
| − | *[https://hal.inria.fr/hal-01054763 Collective mind: Towards practical and collaborative auto-tuning]
| + | |
| − | *[http://arxiv.org/abs/1506.06256 Collective Mind, Part II: Towards Performance- and Cost-Aware Software Engineering as a Natural Science]
| + | |
| − | *[http://www.executablepapers.com Elsevier:The Executable Paper Grand Challenge]
| + | |
| − | *[http://www.sciencemag.org/content/334/6060/1225.full Special section on Data replication & reproducibility], Science magazine, 2 December 2011
| + | |
| − | *Chris Drummond, [http://cogprints.org/7691/7/ICMLws09.pdf "Replicability is not Reproducibility: Nor is it Good Science"]
| + | |
| − | *[http://theconversation.com/science-is-in-a-reproducibility-crisis-how-do-we-resolve-it-16998 Science is in a reproducibility crisis - how do we resolve it?]
| + | |
| − | *[http://www.w3.org/2001/sw/hcls/notes/hcls-dataset W3C Dataset Descriptions: HCLS Community Profile]
| + | |
| − | *My blog article on [http://www.software.ac.uk/blog/2014-07-22-automatic-performance-tuning-and-reproducibility-side-effect "Automatic performance tuning and reproducibility as a side effect"] for the Software Sustainability Institute
| + | |
| − | *[http://www.economist.com/news/briefing/21588057-scientists-think-science-self-correcting-alarming-degree-it-not-trouble Trouble at the Lab], The Economist, 2013
| + | |
| − | *[https://www.force11.org/group/joint-declaration-data-citation-principles-final Joint Declaration of Data Citation Principles]
| + | |
| − | *[http://www.scientificamerican.com/article/puzzling-measurement-of-big-g-gravitational-constant-ignites-debate-slide-show/ Puzzling Measurement of "Big G" Gravitational Constant Ignites Debate]
| + | |
| − | *[http://www.bipm.org/utils/common/documents/jcgm/JCGM_200_2012.pdf International vocabulary of metrology – Basic and general concepts and associated terms (VIM)]
| + | |
| − | *[http://retractionwatch.com/2014/09/05/white-house-takes-notice-of-reproducibility-in-science-and-wants-your-opinion White House takes notice of reproducibility in science, and wants your opinion]
| + | |
| − | *Problems during performance benchmarking:
| + | |
| − | **[https://homes.cs.washington.edu/~bornholt/post/performance-evaluation.html [https://homes.cs.washington.edu/~bornholt/post/performance-evaluation.html]]<br/>''We also experienced many similar issues during our work on auto-tuning and machine learning:''
| + | |
| − | **[http://hal.inria.fr/hal-01054763 [http://hal.inria.fr/hal-01054763]]
| + | |
| − | **[http://arxiv.org/abs/1406.4020 [http://arxiv.org/abs/1406.4020]]
| + | |
| − | *[http://dl.acm.org/citation.cfm?id=2723872 ACM SIGOPS Operating Systems Review - Special Issue on Repeatability and Sharing of Experimental Artifacts]
| + | |
| − | *[http://ieeexplore.ieee.org/xpl/articleDetails.jsp?reload=true&arnumber=5768098 Vinton G. Cerf. "Bit Rot: Long-Term Preservation of Digital Information" [Point of View]]
| + | |
| − | *[http://www.nature.com/news/the-top-100-papers-1.16224 About citations: less related though interesting]
| + | |
| − | | + | |
| − | === Journals with artifact sharing and evaluation ===
| + | |
| − | | + | |
| − | *[http://www.ipol.im IPOL Journal: Image Processing On Line<br/>]
| + | |
| − | *[http://netlib.org/toms ACM TRANSACTIONS ON MATHEMATICAL SOFTWARE (TOMS)]
| + | |
| − | *[http://db-reproducibility.seas.harvard.edu ACM SIGMOD 2015]
| + | |
| − | | + | |
| − | === National requirements ===
| + | |
| − | * [https://www.epsrc.ac.uk/files/aboutus/standards/clarificationsofexpectationsresearchdatamanagement UK EPSRC expectations on research data management]
| + | |
| − | * University of Manchester, UK:
| + | |
| − | ** http://www.library.manchester.ac.uk/services-and-support/staff/research/services/research-data-management/policy
| + | |
| − | ** http://www.library.manchester.ac.uk/services-and-support/staff/research/services/research-data-management/data-management-planning
| + | |
| − | * Cornell University, USA: [http://data.research.cornell.edu/content/best-practices Best Practices]
| + | |
