Reproducibility:Articles - Revision history

Gfursin: Replaced content with "Moved to [https://github.com/ctuning/ck/wiki/Enabling-open-science-articles GitHub]."

2016-09-13T13:36:21Z

Replaced content with "Moved to [https://github.com/ctuning/ck/wiki/Enabling-open-science-articles GitHub]."

Gfursin: /* Assorted articles and presentations */

2015-10-01T11:23:33Z

‎Assorted articles and presentations

Gfursin: /* Assorted articles and presentations */

2015-10-01T11:01:07Z

‎Assorted articles and presentations

Gfursin at 15:31, 24 September 2015

2015-09-24T15:31:09Z

Gfursin at 15:09, 24 September 2015

2015-09-24T15:09:28Z

Gfursin at 15:06, 24 September 2015

2015-09-24T15:06:14Z

Gfursin at 15:02, 24 September 2015

2015-09-24T15:02:59Z

Gfursin at 14:54, 24 September 2015

2015-09-24T14:54:58Z

Gfursin at 14:53, 24 September 2015

2015-09-24T14:53:10Z

Gfursin at 14:31, 24 September 2015

2015-09-24T14:31:35Z

@@ Line 1: / Line 1: @@
-[ '''''[[Reproducibility|Back to main page]]''''' ]
+Moved to [https://github.com/ctuning/ck/wiki/Enabling-open-science-articles  GitHub].
-−
-=== '''Assorted articles and presentations'''<br/> ===
-−
-*[http://www.pl-enthusiast.net/2015/09/01/pl-conference-papers-to-get-a-journal PL conference papers to get a journal] - discussing ACM proposal to formally recognize conference publications as equal in quality as journal publications - controversial
-*Our experience report with encountered problems in computer systems' research and new proposal for ''community-driven reviewing and validation of publications'' [[http://arxiv.org/abs/1406.4020 arXiv] , [http://dl.acm.org/citation.cfm?id=2618142 ACM DL]]
-*[https://hal.inria.fr/inria-00436029 Collective Tuning Initiative: automating and accelerating development and optimization of computing systems]
-*[http://www.the-scientist.com/?articles.view/articleNo/33719/title/Science-s-Reproducibility-Problem Science's Reproducibility Problem], Bob Grant, The Scientist
-*[http://www.cs.huji.ac.il/~feit/papers/Repeat15SIGOPS.pdf From Repeatability to Reproducibility and Corroboration], Dror G. Feitelson, SIGOPS Operating Syst. Rev. 49(1), pp. 3-11, Jan 2015
-*[http://cacm.acm.org/blogs/blog-cacm/98560 Reviewing peer review], Jeannette M. Wing, ACM Blog
-*[http://cacm.acm.org/blogs/blog-cacm/100284 How Should Peer Review Evolve?], Ed H. Chi, ACM Blog
-*[http://crd.lbl.gov/~dhbailey/dhbpapers/twelve-ways.pdf Twelve Ways to Fool the Masses When Giving Performance Results on Parallel Computers],  David H. Bailey, Supercomputing Review, June 11, 1991
-*[Ten Ways to Fool the Masses When Giving Performance Results on GPUs], Scott Pakin, HPCWire, December 13, 2011
-*[http://www.cam.ac.uk/research/news/new-gold-standard-established-for-open-and-reproducible-research Cambridge University news about open and reproducible research]
-*[http://reproduciblescience.blogspot.fr/2015/06/interesting-replicable-badge-for.html Interesting Replicable "Badge" for journal articles], Daniel S. Katz
-*[http://thomasleeper.com/2015/05/open-science-language What's in a Name? The Concepts and Language of Replication and Reproducibility]
-*[http://web.eece.maine.edu/~vweaver/papers/wddd08/wddd08_workshop.pdf Are Cycle Accurate Simulations a Waste of Time?], Vincent M. Weaver and Sally A. McKee
-*[http://ehp.niehs.nih.gov/122-a188 Research Wranglers: Initiatives to Improve Reproducibility of Study Findings] (Environmental Health Perspectives)
-*[https://web.stanford.edu/~vcs/talks/VictoriaStoddenICML2010.pdf Reproducible Research in Computational Science: Problems and Solutions For Data and Code Sharing], Victoria Stodden, ICML workshop 2010
-*[http://www.the-scientist.com/?articles.view/articleNo/32426/title/Predatory-Publishing Predatory publishing]
-*[http://michaelnielsen.org/blog/three-myths-about-scientific-peer-review Three myths about scientific peer review]
-*[http://spyros.zoupanos.net/papers/p39.open-repeatability.pdf The repeatability experiment of SIGMOD 2008], SIGMOD Record, vol. 37, no. 1 (March 2008), 39-45.
-*[http://dl.acm.org/citation.cfm?id=1536616.1536619 "Begin with an Author's response to Reviews"] - proposal to submit past reviews along with articles
-*[https://www.dropbox.com/s/67n49i9k3cb28mo/GOBLE-CW2014.ppt Presentation by Professor Carole Goble]
-*Dennis McCafferty, [http://dl.acm.org/citation.cfm?id=1831415 "Should Code be Released?"]
-*[http://www.elsevier.com/connect/can-data-be-peer-reviewed Elsevier:Can data be peer-reviewed?]
-*[https://hal.inria.fr/hal-01054763 Collective mind: Towards practical and collaborative auto-tuning]
-*[http://arxiv.org/abs/1506.06256 Collective Mind, Part II: Towards Performance- and Cost-Aware Software Engineering as a Natural Science]
-*[http://www.executablepapers.com Elsevier:The Executable Paper Grand Challenge]
-*[http://www.sciencemag.org/content/334/6060/1225.full Special section on Data replication & reproducibility], Science magazine, 2 December 2011
-*Chris Drummond, [http://cogprints.org/7691/7/ICMLws09.pdf "Replicability is not Reproducibility: Nor is it Good Science"]
-*[http://theconversation.com/science-is-in-a-reproducibility-crisis-how-do-we-resolve-it-16998 Science is in a reproducibility crisis - how do we resolve it?]
-*[http://www.w3.org/2001/sw/hcls/notes/hcls-dataset W3C Dataset Descriptions: HCLS Community Profile]
-*My blog article on [http://www.software.ac.uk/blog/2014-07-22-automatic-performance-tuning-and-reproducibility-side-effect "Automatic performance tuning and reproducibility as a side effect"]&nbsp; for the Software Sustainability Institute
-*[http://www.economist.com/news/briefing/21588057-scientists-think-science-self-correcting-alarming-degree-it-not-trouble Trouble at the Lab], The Economist, 2013
-*[https://www.force11.org/group/joint-declaration-data-citation-principles-final Joint Declaration of Data Citation Principles]
-*[http://www.scientificamerican.com/article/puzzling-measurement-of-big-g-gravitational-constant-ignites-debate-slide-show/ Puzzling Measurement of "Big G" Gravitational Constant Ignites Debate]
-*[http://www.bipm.org/utils/common/documents/jcgm/JCGM_200_2012.pdf International vocabulary of metrology – Basic and general concepts and associated terms (VIM)]
-*[http://retractionwatch.com/2014/09/05/white-house-takes-notice-of-reproducibility-in-science-and-wants-your-opinion White House takes notice of reproducibility in science, and wants your opinion]
-*Problems during performance benchmarking:
-**[https://homes.cs.washington.edu/~bornholt/post/performance-evaluation.html [https://homes.cs.washington.edu/~bornholt/post/performance-evaluation.html]]<br/>''We also experienced many similar issues during our work on auto-tuning and machine learning:''
-**[http://hal.inria.fr/hal-01054763 [http://hal.inria.fr/hal-01054763]]
-**[http://arxiv.org/abs/1406.4020 [http://arxiv.org/abs/1406.4020]]
-*[http://dl.acm.org/citation.cfm?id=2723872 ACM SIGOPS Operating Systems Review - Special Issue on Repeatability and Sharing of Experimental Artifacts]
-*[http://ieeexplore.ieee.org/xpl/articleDetails.jsp?reload=true&arnumber=5768098 Vinton G. Cerf. "Bit Rot: Long-Term Preservation of Digital Information" [Point of View]]
-*[http://www.nature.com/news/the-top-100-papers-1.16224 About citations: less related though interesting]
-−
-=== Journals with artifact sharing and evaluation ===
-−
-*[http://www.ipol.im IPOL Journal: Image Processing On Line<br/>]
-*[http://netlib.org/toms ACM TRANSACTIONS ON MATHEMATICAL SOFTWARE (TOMS)]
-*[http://db-reproducibility.seas.harvard.edu ACM SIGMOD 2015]
-−
-=== National requirements ===
-* [https://www.epsrc.ac.uk/files/aboutus/standards/clarificationsofexpectationsresearchdatamanagement UK EPSRC expectations on research data management]
-* University of Manchester, UK:
-** http://www.library.manchester.ac.uk/services-and-support/staff/research/services/research-data-management/policy
-** http://www.library.manchester.ac.uk/services-and-support/staff/research/services/research-data-management/data-management-planning
-* Cornell University, USA: [http://data.research.cornell.edu/content/best-practices Best Practices]

@@ Line 15: / Line 15: @@
 *[http://reproduciblescience.blogspot.fr/2015/06/interesting-replicable-badge-for.html Interesting Replicable "Badge" for journal articles], Daniel S. Katz
 *[http://thomasleeper.com/2015/05/open-science-language What's in a Name? The Concepts and Language of Replication and Reproducibility]
 *[http://ehp.niehs.nih.gov/122-a188 Research Wranglers: Initiatives to Improve Reproducibility of Study Findings] (Environmental Health Perspectives)
 *[https://web.stanford.edu/~vcs/talks/VictoriaStoddenICML2010.pdf Reproducible Research in Computational Science: Problems and Solutions For Data and Code Sharing], Victoria Stodden, ICML workshop 2010
@@ Line 24: / Line 25: @@
 *Dennis McCafferty, [http://dl.acm.org/citation.cfm?id=1831415 "Should Code be Released?"]
 *[http://www.elsevier.com/connect/can-data-be-peer-reviewed Elsevier:Can data be peer-reviewed?]
 *[http://www.executablepapers.com Elsevier:The Executable Paper Grand Challenge]
 *[http://www.sciencemag.org/content/334/6060/1225.full Special section on Data replication & reproducibility], Science magazine, 2 December 2011

@@ Line 5: / Line 5: @@
 *[http://www.pl-enthusiast.net/2015/09/01/pl-conference-papers-to-get-a-journal PL conference papers to get a journal] - discussing ACM proposal to formally recognize conference publications as equal in quality as journal publications - controversial
 *Our experience report with encountered problems in computer systems' research and new proposal for ''community-driven reviewing and validation of publications'' [[http://arxiv.org/abs/1406.4020 arXiv] , [http://dl.acm.org/citation.cfm?id=2618142 ACM DL]]
 *[http://www.the-scientist.com/?articles.view/articleNo/33719/title/Science-s-Reproducibility-Problem Science's Reproducibility Problem], Bob Grant, The Scientist
 *[http://www.cs.huji.ac.il/~feit/papers/Repeat15SIGOPS.pdf From Repeatability to Reproducibility and Corroboration], Dror G. Feitelson, SIGOPS Operating Syst. Rev. 49(1), pp. 3-11, Jan 2015

@@ Line 3: / Line 3: @@
 === '''Assorted articles and presentations'''<br/> ===
 *Our experience report with encountered problems in computer systems' research and new proposal for ''community-driven reviewing and validation of publications'' [[http://arxiv.org/abs/1406.4020 arXiv] , [http://dl.acm.org/citation.cfm?id=2618142 ACM DL]]
 *[http://www.the-scientist.com/?articles.view/articleNo/33719/title/Science-s-Reproducibility-Problem Science's Reproducibility Problem], Bob Grant, The Scientist

@@ Line 8: / Line 8: @@
 *[http://cacm.acm.org/blogs/blog-cacm/98560 Reviewing peer review], Jeannette M. Wing, ACM Blog
 *[http://cacm.acm.org/blogs/blog-cacm/100284 How Should Peer Review Evolve?], Ed H. Chi, ACM Blog
 *[http://www.cam.ac.uk/research/news/new-gold-standard-established-for-open-and-reproducible-research Cambridge University news about open and reproducible research]
 *[http://reproduciblescience.blogspot.fr/2015/06/interesting-replicable-badge-for.html Interesting Replicable "Badge" for journal articles], Daniel S. Katz

← Older revision		Revision as of 14:54, 24 September 2015
Line 48:		Line 48:
	** http://www.library.manchester.ac.uk/services-and-support/staff/research/services/research-data-management/policy		** http://www.library.manchester.ac.uk/services-and-support/staff/research/services/research-data-management/policy
	** http://www.library.manchester.ac.uk/services-and-support/staff/research/services/research-data-management/data-management-planning		** http://www.library.manchester.ac.uk/services-and-support/staff/research/services/research-data-management/data-management-planning
		+	* Cornell University, USA: [http://data.research.cornell.edu/content/best-practices Best Practices]