|
| 1 | +# To be filled by the author(s) at the time of submission |
| 2 | +# ------------------------------------------------------- |
| 3 | + |
| 4 | +# Title of the article: |
| 5 | +# - For a successful replication, it should be prefixed with "[Re]" |
| 6 | +# - For a failed replication, it should be prefixed with "[-Re]" |
| 7 | +# - For other article types, no instruction (but please, not too long) |
| 8 | +title: "Eliminating the Variability of Cross-Validation Results with LIBLINEAR |
| 9 | +due to Randomization and Parallelization" |
| 10 | + |
| 11 | + |
| 12 | +# List of authors with name, orcid number, email and affiliation |
| 13 | +# Affiliation "*" means contact author |
| 14 | +authors: |
| 15 | + - name: Vladimir Sukhoy |
| 16 | + orcid: 0000-0003-2208-9459 |
| 17 | + email: sukhoy@iastate.edu |
| 18 | + affiliations: 1,* |
| 19 | + |
| 20 | + - name: Alexander Stoytchev |
| 21 | + orcid: 0000-0002-9871-4663 |
| 22 | + email: alexs@iastate.edu |
| 23 | + affiliations: 1 # * is for contact author |
| 24 | + |
| 25 | +# List of affiliations with code (corresponding to author affiliations), name |
| 26 | +# and address. You can also use these affiliations to add text such as "Equal |
| 27 | +# contributions" as name (with no address). |
| 28 | +affiliations: |
| 29 | + - code: 1 |
| 30 | + name: Iowa State University, Department of Electrical and Computer Engineering |
| 31 | + address: Ames, IA 50011, USA |
| 32 | + |
| 33 | + |
| 34 | +# List of keywords (adding the programming language might be a good idea) |
| 35 | +keywords: rescience c, c, c++, randomized algorithms, support vector machines, cross-validation, parallelized learning algorithms |
| 36 | + |
| 37 | +# Code URL and DOI (url is mandatory for replication, doi after acceptance) |
| 38 | +# You can get a DOI for your code from Zenodo, |
| 39 | +# see https://guides.github.com/activities/citable-code/ |
| 40 | +code: |
| 41 | + - url: https://github.com/sukhoy/cvrep |
| 42 | + - doi: |
| 43 | + |
| 44 | +# Date URL and DOI (optional if no data) |
| 45 | +data: |
| 46 | + - url: |
| 47 | + - doi: |
| 48 | + |
| 49 | +# Information about the original article that has been replicated |
| 50 | +replication: |
| 51 | + - cite: # Full textual citation |
| 52 | + - bib: # Bibtex key (if any) in your bibliography file |
| 53 | + - url: # URL to the PDF, try to link to a non-paywall version |
| 54 | + - doi: # Regular digital object identifier |
| 55 | + |
| 56 | +# Don't forget to surround abstract with double quotes |
| 57 | +abstract: "Cross-validation is the gold standard for evaluating machine learning |
| 58 | +algorithms or fine-tuning their parameters. |
| 59 | +The results of this technique, however, are not always reproducible and may |
| 60 | +depend on the computing platform and the number of parallel threads, |
| 61 | +especially if the underlying learning algorithm uses a pseudo-random number |
| 62 | +generator (PRNG). This paper gives a recipe for solving these reproducibility problems and |
| 63 | +applies it to LIBLINEAR, a popular software library |
| 64 | +that implements randomized learning algorithms based on support vector |
| 65 | +machines. The proposed approach solves these |
| 66 | +problems by using a cross-platform PRNG and by making the PRNG state private |
| 67 | +in each thread. The cross-validation results obtained with the modified |
| 68 | +version of LIBLINEAR were the same across platforms. Furthermore, the |
| 69 | +parallelized cross-validation results were no longer affected by random |
| 70 | +fluctuations arising from the sharing of the PRNG state by parallel threads." |
| 71 | + |
| 72 | +# Bibliography file (yours) |
| 73 | +bibliography: bibliography.bib |
| 74 | + |
| 75 | +# Type of the article |
| 76 | +# Type can be: |
| 77 | +# * Editorial |
| 78 | +# * Letter |
| 79 | +# * Replication |
| 80 | +type: Letter |
| 81 | + |
| 82 | +# Scientific domain of the article (e.g. Computational Neuroscience) |
| 83 | +# (one domain only & try to be not overly specific) |
| 84 | +domain: Machine Learning |
| 85 | + |
| 86 | +# Coding language (main one only if several) |
| 87 | +language: c++ |
| 88 | + |
| 89 | + |
| 90 | +# To be filled by the author(s) after acceptance |
| 91 | +# ----------------------------------------------------------------------------- |
| 92 | + |
| 93 | +# The URL of the GitHub issue where review actually occured |
| 94 | +review: |
| 95 | + - url: https://github.com/ReScience/submissions/issues/6 |
| 96 | + |
| 97 | +contributors: |
| 98 | + - name: Konrad Hinsen |
| 99 | + orcid: 0000-0003-0330-9428 |
| 100 | + role: editor |
| 101 | + - name: Georgios Detorakis |
| 102 | + orcid: 0000-0001-5891-1702 |
| 103 | + role: reviewer |
| 104 | + - name: Xavier Hinaut |
| 105 | + orcid: 0000-0002-1924-1184 |
| 106 | + role: reviewer |
| 107 | + |
| 108 | +# This information will be provided by the editor |
| 109 | +dates: |
| 110 | + - received: July 15, 2019 |
| 111 | + - accepted: October 30, 2019 |
| 112 | + - published: November 4, 2019 |
| 113 | + |
| 114 | +# This information will be provided by the editor |
| 115 | +article: |
| 116 | + - number: 1 |
| 117 | + - doi: 10.5281/zenodo.3528175 |
| 118 | + - url: https://zenodo.org/record/3528175/files/article.pdf |
| 119 | + |
| 120 | +# This information will be provided by the editor |
| 121 | +journal: |
| 122 | + - name: "ReScience C" |
| 123 | + - issn: 2430-3658 |
| 124 | + - volume: 5 |
| 125 | + - issue: 3 |
0 commit comments