Updated on 2024-10-26

Author: lxaw

papers/list.json

@@ -1,4 +1,13 @@
 [
+  {
+    "title": "Why Does Unsupervised Pre-Training Help Deep Learning?",
+    "author": "Dumitru Erhan et al",
+    "year": "2010",
+    "topic": "pretraining, unsupervised",
+    "venue": "JMLR",
+    "description": "This paper argues that standard training schemes place parameters in regions of the parameter space that generalize poorly, while greedy layer-wise unsupervised pre-training allows each layer to learn a nonlinear transformation of its input that captures the main variations in the input, which acts as a regularizer: minimizing variance and introducing bias towards good initializations for the parameters. They argue that defining particular initialization points implicitly imposes constraints on the parameters in that it specifies which minima (out of many possible minima) of the cost function are allowed. They further argue that small perturbations in the trajectory of the parameters have a larger effect early on, and hint that early examples have larger influence and may trap model parameters in particular regions of parameter space corresponding to the arbitrary ordering of training examples (similar to the \"critical period\" in developmental psychology).",
+    "link": "https://jmlr.org/papers/volume11/erhan10a/erhan10a.pdf"
+  },
   {
     "title": "Improving Language Understanding by Generative Pre-Training",
     "author": "Alec Radford et al",

papers_read.html

@@ -75,10 +75,10 @@ <h1>Here's where I keep a list of papers I have read.</h1>
         I typically use this to organize papers I found interesting. Please feel free to do whatever you want with it. Note that this is not every single paper I have ever read, just a collection of ones that I remember to put down.
     </p>
     <p id="paperCount">
-        So far, we have read 144 papers. Let's keep it up!
+        So far, we have read 145 papers. Let's keep it up!
     </p> 
     <small id="searchCount">
-        Your search returned 144 papers. Nice! 
+        Your search returned 145 papers. Nice! 
     </small>
 
     <div class="search-inputs">
@@ -105,6 +105,16 @@ <h1>Here's where I keep a list of papers I have read.</h1>
         </thead>
         <tbody>
 
+            <tr>
+                <td>Why Does Unsupervised Pre-Training Help Deep Learning?</td>
+                <td>Dumitru Erhan et al</td>
+                <td>2010</td>
+                <td>pretraining, unsupervised</td>
+                <td>JMLR</td>
+                <td>This paper argues that standard training schemes place parameters in regions of the parameter space that generalize poorly, while greedy layer-wise unsupervised pre-training allows each layer to learn a nonlinear transformation of its input that captures the main variations in the input, which acts as a regularizer: minimizing variance and introducing bias towards good initializations for the parameters. They argue that defining particular initialization points implicitly imposes constraints on the parameters in that it specifies which minima (out of many possible minima) of the cost function are allowed. They further argue that small perturbations in the trajectory of the parameters have a larger effect early on, and hint that early examples have larger influence and may trap model parameters in particular regions of parameter space corresponding to the arbitrary ordering of training examples (similar to the &quot;critical period&quot; in developmental psychology).</td>
+                <td><a href="https://jmlr.org/papers/volume11/erhan10a/erhan10a.pdf" target="_blank">Link</a></td>
+            </tr>
+        
             <tr>
                 <td>Improving Language Understanding by Generative Pre-Training</td>
                 <td>Alec Radford et al</td>