Merge remote-tracking branch 'upstream/master'

Author: rickwierenga

README.md

@@ -46,8 +46,7 @@ to receive updates on the project.
 import tensorflow_datasets as tfds
 import tensorflow as tf
 
-# tfds works in both Eager and Graph modes
-tf.compat.v1.enable_eager_execution()
+# Here we assume Eager mode is enabled (TF2), but tfds also works in Graph mode.
 
 # See available datasets
 print(tfds.list_builders())
@@ -92,32 +91,36 @@ ds = mnist_builder.as_dataset(split='train')
 # dataset and its features
 info = mnist_builder.info
 print(info)
+```
+
+This will print the dataset info content:
 
-    tfds.core.DatasetInfo(
-        name='mnist',
-        version=1.0.0,
-        description='The MNIST database of handwritten digits.',
-        homepage='http://yann.lecun.com/exdb/mnist/',
-        features=FeaturesDict({
-            'image': Image(shape=(28, 28, 1), dtype=tf.uint8),
-            'label': ClassLabel(shape=(), dtype=tf.int64, num_classes=10)
-        },
-        total_num_examples=70000,
-        splits={
-            'test': <tfds.core.SplitInfo num_examples=10000>,
-            'train': <tfds.core.SplitInfo num_examples=60000>
-        },
-        supervised_keys=('image', 'label'),
-        citation='"""
-            @article{lecun2010mnist,
-              title={MNIST handwritten digit database},
-              author={LeCun, Yann and Cortes, Corinna and Burges, CJ},
-              journal={ATT Labs [Online]. Available: http://yann. lecun. com/exdb/mnist},
-              volume={2},
-              year={2010}
-            }
-      """',
-  )
+```
+tfds.core.DatasetInfo(
+    name='mnist',
+    version=1.0.0,
+    description='The MNIST database of handwritten digits.',
+    homepage='http://yann.lecun.com/exdb/mnist/',
+    features=FeaturesDict({
+        'image': Image(shape=(28, 28, 1), dtype=tf.uint8),
+        'label': ClassLabel(shape=(), dtype=tf.int64, num_classes=10)
+    },
+    total_num_examples=70000,
+    splits={
+        'test': <tfds.core.SplitInfo num_examples=10000>,
+        'train': <tfds.core.SplitInfo num_examples=60000>
+    },
+    supervised_keys=('image', 'label'),
+    citation='"""
+        @article{lecun2010mnist,
+          title={MNIST handwritten digit database},
+          author={LeCun, Yann and Cortes, Corinna and Burges, CJ},
+          journal={ATT Labs [Online]. Available: http://yann. lecun. com/exdb/mnist},
+          volume={2},
+          year={2010}
+        }
+    """',
+)
 ```
 
 You can also get details about the classes (number of classes and their names).

docs/_index.ipynb

@@ -37,7 +37,7 @@
         "from __future__ import division\n",
         "from __future__ import print_function\n",
         "\n",
-        "import tensorflow as tf\n",
+        "import tensorflow.compat.v2 as tf\n",
         "import tensorflow_datasets as tfds\n",
         "\n",
         "# tfds works in both Eager and Graph modes\n",

docs/_index.yaml

@@ -23,7 +23,7 @@ landing_page:
         <a href="./datasets">list of datasets</a>.
     - code_block: |
         <pre class = "prettyprint">
-        import tensorflow as tf
+        import tensorflow.compat.v2 as tf
         import tensorflow_datasets as tfds
 
         # tfds works in both Eager and Graph modes
@@ -48,10 +48,10 @@ landing_page:
     items:
     - heading: Introducing TensorFlow Datasets
       image_path: /resources/images/tf-logo-card-16x9.png
-      path: https://github.com/tensorflow/datasets/blob/master/docs/announce_proxy.md
+      path: https://blog.tensorflow.org/2019/02/introducing-tensorflow-datasets.html
       buttons:
       - label: Read on TensorFlow Blog
-        path: https://github.com/tensorflow/datasets/blob/master/docs/announce_proxy.md
+        path: https://blog.tensorflow.org/2019/02/introducing-tensorflow-datasets.html
     - heading: TensorFlow Datasets on GitHub
       image_path: /resources/images/github-card-16x9.png
       path: https://github.com/tensorflow/datasets

docs/add_dataset.md

@@ -30,8 +30,7 @@ isn't already added.
     *   [3. Double-check the citation](#3-double-check-the-citation)
     *   [4. Add a test](#4-add-a-test)
     *   [5. Check your code style](#5-check-your-code-style)
-    *   [6. Add release notes](#6-add-release-notes)
-    *   [7. Send for review!](#7-send-for-review)
+    *   [6. Send for review!](#6-send-for-review)
 *   [Define the dataset outside TFDS](#define-the-dataset-outside-tfds)
 *   [Large datasets and distributed generation](#large-datasets-and-distributed-generation)
 *   [Testing `MyDataset`](#testing-mydataset)
@@ -543,7 +542,7 @@ except TensorFlow uses 2 spaces instead of 4. Please conform to the
 [Google Python Style Guide](https://github.com/google/styleguide/blob/gh-pages/pyguide.md),
 
 Most importantly, use
-[`tensorflow_datasets/oss_scripts/lint.sh`](https://github.com/tensorflow/datasets/tree/master/tensorflow_datasets/oss_scripts/lint.sh)
+[`tensorflow_datasets/oss_scripts/lint.sh`](https://github.com/tensorflow/datasets/tree/master/oss_scripts/lint.sh)
 to ensure your code is properly formatted. For example, to lint the `image`
 directory:
 
@@ -555,13 +554,7 @@ See
 [TensorFlow code style guide](https://www.tensorflow.org/community/contribute/code_style)
 for more information.
 
-### 6. Add release notes
-
-Add the dataset to the
-[release notes](https://github.com/tensorflow/datasets/tree/master/docs/release_notes.md).
-The release note will be published for the next release.
-
-### 7. Send for review!
+### 6. Send for review!
 
 Send the pull request for review.
 
@@ -586,7 +579,7 @@ To create this checksum file the first time, you can use the
 `tensorflow_datasets.scripts.download_and_prepare` script and pass the flags
 `--register_checksums --checksums_dir=/path/to/checksums_dir`.
 
-### 2. Adjust the fake example direcory
+### 2. Adjust the fake example directory
 
 For testing, instead of using the default
 [fake example directory](https://github.com/tensorflow/datasets/tree/master/tensorflow_datasets/testing/test_data/fake_examples)
@@ -595,7 +588,7 @@ you can define your own by setting the `EXAMPLE_DIR` property of
 
 ```
 class MyDatasetTest(tfds.testing.DatasetBuilderTestCase):
-  EXAMPLE_DIR = 'path/to/fakedata'`
+  EXAMPLE_DIR = 'path/to/fakedata'
 ```
 
 ## Large datasets and distributed generation
@@ -617,6 +610,9 @@ as downloaded and extracted. It can be created manually or automatically with a
 script
 ([example script](https://github.com/tensorflow/datasets/tree/master/tensorflow_datasets/testing/cifar.py)).
 
+If you're using automation to generate the test data, please include that script
+in [`testing`](https://github.com/tensorflow/datasets/tree/master/tensorflow_datasets/testing).
+
 Make sure to use different data in your test data splits, as the test will
 fail if your dataset splits overlap.
 

docs/api_docs/python/tfds/ReadConfig.md

@@ -20,6 +20,8 @@
 <a target="_blank" href="https://github.com/tensorflow/datasets/tree/master/tensorflow_datasets/core/utils/read_config.py">View
 source</a>
 
+<!-- Equality marker -->
+
 ## Class `ReadConfig`
 
 Configures input reading pipeline.

docs/api_docs/python/tfds/Split.md

@@ -18,6 +18,8 @@
 <a target="_blank" href="https://github.com/tensorflow/datasets/tree/master/tensorflow_datasets/core/splits.py">View
 source</a>
 
+<!-- Equality marker -->
+
 ## Class `Split`
 
 `Enum` for dataset splits.

docs/api_docs/python/tfds/as_numpy.md

@@ -13,6 +13,8 @@
 <a target="_blank" href="https://github.com/tensorflow/datasets/tree/master/tensorflow_datasets/core/dataset_utils.py">View
 source</a>
 
+<!-- Equality marker -->
+
 Converts a `tf.data.Dataset` to an iterable of NumPy arrays.
 
 ``` python

docs/api_docs/python/tfds/builder.md

@@ -13,6 +13,8 @@
 <a target="_blank" href="https://github.com/tensorflow/datasets/tree/master/tensorflow_datasets/core/registered.py">View
 source</a>
 
+<!-- Equality marker -->
+
 Fetches a
 <a href="../tfds/core/DatasetBuilder.md"><code>tfds.core.DatasetBuilder</code></a>
 by string name.

docs/api_docs/python/tfds/core/BeamBasedBuilder.md

@@ -29,6 +29,8 @@
 <a target="_blank" href="https://github.com/tensorflow/datasets/tree/master/tensorflow_datasets/core/dataset_builder.py">View
 source</a>
 
+<!-- Equality marker -->
+
 ## Class `BeamBasedBuilder`
 
 Beam based Builder.

docs/api_docs/python/tfds/core/BeamMetadataDict.md

@@ -16,6 +16,8 @@
 <a target="_blank" href="https://github.com/tensorflow/datasets/tree/master/tensorflow_datasets/core/dataset_info.py">View
 source</a>
 
+<!-- Equality marker -->
+
 ## Class `BeamMetadataDict`
 
 A <a href="../../tfds/core/Metadata.md"><code>tfds.core.Metadata</code></a>