Skip to content

Commit 01aeeaa

Browse files
Conchylicultorcopybara-github
authored andcommitted
Update doc
PiperOrigin-RevId: 249559914
1 parent 52ba3c5 commit 01aeeaa

33 files changed

+3141
-426
lines changed

docs/api_docs/python/_redirects.yaml

Lines changed: 2 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -3,6 +3,8 @@ redirects:
33
to: /datasets/api_docs/python/tfds/download/GenerateMode
44
- from: /datasets/api_docs/python/tfds/testing/FeatureExpectationsTestCase/failureException
55
to: /datasets/api_docs/python/tfds/testing/DatasetBuilderTestCase/failureException
6+
- from: /datasets/api_docs/python/tfds/testing/SubTestCase/failureException
7+
to: /datasets/api_docs/python/tfds/testing/DatasetBuilderTestCase/failureException
68
- from: /datasets/api_docs/python/tfds/testing/TestCase/failureException
79
to: /datasets/api_docs/python/tfds/testing/DatasetBuilderTestCase/failureException
810
- from: /datasets/api_docs/python/tfds/features/text

docs/api_docs/python/_toc.yaml

Lines changed: 2 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -148,6 +148,8 @@ toc:
148148
path: /datasets/api_docs/python/tfds/testing/rm_tmp_dir
149149
- title: run_in_graph_and_eager_modes
150150
path: /datasets/api_docs/python/tfds/testing/run_in_graph_and_eager_modes
151+
- title: SubTestCase
152+
path: /datasets/api_docs/python/tfds/testing/SubTestCase
151153
- title: TestCase
152154
path: /datasets/api_docs/python/tfds/testing/TestCase
153155
- title: test_main

docs/api_docs/python/index.md

Lines changed: 2 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -65,6 +65,8 @@
6565
* <a href="./tfds/testing/FeatureExpectationItem.md"><code>tfds.testing.FeatureExpectationItem</code></a>
6666
* <a href="./tfds/testing/FeatureExpectationsTestCase.md"><code>tfds.testing.FeatureExpectationsTestCase</code></a>
6767
* <a href="./tfds/testing/DatasetBuilderTestCase/failureException.md"><code>tfds.testing.FeatureExpectationsTestCase.failureException</code></a>
68+
* <a href="./tfds/testing/SubTestCase.md"><code>tfds.testing.SubTestCase</code></a>
69+
* <a href="./tfds/testing/DatasetBuilderTestCase/failureException.md"><code>tfds.testing.SubTestCase.failureException</code></a>
6870
* <a href="./tfds/testing/TestCase.md"><code>tfds.testing.TestCase</code></a>
6971
* <a href="./tfds/testing/DatasetBuilderTestCase/failureException.md"><code>tfds.testing.TestCase.failureException</code></a>
7072
* <a href="./tfds/testing/make_tmp_dir.md"><code>tfds.testing.make_tmp_dir</code></a>

docs/api_docs/python/tfds/_api_cache.json

Lines changed: 200 additions & 60 deletions
Large diffs are not rendered by default.

docs/api_docs/python/tfds/as_numpy.md

Lines changed: 0 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -30,6 +30,5 @@ and `tf.Tensor`s to iterables of NumPy arrays and NumPy arrays, respectively.
3030
* <b>`graph`</b>: `tf.Graph`, optional, explicitly set the graph to use.
3131

3232
#### Returns:
33-
3433
A structure matching `dataset` where `tf.data.Dataset`s are converted to
3534
generators of NumPy arrays and `tf.Tensor`s are converted to NumPy arrays.

docs/api_docs/python/tfds/core/BeamBasedBuilder.md

Lines changed: 8 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -72,12 +72,13 @@ Callers must pass arguments as keyword arguments.
7272

7373
<h3 id="as_dataset"><code>as_dataset</code></h3>
7474

75-
``` python
75+
```python
7676
as_dataset(
7777
split=None,
7878
batch_size=1,
7979
shuffle_files=None,
80-
as_supervised=False
80+
as_supervised=False,
81+
in_memory=None
8182
)
8283
```
8384

@@ -101,6 +102,11 @@ Callers must pass arguments as keyword arguments.
101102
will have a 2-tuple structure `(input, label)` according to
102103
`builder.info.supervised_keys`. If `False`, the default, the returned
103104
`tf.data.Dataset` will have a dictionary with all the features.
105+
* <b>`in_memory`</b>: `bool`, if `True`, loads the dataset in memory which
106+
increases iteration speeds. Note that if `True` and the dataset has unknown
107+
dimensions, the features will be padded to the maximum size across the
108+
dataset. By default (when `None`), will load the dataset in memory if the
109+
size is <1GB and all feature dimensions are statically known.
104110

105111
#### Returns:
106112
`tf.data.Dataset`, or if `split=None`, `dict<key: tfds.Split, value:

docs/api_docs/python/tfds/core/DatasetBuilder.md

Lines changed: 8 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -103,12 +103,13 @@ Callers must pass arguments as keyword arguments.
103103

104104
<h3 id="as_dataset"><code>as_dataset</code></h3>
105105

106-
``` python
106+
```python
107107
as_dataset(
108108
split=None,
109109
batch_size=1,
110110
shuffle_files=None,
111-
as_supervised=False
111+
as_supervised=False,
112+
in_memory=None
112113
)
113114
```
114115

@@ -132,6 +133,11 @@ Callers must pass arguments as keyword arguments.
132133
will have a 2-tuple structure `(input, label)` according to
133134
`builder.info.supervised_keys`. If `False`, the default, the returned
134135
`tf.data.Dataset` will have a dictionary with all the features.
136+
* <b>`in_memory`</b>: `bool`, if `True`, loads the dataset in memory which
137+
increases iteration speeds. Note that if `True` and the dataset has unknown
138+
dimensions, the features will be padded to the maximum size across the
139+
dataset. By default (when `None`), will load the dataset in memory if the
140+
size is <1GB and all feature dimensions are statically known.
135141

136142
#### Returns:
137143
`tf.data.Dataset`, or if `split=None`, `dict<key: tfds.Split, value:

docs/api_docs/python/tfds/core/GeneratorBasedBuilder.md

Lines changed: 8 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -81,12 +81,13 @@ Callers must pass arguments as keyword arguments.
8181

8282
<h3 id="as_dataset"><code>as_dataset</code></h3>
8383

84-
``` python
84+
```python
8585
as_dataset(
8686
split=None,
8787
batch_size=1,
8888
shuffle_files=None,
89-
as_supervised=False
89+
as_supervised=False,
90+
in_memory=None
9091
)
9192
```
9293

@@ -110,6 +111,11 @@ Callers must pass arguments as keyword arguments.
110111
will have a 2-tuple structure `(input, label)` according to
111112
`builder.info.supervised_keys`. If `False`, the default, the returned
112113
`tf.data.Dataset` will have a dictionary with all the features.
114+
* <b>`in_memory`</b>: `bool`, if `True`, loads the dataset in memory which
115+
increases iteration speeds. Note that if `True` and the dataset has unknown
116+
dimensions, the features will be padded to the maximum size across the
117+
dataset. By default (when `None`), will load the dataset in memory if the
118+
size is <1GB and all feature dimensions are statically known.
113119

114120
#### Returns:
115121
`tf.data.Dataset`, or if `split=None`, `dict<key: tfds.Split, value:

docs/api_docs/python/tfds/core/NamedSplit.md

Lines changed: 0 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -32,7 +32,6 @@ The resulting split will correspond to 25% of the train split merged with
3232
100% of the test split.
3333

3434
#### Warning:
35-
3635
A split cannot be added twice, so the following will fail:
3736

3837
```
@@ -44,7 +43,6 @@ split = tfds.Split.TEST + tfds.Split.ALL # Error
4443
```
4544

4645
#### Warning:
47-
4846
The slices can be applied only one time. So the following are valid:
4947

5048
```

docs/api_docs/python/tfds/features.md

Lines changed: 3 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -33,10 +33,11 @@ Defined in [`core/features/__init__.py`](https://github.com/tensorflow/datasets/
3333

3434
[`class Tensor`](../tfds/features/Tensor.md): `FeatureConnector` for generic data of arbitrary shape and type.
3535

36-
[`class TensorInfo`](../tfds/features/TensorInfo.md): TensorInfo(shape, dtype)
36+
[`class TensorInfo`](../tfds/features/TensorInfo.md): Structure containing info
37+
on the `tf.Tensor` shape/dtype.
3738

3839
[`class Sequence`](../tfds/features/Sequence.md): Composite `FeatureConnector`
39-
where each value is a list.
40+
for a `dict` where each value is a list.
4041

4142
[`class Image`](../tfds/features/Image.md): `FeatureConnector` for images.
4243

0 commit comments

Comments
 (0)