Skip to content

Commit 2dae77d

Browse files
ConchylicultorRyan Sepassi
authored andcommitted
Update doc
PiperOrigin-RevId: 250975797
1 parent b0e7de1 commit 2dae77d

30 files changed

+151
-1539
lines changed

docs/api_docs/python/tfds.md

Lines changed: 10 additions & 8 deletions
Original file line numberDiff line numberDiff line change
@@ -39,30 +39,32 @@ The main library entrypoints are:
3939

4040
[`file_adapter`](./tfds/file_adapter.md) module: <a href="./tfds/file_adapter/FileFormatAdapter.md"><code>tfds.file_adapter.FileFormatAdapter</code></a>s for GeneratorBasedBuilder.
4141

42-
[`units`](./tfds/units.md) module: Defines convenience constants/functions for converting various units.
43-
4442
[`testing`](./tfds/testing.md) module: Testing utilities.
4543

44+
[`units`](./tfds/units.md) module: Defines convenience constants/functions for
45+
converting various units.
46+
4647
## Classes
4748

4849
[`class GenerateMode`](./tfds/download/GenerateMode.md): `Enum` for how to treat pre-existing downloads and data.
4950

50-
[`class percent`](./tfds/percent.md): Syntactic sugar for defining slice subsplits: `tfds.percent[75:-5]`.
51-
5251
[`class Split`](./tfds/Split.md): `Enum` for dataset splits.
5352

53+
[`class percent`](./tfds/percent.md): Syntactic sugar for defining slice subsplits: `tfds.percent[75:-5]`.
54+
5455
## Functions
5556

5657
[`as_numpy(...)`](./tfds/as_numpy.md): Converts a `tf.data.Dataset` to an iterable of NumPy arrays.
5758

5859
[`builder(...)`](./tfds/builder.md): Fetches a <a href="./tfds/core/DatasetBuilder.md"><code>tfds.core.DatasetBuilder</code></a> by string name.
5960

60-
[`list_builders(...)`](./tfds/list_builders.md): Returns the string names of all <a href="./tfds/core/DatasetBuilder.md"><code>tfds.core.DatasetBuilder</code></a>s.
61-
62-
[`load(...)`](./tfds/load.md): Loads the named dataset into a `tf.data.Dataset`.
63-
6461
[`disable_progress_bar(...)`](./tfds/disable_progress_bar.md): Disabled Tqdm
6562
progress bar.
6663

6764
[`is_dataset_on_gcs(...)`](./tfds/is_dataset_on_gcs.md): If the dataset is
6865
available on the GCS bucket gs://tfds-data/datasets.
66+
67+
[`list_builders(...)`](./tfds/list_builders.md): Returns the string names of all <a href="./tfds/core/DatasetBuilder.md"><code>tfds.core.DatasetBuilder</code></a>s.
68+
69+
[`load(...)`](./tfds/load.md): Loads the named dataset into a `tf.data.Dataset`.
70+

docs/api_docs/python/tfds/Split.md

Lines changed: 4 additions & 21 deletions
Original file line numberDiff line numberDiff line change
@@ -1,7 +1,6 @@
11
<div itemscope itemtype="http://developers.google.com/ReferenceObject">
22
<meta itemprop="name" content="tfds.Split" />
33
<meta itemprop="path" content="Stable" />
4-
<meta itemprop="property" content="__new__"/>
54
<meta itemprop="property" content="ALL"/>
65
<meta itemprop="property" content="TEST"/>
76
<meta itemprop="property" content="TRAIN"/>
@@ -36,25 +35,9 @@ See the
3635
[guide on splits](https://github.com/tensorflow/datasets/tree/master/docs/splits.md)
3736
for more information.
3837

39-
<h2 id="__new__"><code>__new__</code></h2>
40-
41-
``` python
42-
@staticmethod
43-
__new__(
44-
cls,
45-
name
46-
)
47-
```
48-
49-
Create a custom split with tfds.Split('custom_name').
50-
5138
## Class Members
5239

53-
<h3 id="ALL"><code>ALL</code></h3>
54-
55-
<h3 id="TEST"><code>TEST</code></h3>
56-
57-
<h3 id="TRAIN"><code>TRAIN</code></h3>
58-
59-
<h3 id="VALIDATION"><code>VALIDATION</code></h3>
60-
40+
* `ALL` <a id="ALL"></a>
41+
* `TEST` <a id="TEST"></a>
42+
* `TRAIN` <a id="TRAIN"></a>
43+
* `VALIDATION` <a id="VALIDATION"></a>

docs/api_docs/python/tfds/_api_cache.json

Lines changed: 9 additions & 1188 deletions
Large diffs are not rendered by default.

docs/api_docs/python/tfds/core.md

Lines changed: 6 additions & 5 deletions
Original file line numberDiff line numberDiff line change
@@ -14,19 +14,15 @@ Defined in [`core/__init__.py`](https://github.com/tensorflow/datasets/tree/mast
1414

1515
## Classes
1616

17-
[`class lazy_imports`](../tfds/core/lazy_imports.md): Lazy importer for heavy dependencies.
18-
1917
[`class BeamBasedBuilder`](../tfds/core/BeamBasedBuilder.md): Beam based Builder.
2018

2119
[`class BuilderConfig`](../tfds/core/BuilderConfig.md): Base class for `DatasetBuilder` data configuration.
2220

2321
[`class DatasetBuilder`](../tfds/core/DatasetBuilder.md): Abstract base class for all datasets.
2422

25-
[`class GeneratorBasedBuilder`](../tfds/core/GeneratorBasedBuilder.md): Base class for datasets with data generation based on dict generators.
26-
2723
[`class DatasetInfo`](../tfds/core/DatasetInfo.md): Information about a dataset.
2824

29-
[`class NamedSplit`](../tfds/core/NamedSplit.md): Descriptor corresponding to a named split (train, test, ...).
25+
[`class GeneratorBasedBuilder`](../tfds/core/GeneratorBasedBuilder.md): Base class for datasets with data generation based on dict generators.
3026

3127
[`class Metadata`](../tfds/core/Metadata.md): Abstract base class for
3228
DatasetInfo metadata container.
@@ -35,6 +31,8 @@ DatasetInfo metadata container.
3531
<a href="../tfds/core/Metadata.md"><code>tfds.core.Metadata</code></a> object
3632
that acts as a `dict`.
3733

34+
[`class NamedSplit`](../tfds/core/NamedSplit.md): Descriptor corresponding to a named split (train, test, ...).
35+
3836
[`class SplitBase`](../tfds/core/SplitBase.md): Abstract base class for Split compositionality.
3937

4038
[`class SplitDict`](../tfds/core/SplitDict.md): Split info object.
@@ -45,6 +43,9 @@ that acts as a `dict`.
4543

4644
[`class Version`](../tfds/core/Version.md): Dataset version MAJOR.MINOR.PATCH.
4745

46+
[`class lazy_imports`](../tfds/core/lazy_imports.md): Lazy importer for heavy
47+
dependencies.
48+
4849
## Functions
4950

5051
[`get_tfds_path(...)`](../tfds/core/get_tfds_path.md): Returns absolute path to file given path relative to tfds root.

docs/api_docs/python/tfds/core/BeamBasedBuilder.md

Lines changed: 10 additions & 20 deletions
Original file line numberDiff line numberDiff line change
@@ -10,6 +10,7 @@
1010
<meta itemprop="property" content="BUILDER_CONFIGS"/>
1111
<meta itemprop="property" content="GOOGLE_DISABLED"/>
1212
<meta itemprop="property" content="IN_DEVELOPMENT"/>
13+
<meta itemprop="property" content="SUPPORTED_VERSIONS"/>
1314
<meta itemprop="property" content="VERSION"/>
1415
<meta itemprop="property" content="builder_configs"/>
1516
<meta itemprop="property" content="name"/>
@@ -51,7 +52,7 @@ Callers must pass arguments as keyword arguments.
5152
subdirectories and versions.
5253
* <b>`version`</b>: `str`. Optional version at which to load the dataset. An
5354
error is raised if specified version cannot be satisfied. Eg: '1.2.3',
54-
'1.2.*'. Note that only the currently defined version can be loaded.
55+
'1.2.*'.
5556

5657
## Properties
5758

@@ -74,8 +75,7 @@ as_dataset(
7475
split=None,
7576
batch_size=1,
7677
shuffle_files=None,
77-
as_supervised=False,
78-
in_memory=None
78+
as_supervised=False
7979
)
8080
```
8181

@@ -100,11 +100,6 @@ Callers must pass arguments as keyword arguments.
100100
will have a 2-tuple structure `(input, label)` according to
101101
`builder.info.supervised_keys`. If `False`, the default, the returned
102102
`tf.data.Dataset` will have a dictionary with all the features.
103-
* <b>`in_memory`</b>: `bool`, if `True`, loads the dataset in memory which
104-
increases iteration speeds. Note that if `True` and the dataset has unknown
105-
dimensions, the features will be padded to the maximum size across the
106-
dataset. By default (when `None`), will load the dataset in memory if the
107-
size is <1GB and all feature dimensions are statically known.
108103

109104
#### Returns:
110105

@@ -141,15 +136,10 @@ Downloads and prepares dataset for reading.
141136

142137
## Class Members
143138

144-
<h3 id="BUILDER_CONFIGS"><code>BUILDER_CONFIGS</code></h3>
145-
146-
<h3 id="GOOGLE_DISABLED"><code>GOOGLE_DISABLED</code></h3>
147-
148-
<h3 id="IN_DEVELOPMENT"><code>IN_DEVELOPMENT</code></h3>
149-
150-
<h3 id="VERSION"><code>VERSION</code></h3>
151-
152-
<h3 id="builder_configs"><code>builder_configs</code></h3>
153-
154-
<h3 id="name"><code>name</code></h3>
155-
139+
* `BUILDER_CONFIGS` <a id="BUILDER_CONFIGS"></a>
140+
* `GOOGLE_DISABLED = False` <a id="GOOGLE_DISABLED"></a>
141+
* `IN_DEVELOPMENT = False` <a id="IN_DEVELOPMENT"></a>
142+
* `SUPPORTED_VERSIONS` <a id="SUPPORTED_VERSIONS"></a>
143+
* `VERSION = None` <a id="VERSION"></a>
144+
* `builder_configs` <a id="builder_configs"></a>
145+
* `name = 'beam_based_builder'` <a id="name"></a>

docs/api_docs/python/tfds/core/BuilderConfig.md

Lines changed: 5 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -3,6 +3,7 @@
33
<meta itemprop="path" content="Stable" />
44
<meta itemprop="property" content="description"/>
55
<meta itemprop="property" content="name"/>
6+
<meta itemprop="property" content="supported_versions"/>
67
<meta itemprop="property" content="version"/>
78
<meta itemprop="property" content="__init__"/>
89
</div>
@@ -22,10 +23,11 @@ DatasetBuilder subclasses with data configuration options should subclass
2223

2324
<h2 id="__init__"><code>__init__</code></h2>
2425

25-
``` python
26+
```python
2627
__init__(
2728
name,
2829
version=None,
30+
supported_versions=None,
2931
description=None
3032
)
3133
```
@@ -36,4 +38,6 @@ __init__(
3638

3739
<h3 id="name"><code>name</code></h3>
3840

41+
<h3 id="supported_versions"><code>supported_versions</code></h3>
42+
3943
<h3 id="version"><code>version</code></h3>

docs/api_docs/python/tfds/core/DatasetBuilder.md

Lines changed: 10 additions & 20 deletions
Original file line numberDiff line numberDiff line change
@@ -10,6 +10,7 @@
1010
<meta itemprop="property" content="BUILDER_CONFIGS"/>
1111
<meta itemprop="property" content="GOOGLE_DISABLED"/>
1212
<meta itemprop="property" content="IN_DEVELOPMENT"/>
13+
<meta itemprop="property" content="SUPPORTED_VERSIONS"/>
1314
<meta itemprop="property" content="VERSION"/>
1415
<meta itemprop="property" content="builder_configs"/>
1516
<meta itemprop="property" content="name"/>
@@ -82,7 +83,7 @@ Callers must pass arguments as keyword arguments.
8283
subdirectories and versions.
8384
* <b>`version`</b>: `str`. Optional version at which to load the dataset. An
8485
error is raised if specified version cannot be satisfied. Eg: '1.2.3',
85-
'1.2.*'. Note that only the currently defined version can be loaded.
86+
'1.2.*'.
8687

8788
## Properties
8889

@@ -105,8 +106,7 @@ as_dataset(
105106
split=None,
106107
batch_size=1,
107108
shuffle_files=None,
108-
as_supervised=False,
109-
in_memory=None
109+
as_supervised=False
110110
)
111111
```
112112

@@ -131,11 +131,6 @@ Callers must pass arguments as keyword arguments.
131131
will have a 2-tuple structure `(input, label)` according to
132132
`builder.info.supervised_keys`. If `False`, the default, the returned
133133
`tf.data.Dataset` will have a dictionary with all the features.
134-
* <b>`in_memory`</b>: `bool`, if `True`, loads the dataset in memory which
135-
increases iteration speeds. Note that if `True` and the dataset has unknown
136-
dimensions, the features will be padded to the maximum size across the
137-
dataset. By default (when `None`), will load the dataset in memory if the
138-
size is <1GB and all feature dimensions are statically known.
139134

140135
#### Returns:
141136

@@ -172,15 +167,10 @@ Downloads and prepares dataset for reading.
172167

173168
## Class Members
174169

175-
<h3 id="BUILDER_CONFIGS"><code>BUILDER_CONFIGS</code></h3>
176-
177-
<h3 id="GOOGLE_DISABLED"><code>GOOGLE_DISABLED</code></h3>
178-
179-
<h3 id="IN_DEVELOPMENT"><code>IN_DEVELOPMENT</code></h3>
180-
181-
<h3 id="VERSION"><code>VERSION</code></h3>
182-
183-
<h3 id="builder_configs"><code>builder_configs</code></h3>
184-
185-
<h3 id="name"><code>name</code></h3>
186-
170+
* `BUILDER_CONFIGS` <a id="BUILDER_CONFIGS"></a>
171+
* `GOOGLE_DISABLED = False` <a id="GOOGLE_DISABLED"></a>
172+
* `IN_DEVELOPMENT = False` <a id="IN_DEVELOPMENT"></a>
173+
* `SUPPORTED_VERSIONS` <a id="SUPPORTED_VERSIONS"></a>
174+
* `VERSION = None` <a id="VERSION"></a>
175+
* `builder_configs` <a id="builder_configs"></a>
176+
* `name = 'dataset_builder'` <a id="name"></a>

docs/api_docs/python/tfds/core/GeneratorBasedBuilder.md

Lines changed: 10 additions & 20 deletions
Original file line numberDiff line numberDiff line change
@@ -10,6 +10,7 @@
1010
<meta itemprop="property" content="BUILDER_CONFIGS"/>
1111
<meta itemprop="property" content="GOOGLE_DISABLED"/>
1212
<meta itemprop="property" content="IN_DEVELOPMENT"/>
13+
<meta itemprop="property" content="SUPPORTED_VERSIONS"/>
1314
<meta itemprop="property" content="VERSION"/>
1415
<meta itemprop="property" content="builder_configs"/>
1516
<meta itemprop="property" content="name"/>
@@ -60,7 +61,7 @@ Callers must pass arguments as keyword arguments.
6061
subdirectories and versions.
6162
* <b>`version`</b>: `str`. Optional version at which to load the dataset. An
6263
error is raised if specified version cannot be satisfied. Eg: '1.2.3',
63-
'1.2.*'. Note that only the currently defined version can be loaded.
64+
'1.2.*'.
6465

6566
## Properties
6667

@@ -83,8 +84,7 @@ as_dataset(
8384
split=None,
8485
batch_size=1,
8586
shuffle_files=None,
86-
as_supervised=False,
87-
in_memory=None
87+
as_supervised=False
8888
)
8989
```
9090

@@ -109,11 +109,6 @@ Callers must pass arguments as keyword arguments.
109109
will have a 2-tuple structure `(input, label)` according to
110110
`builder.info.supervised_keys`. If `False`, the default, the returned
111111
`tf.data.Dataset` will have a dictionary with all the features.
112-
* <b>`in_memory`</b>: `bool`, if `True`, loads the dataset in memory which
113-
increases iteration speeds. Note that if `True` and the dataset has unknown
114-
dimensions, the features will be padded to the maximum size across the
115-
dataset. By default (when `None`), will load the dataset in memory if the
116-
size is <1GB and all feature dimensions are statically known.
117112

118113
#### Returns:
119114

@@ -150,15 +145,10 @@ Downloads and prepares dataset for reading.
150145

151146
## Class Members
152147

153-
<h3 id="BUILDER_CONFIGS"><code>BUILDER_CONFIGS</code></h3>
154-
155-
<h3 id="GOOGLE_DISABLED"><code>GOOGLE_DISABLED</code></h3>
156-
157-
<h3 id="IN_DEVELOPMENT"><code>IN_DEVELOPMENT</code></h3>
158-
159-
<h3 id="VERSION"><code>VERSION</code></h3>
160-
161-
<h3 id="builder_configs"><code>builder_configs</code></h3>
162-
163-
<h3 id="name"><code>name</code></h3>
164-
148+
* `BUILDER_CONFIGS` <a id="BUILDER_CONFIGS"></a>
149+
* `GOOGLE_DISABLED = False` <a id="GOOGLE_DISABLED"></a>
150+
* `IN_DEVELOPMENT = False` <a id="IN_DEVELOPMENT"></a>
151+
* `SUPPORTED_VERSIONS` <a id="SUPPORTED_VERSIONS"></a>
152+
* `VERSION = None` <a id="VERSION"></a>
153+
* `builder_configs` <a id="builder_configs"></a>
154+
* `name = 'generator_based_builder'` <a id="name"></a>

docs/api_docs/python/tfds/core/SplitDict.md

Lines changed: 0 additions & 10 deletions
Original file line numberDiff line numberDiff line change
@@ -4,7 +4,6 @@
44
<meta itemprop="property" content="total_num_examples"/>
55
<meta itemprop="property" content="__getitem__"/>
66
<meta itemprop="property" content="__init__"/>
7-
<meta itemprop="property" content="__setitem__"/>
87
<meta itemprop="property" content="add"/>
98
<meta itemprop="property" content="copy"/>
109
<meta itemprop="property" content="from_proto"/>
@@ -43,15 +42,6 @@ Return the total number of examples.
4342
__getitem__(key)
4443
```
4544

46-
<h3 id="__setitem__"><code>__setitem__</code></h3>
47-
48-
``` python
49-
__setitem__(
50-
key,
51-
value
52-
)
53-
```
54-
5545
<h3 id="add"><code>add</code></h3>
5646

5747
``` python

docs/api_docs/python/tfds/core/SplitInfo.md

Lines changed: 0 additions & 7 deletions
Original file line numberDiff line numberDiff line change
@@ -3,7 +3,6 @@
33
<meta itemprop="path" content="Stable" />
44
<meta itemprop="property" content="num_examples"/>
55
<meta itemprop="property" content="__eq__"/>
6-
<meta itemprop="property" content="__getattr__"/>
76
<meta itemprop="property" content="__init__"/>
87
<meta itemprop="property" content="get_proto"/>
98
</div>
@@ -40,12 +39,6 @@ __init__(
4039
__eq__(other)
4140
```
4241

43-
<h3 id="__getattr__"><code>__getattr__</code></h3>
44-
45-
``` python
46-
__getattr__(attr_name)
47-
```
48-
4942
<h3 id="get_proto"><code>get_proto</code></h3>
5043

5144
``` python

0 commit comments

Comments
 (0)