Skip to content

Commit f7d1650

Browse files
Conchylicultorcopybara-github
authored andcommitted
Automated documentation update
PiperOrigin-RevId: 258476803
1 parent dd6cd9f commit f7d1650

20 files changed

+745
-129
lines changed

docs/api_docs/python/_toc.yaml

Lines changed: 10 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -56,6 +56,16 @@ toc:
5656
path: /datasets/api_docs/python/tfds/core/SplitInfo
5757
- title: Version
5858
path: /datasets/api_docs/python/tfds/core/Version
59+
- title: tfds.decode
60+
section:
61+
- title: Overview
62+
path: /datasets/api_docs/python/tfds/decode
63+
- title: Decoder
64+
path: /datasets/api_docs/python/tfds/decode/Decoder
65+
- title: make_decoder
66+
path: /datasets/api_docs/python/tfds/decode/make_decoder
67+
- title: SkipDecoding
68+
path: /datasets/api_docs/python/tfds/decode/SkipDecoding
5969
- title: tfds.download
6070
section:
6171
- title: Overview

docs/api_docs/python/index.md

Lines changed: 4 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -22,6 +22,10 @@
2222
* <a href="./tfds/core/Version.md"><code>tfds.core.Version</code></a>
2323
* <a href="./tfds/core/get_tfds_path.md"><code>tfds.core.get_tfds_path</code></a>
2424
* <a href="./tfds/core/lazy_imports.md"><code>tfds.core.lazy_imports</code></a>
25+
* <a href="./tfds/decode.md"><code>tfds.decode</code></a>
26+
* <a href="./tfds/decode/Decoder.md"><code>tfds.decode.Decoder</code></a>
27+
* <a href="./tfds/decode/SkipDecoding.md"><code>tfds.decode.SkipDecoding</code></a>
28+
* <a href="./tfds/decode/make_decoder.md"><code>tfds.decode.make_decoder</code></a>
2529
* <a href="./tfds/disable_progress_bar.md"><code>tfds.disable_progress_bar</code></a>
2630
* <a href="./tfds/download.md"><code>tfds.download</code></a>
2731
* <a href="./tfds/download/ComputeStatsMode.md"><code>tfds.download.ComputeStatsMode</code></a>

docs/api_docs/python/tfds.md

Lines changed: 2 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -34,6 +34,8 @@ The main library entrypoints are:
3434

3535
[`core`](./tfds/core.md) module: API to define datasets.
3636

37+
[`decode`](./tfds/decode.md) module: Decoder public API.
38+
3739
[`download`](./tfds/download.md) module: <a href="./tfds/download/DownloadManager.md"><code>tfds.download.DownloadManager</code></a> API.
3840

3941
[`features`](./tfds/features.md) module: <a href="./tfds/features/FeatureConnector.md"><code>tfds.features.FeatureConnector</code></a> API defining feature types.

docs/api_docs/python/tfds/_api_cache.json

Lines changed: 12 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -339,6 +339,18 @@
339339
"tfds.core.Version.tuple": true,
340340
"tfds.core.get_tfds_path": false,
341341
"tfds.core.lazy_imports": false,
342+
"tfds.decode": false,
343+
"tfds.decode.Decoder": false,
344+
"tfds.decode.Decoder.__init__": true,
345+
"tfds.decode.Decoder.decode_example": true,
346+
"tfds.decode.Decoder.dtype": true,
347+
"tfds.decode.Decoder.setup": true,
348+
"tfds.decode.SkipDecoding": false,
349+
"tfds.decode.SkipDecoding.__init__": true,
350+
"tfds.decode.SkipDecoding.decode_example": true,
351+
"tfds.decode.SkipDecoding.dtype": true,
352+
"tfds.decode.SkipDecoding.setup": true,
353+
"tfds.decode.make_decoder": false,
342354
"tfds.disable_progress_bar": false,
343355
"tfds.download": false,
344356
"tfds.download.ComputeStatsMode": false,

docs/api_docs/python/tfds/core/BeamBasedBuilder.md

Lines changed: 4 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -85,6 +85,7 @@ as_dataset(
8585
split=None,
8686
batch_size=None,
8787
shuffle_files=None,
88+
decoders=None,
8889
as_supervised=False,
8990
in_memory=None
9091
)
@@ -107,6 +108,9 @@ Callers must pass arguments as keyword arguments.
107108
of the whole dataset with `tf.Tensor`s instead of a `tf.data.Dataset`.
108109
* <b>`shuffle_files`</b>: `bool`, whether to shuffle the input files. Defaults
109110
to `True` if `split == tfds.Split.TRAIN` and `False` otherwise.
111+
* <b>`decoders`</b>: Nested dict of `Decoder` objects which allow to customize
112+
the decoding. The structure should match the feature structure, but only
113+
customized feature keys need to be present.
110114
* <b>`as_supervised`</b>: `bool`, if `True`, the returned `tf.data.Dataset`
111115
will have a 2-tuple structure `(input, label)` according to
112116
`builder.info.supervised_keys`. If `False`, the default, the returned

docs/api_docs/python/tfds/core/DatasetBuilder.md

Lines changed: 4 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -116,6 +116,7 @@ as_dataset(
116116
split=None,
117117
batch_size=None,
118118
shuffle_files=None,
119+
decoders=None,
119120
as_supervised=False,
120121
in_memory=None
121122
)
@@ -138,6 +139,9 @@ Callers must pass arguments as keyword arguments.
138139
of the whole dataset with `tf.Tensor`s instead of a `tf.data.Dataset`.
139140
* <b>`shuffle_files`</b>: `bool`, whether to shuffle the input files. Defaults
140141
to `True` if `split == tfds.Split.TRAIN` and `False` otherwise.
142+
* <b>`decoders`</b>: Nested dict of `Decoder` objects which allow to customize
143+
the decoding. The structure should match the feature structure, but only
144+
customized feature keys need to be present.
141145
* <b>`as_supervised`</b>: `bool`, if `True`, the returned `tf.data.Dataset`
142146
will have a 2-tuple structure `(input, label)` according to
143147
`builder.info.supervised_keys`. If `False`, the default, the returned

docs/api_docs/python/tfds/core/GeneratorBasedBuilder.md

Lines changed: 4 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -94,6 +94,7 @@ as_dataset(
9494
split=None,
9595
batch_size=None,
9696
shuffle_files=None,
97+
decoders=None,
9798
as_supervised=False,
9899
in_memory=None
99100
)
@@ -116,6 +117,9 @@ Callers must pass arguments as keyword arguments.
116117
of the whole dataset with `tf.Tensor`s instead of a `tf.data.Dataset`.
117118
* <b>`shuffle_files`</b>: `bool`, whether to shuffle the input files. Defaults
118119
to `True` if `split == tfds.Split.TRAIN` and `False` otherwise.
120+
* <b>`decoders`</b>: Nested dict of `Decoder` objects which allow to customize
121+
the decoding. The structure should match the feature structure, but only
122+
customized feature keys need to be present.
119123
* <b>`as_supervised`</b>: `bool`, if `True`, the returned `tf.data.Dataset`
120124
will have a 2-tuple structure `(input, label)` according to
121125
`builder.info.supervised_keys`. If `False`, the default, the returned

docs/api_docs/python/tfds/decode.md

Lines changed: 25 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,25 @@
1+
<div itemscope itemtype="http://developers.google.com/ReferenceObject">
2+
<meta itemprop="name" content="tfds.decode" />
3+
<meta itemprop="path" content="Stable" />
4+
</div>
5+
6+
# Module: tfds.decode
7+
8+
Decoder public API.
9+
10+
<a target="_blank" href=https://github.com/tensorflow/datasets/tree/master/tensorflow_datasets/core/decode/__init__.py>View
11+
source</a>
12+
13+
<!-- Placeholder for "Used in" -->
14+
15+
## Classes
16+
17+
[`class Decoder`](../tfds/decode/Decoder.md): Base decoder object.
18+
19+
[`class SkipDecoding`](../tfds/decode/SkipDecoding.md): Transformation which
20+
skip the decoding entirelly.
21+
22+
## Functions
23+
24+
[`make_decoder(...)`](../tfds/decode/make_decoder.md): Decorator to create a
25+
decoder.
Lines changed: 97 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,97 @@
1+
<div itemscope itemtype="http://developers.google.com/ReferenceObject">
2+
<meta itemprop="name" content="tfds.decode.Decoder" />
3+
<meta itemprop="path" content="Stable" />
4+
<meta itemprop="property" content="dtype"/>
5+
<meta itemprop="property" content="__init__"/>
6+
<meta itemprop="property" content="decode_example"/>
7+
<meta itemprop="property" content="setup"/>
8+
</div>
9+
10+
# tfds.decode.Decoder
11+
12+
## Class `Decoder`
13+
14+
Base decoder object.
15+
16+
<a target="_blank" href=https://github.com/tensorflow/datasets/tree/master/tensorflow_datasets/core/decode/base.py>View
17+
source</a>
18+
19+
<!-- Placeholder for "Used in" -->
20+
21+
<a href="../../tfds/decode/Decoder.md"><code>tfds.decode.Decoder</code></a>
22+
allows for overriding the default decoding by implementing a subclass, or
23+
skipping it entirely with
24+
<a href="../../tfds/decode/SkipDecoding.md"><code>tfds.decode.SkipDecoding</code></a>.
25+
26+
Instead of subclassing, you can also create a `Decoder` from a function with the
27+
<a href="../../tfds/decode/make_decoder.md"><code>tfds.decode.make_decoder</code></a>
28+
decorator.
29+
30+
All decoders must derive from this base class. The implementation can access the
31+
`self.feature` property which will correspond to the `FeatureConnector` to which
32+
this decoder is applied.
33+
34+
To implement a decoder, the main method to override is `decode_example`, which
35+
takes the serialized feature as input and returns the decoded feature.
36+
37+
If `decode_example` changes the output dtype, you must also override the `dtype`
38+
property. This enables compatibility with
39+
<a href="../../tfds/features/Sequence.md"><code>tfds.features.Sequence</code></a>.
40+
41+
<h2 id="__init__"><code>__init__</code></h2>
42+
43+
<a target="_blank" href=https://github.com/tensorflow/datasets/tree/master/tensorflow_datasets/core/decode/base.py>View
44+
source</a>
45+
46+
```python
47+
__init__()
48+
```
49+
50+
## Properties
51+
52+
<h3 id="dtype"><code>dtype</code></h3>
53+
54+
Returns the `dtype` after decoding.
55+
56+
## Methods
57+
58+
<h3 id="decode_example"><code>decode_example</code></h3>
59+
60+
<a target="_blank" href=https://github.com/tensorflow/datasets/tree/master/tensorflow_datasets/core/decode/base.py>View
61+
source</a>
62+
63+
```python
64+
decode_example(serialized_example)
65+
```
66+
67+
Decode the example feature field (eg: image).
68+
69+
#### Args:
70+
71+
* <b>`serialized_example`</b>: `tf.Tensor` as decoded, the dtype/shape should
72+
be identical to `feature.get_serialized_info()`
73+
74+
#### Returns:
75+
76+
* <b>`example`</b>: Decoded example.
77+
78+
<h3 id="setup"><code>setup</code></h3>
79+
80+
<a target="_blank" href=https://github.com/tensorflow/datasets/tree/master/tensorflow_datasets/core/decode/base.py>View
81+
source</a>
82+
83+
```python
84+
setup(feature)
85+
```
86+
87+
Transformation contructor.
88+
89+
The initialization of decode object is deferred because the objects only know
90+
the builder/features on which it is used after it has been constructed, the
91+
initialization is done in this function.
92+
93+
#### Args:
94+
95+
* <b>`feature`</b>:
96+
<a href="../../tfds/features/FeatureConnector.md"><code>tfds.features.FeatureConnector</code></a>,
97+
the feature to which is applied this transformation.
Lines changed: 83 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,83 @@
1+
<div itemscope itemtype="http://developers.google.com/ReferenceObject">
2+
<meta itemprop="name" content="tfds.decode.SkipDecoding" />
3+
<meta itemprop="path" content="Stable" />
4+
<meta itemprop="property" content="dtype"/>
5+
<meta itemprop="property" content="__init__"/>
6+
<meta itemprop="property" content="decode_example"/>
7+
<meta itemprop="property" content="setup"/>
8+
</div>
9+
10+
# tfds.decode.SkipDecoding
11+
12+
## Class `SkipDecoding`
13+
14+
Transformation which skip the decoding entirelly.
15+
16+
Inherits From: [`Decoder`](../../tfds/decode/Decoder.md)
17+
18+
<a target="_blank" href=https://github.com/tensorflow/datasets/tree/master/tensorflow_datasets/core/decode/base.py>View
19+
source</a>
20+
21+
<!-- Placeholder for "Used in" -->
22+
23+
#### Example of usage:
24+
25+
```python
26+
ds = ds.load(
27+
'imagenet2012',
28+
split='train',
29+
decoders={
30+
'image': tfds.decode.SkipDecoding(),
31+
}
32+
)
33+
34+
for ex in ds.take(1):
35+
assert ex['image'].dtype == tf.string
36+
```
37+
38+
<h2 id="__init__"><code>__init__</code></h2>
39+
40+
<a target="_blank" href=https://github.com/tensorflow/datasets/tree/master/tensorflow_datasets/core/decode/base.py>View
41+
source</a>
42+
43+
```python
44+
__init__()
45+
```
46+
47+
## Properties
48+
49+
<h3 id="dtype"><code>dtype</code></h3>
50+
51+
## Methods
52+
53+
<h3 id="decode_example"><code>decode_example</code></h3>
54+
55+
<a target="_blank" href=https://github.com/tensorflow/datasets/tree/master/tensorflow_datasets/core/decode/base.py>View
56+
source</a>
57+
58+
```python
59+
decode_example(serialized_example)
60+
```
61+
62+
Forward the serialized feature field.
63+
64+
<h3 id="setup"><code>setup</code></h3>
65+
66+
<a target="_blank" href=https://github.com/tensorflow/datasets/tree/master/tensorflow_datasets/core/decode/base.py>View
67+
source</a>
68+
69+
```python
70+
setup(feature)
71+
```
72+
73+
Transformation contructor.
74+
75+
The initialization of decode object is deferred because the objects only know
76+
the builder/features on which it is used after it has been constructed, the
77+
initialization is done in this function.
78+
79+
#### Args:
80+
81+
* <b>`feature`</b>:
82+
<a href="../../tfds/features/FeatureConnector.md"><code>tfds.features.FeatureConnector</code></a>,
83+
the feature to which is applied this transformation.

0 commit comments

Comments
 (0)