Skip to content

Commit c8439a3

Browse files
Conchylicultorcopybara-github
authored andcommitted
Remove ragged_test_utils
PiperOrigin-RevId: 278907021
1 parent 6e3fdae commit c8439a3

File tree

11 files changed

+45
-343
lines changed

11 files changed

+45
-343
lines changed

docs/api_docs/python/tfds/_api_cache.json

Lines changed: 0 additions & 12 deletions
Original file line numberDiff line numberDiff line change
@@ -847,7 +847,6 @@
847847
"tfds.testing.DatasetBuilderTestCase.assertMultiLineEqual": true,
848848
"tfds.testing.DatasetBuilderTestCase.assertNDArrayNear": true,
849849
"tfds.testing.DatasetBuilderTestCase.assertNear": true,
850-
"tfds.testing.DatasetBuilderTestCase.assertNestedListAlmostEqual": true,
851850
"tfds.testing.DatasetBuilderTestCase.assertNoCommonElements": true,
852851
"tfds.testing.DatasetBuilderTestCase.assertNotAllClose": true,
853852
"tfds.testing.DatasetBuilderTestCase.assertNotAllEqual": true,
@@ -864,8 +863,6 @@
864863
"tfds.testing.DatasetBuilderTestCase.assertNotStartsWith": true,
865864
"tfds.testing.DatasetBuilderTestCase.assertProtoEquals": true,
866865
"tfds.testing.DatasetBuilderTestCase.assertProtoEqualsVersion": true,
867-
"tfds.testing.DatasetBuilderTestCase.assertRaggedAlmostEqual": true,
868-
"tfds.testing.DatasetBuilderTestCase.assertRaggedEqual": true,
869866
"tfds.testing.DatasetBuilderTestCase.assertRaises": true,
870867
"tfds.testing.DatasetBuilderTestCase.assertRaisesOpError": true,
871868
"tfds.testing.DatasetBuilderTestCase.assertRaisesRegex": true,
@@ -899,7 +896,6 @@
899896
"tfds.testing.DatasetBuilderTestCase.defaultTestResult": true,
900897
"tfds.testing.DatasetBuilderTestCase.doCleanups": true,
901898
"tfds.testing.DatasetBuilderTestCase.enter_context": true,
902-
"tfds.testing.DatasetBuilderTestCase.eval_to_list": true,
903899
"tfds.testing.DatasetBuilderTestCase.evaluate": true,
904900
"tfds.testing.DatasetBuilderTestCase.fail": true,
905901
"tfds.testing.DatasetBuilderTestCase.failIf": true,
@@ -1029,7 +1025,6 @@
10291025
"tfds.testing.FeatureExpectationsTestCase.assertMultiLineEqual": true,
10301026
"tfds.testing.FeatureExpectationsTestCase.assertNDArrayNear": true,
10311027
"tfds.testing.FeatureExpectationsTestCase.assertNear": true,
1032-
"tfds.testing.FeatureExpectationsTestCase.assertNestedListAlmostEqual": true,
10331028
"tfds.testing.FeatureExpectationsTestCase.assertNoCommonElements": true,
10341029
"tfds.testing.FeatureExpectationsTestCase.assertNotAllClose": true,
10351030
"tfds.testing.FeatureExpectationsTestCase.assertNotAllEqual": true,
@@ -1046,8 +1041,6 @@
10461041
"tfds.testing.FeatureExpectationsTestCase.assertNotStartsWith": true,
10471042
"tfds.testing.FeatureExpectationsTestCase.assertProtoEquals": true,
10481043
"tfds.testing.FeatureExpectationsTestCase.assertProtoEqualsVersion": true,
1049-
"tfds.testing.FeatureExpectationsTestCase.assertRaggedAlmostEqual": true,
1050-
"tfds.testing.FeatureExpectationsTestCase.assertRaggedEqual": true,
10511044
"tfds.testing.FeatureExpectationsTestCase.assertRaises": true,
10521045
"tfds.testing.FeatureExpectationsTestCase.assertRaisesOpError": true,
10531046
"tfds.testing.FeatureExpectationsTestCase.assertRaisesRegex": true,
@@ -1081,7 +1074,6 @@
10811074
"tfds.testing.FeatureExpectationsTestCase.defaultTestResult": true,
10821075
"tfds.testing.FeatureExpectationsTestCase.doCleanups": true,
10831076
"tfds.testing.FeatureExpectationsTestCase.enter_context": true,
1084-
"tfds.testing.FeatureExpectationsTestCase.eval_to_list": true,
10851077
"tfds.testing.FeatureExpectationsTestCase.evaluate": true,
10861078
"tfds.testing.FeatureExpectationsTestCase.fail": true,
10871079
"tfds.testing.FeatureExpectationsTestCase.failIf": true,
@@ -1174,7 +1166,6 @@
11741166
"tfds.testing.SubTestCase.assertMultiLineEqual": true,
11751167
"tfds.testing.SubTestCase.assertNDArrayNear": true,
11761168
"tfds.testing.SubTestCase.assertNear": true,
1177-
"tfds.testing.SubTestCase.assertNestedListAlmostEqual": true,
11781169
"tfds.testing.SubTestCase.assertNoCommonElements": true,
11791170
"tfds.testing.SubTestCase.assertNotAllClose": true,
11801171
"tfds.testing.SubTestCase.assertNotAllEqual": true,
@@ -1191,8 +1182,6 @@
11911182
"tfds.testing.SubTestCase.assertNotStartsWith": true,
11921183
"tfds.testing.SubTestCase.assertProtoEquals": true,
11931184
"tfds.testing.SubTestCase.assertProtoEqualsVersion": true,
1194-
"tfds.testing.SubTestCase.assertRaggedAlmostEqual": true,
1195-
"tfds.testing.SubTestCase.assertRaggedEqual": true,
11961185
"tfds.testing.SubTestCase.assertRaises": true,
11971186
"tfds.testing.SubTestCase.assertRaisesOpError": true,
11981187
"tfds.testing.SubTestCase.assertRaisesRegex": true,
@@ -1226,7 +1215,6 @@
12261215
"tfds.testing.SubTestCase.defaultTestResult": true,
12271216
"tfds.testing.SubTestCase.doCleanups": true,
12281217
"tfds.testing.SubTestCase.enter_context": true,
1229-
"tfds.testing.SubTestCase.eval_to_list": true,
12301218
"tfds.testing.SubTestCase.evaluate": true,
12311219
"tfds.testing.SubTestCase.fail": true,
12321220
"tfds.testing.SubTestCase.failIf": true,

docs/api_docs/python/tfds/as_numpy.md

Lines changed: 7 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -28,6 +28,13 @@ tfds.as_numpy(
2828
`as_numpy` converts a possibly nested structure of `tf.data.Dataset`s
2929
and `tf.Tensor`s to iterables of NumPy arrays and NumPy arrays, respectively.
3030

31+
Note that because TensorFlow has support for ragged tensors and NumPy has no
32+
equivalent representation,
33+
[`tf.RaggedTensor`s](https://www.tensorflow.org/api_docs/python/tf/RaggedTensor)
34+
are left as-is for the user to deal with them (e.g. using `to_list()`). In TF 1
35+
(i.e. graph mode), `tf.RaggedTensor`s are returned as
36+
`tf.ragged.RaggedTensorValue`s.
37+
3138
#### Args:
3239

3340
* <b>`dataset`</b>: a possibly nested structure of `tf.data.Dataset`s and/or

docs/api_docs/python/tfds/download/DownloadConfig.md

Lines changed: 6 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -29,7 +29,7 @@ Configuration for
2929
<a target="_blank" href="https://github.com/tensorflow/datasets/tree/master/tensorflow_datasets/core/download/download_manager.py">View
3030
source</a>
3131

32-
``` python
32+
```python
3333
__init__(
3434
extract_dir=None,
3535
manual_dir=None,
@@ -38,7 +38,8 @@ __init__(
3838
max_examples_per_split=None,
3939
register_checksums=False,
4040
beam_runner=None,
41-
beam_options=None
41+
beam_options=None,
42+
try_download_gcs=True
4243
)
4344
```
4445

@@ -65,3 +66,6 @@ Constructs a `DownloadConfig`.
6566
datasets based on Beam for the generation.
6667
* <b>`beam_options`</b>: `PipelineOptions` to pass to `beam.Pipeline`, only
6768
used for datasets based on Beam for the generation.
69+
* <b>`try_download_gcs`</b>: `bool`, defaults to True. If True, prepared
70+
dataset will be downloaded from GCS, when available. If False, dataset will
71+
be downloaded and prepared from scratch.

docs/api_docs/python/tfds/testing/DatasetBuilderTestCase.md

Lines changed: 9 additions & 60 deletions
Original file line numberDiff line numberDiff line change
@@ -57,7 +57,6 @@
5757
<meta itemprop="property" content="assertMultiLineEqual"/>
5858
<meta itemprop="property" content="assertNDArrayNear"/>
5959
<meta itemprop="property" content="assertNear"/>
60-
<meta itemprop="property" content="assertNestedListAlmostEqual"/>
6160
<meta itemprop="property" content="assertNoCommonElements"/>
6261
<meta itemprop="property" content="assertNotAllClose"/>
6362
<meta itemprop="property" content="assertNotAllEqual"/>
@@ -74,8 +73,6 @@
7473
<meta itemprop="property" content="assertNotStartsWith"/>
7574
<meta itemprop="property" content="assertProtoEquals"/>
7675
<meta itemprop="property" content="assertProtoEqualsVersion"/>
77-
<meta itemprop="property" content="assertRaggedAlmostEqual"/>
78-
<meta itemprop="property" content="assertRaggedEqual"/>
7976
<meta itemprop="property" content="assertRaises"/>
8077
<meta itemprop="property" content="assertRaisesOpError"/>
8178
<meta itemprop="property" content="assertRaisesRegex"/>
@@ -109,7 +106,6 @@
109106
<meta itemprop="property" content="defaultTestResult"/>
110107
<meta itemprop="property" content="doCleanups"/>
111108
<meta itemprop="property" content="enter_context"/>
112-
<meta itemprop="property" content="eval_to_list"/>
113109
<meta itemprop="property" content="evaluate"/>
114110
<meta itemprop="property" content="fail"/>
115111
<meta itemprop="property" content="failIf"/>
@@ -375,17 +371,20 @@ one of the arguments is of type float16.
375371

376372
<h3 id="assertAllEqual"><code>assertAllEqual</code></h3>
377373

378-
<a target="_blank" href="https://github.com/tensorflow/datasets/tree/master/tensorflow_datasets/testing/test_utils.py">View
379-
source</a>
380-
381374
```python
382375
assertAllEqual(
383-
d1,
384-
d2
376+
*args,
377+
**kwds
385378
)
386379
```
387380

388-
Same as assertAllEqual but with RaggedTensor support.
381+
Asserts that two numpy arrays or Tensors have the same values.
382+
383+
#### Args:
384+
385+
* <b>`a`</b>: the expected numpy ndarray or anything can be converted to one.
386+
* <b>`b`</b>: the actual numpy ndarray or anything can be converted to one.
387+
* <b>`msg`</b>: Optional message to report on failure.
389388

390389
<h3 id="assertAllEqualNested"><code>assertAllEqualNested</code></h3>
391390

@@ -1164,20 +1163,6 @@ if not.
11641163
* <b>`err`</b>: A float value.
11651164
* <b>`msg`</b>: An optional string message to append to the failure message.
11661165

1167-
<h3 id="assertNestedListAlmostEqual"><code>assertNestedListAlmostEqual</code></h3>
1168-
1169-
<a target="_blank" href="https://github.com/tensorflow/datasets/tree/master/tensorflow_datasets/testing/ragged_test_util.py">View
1170-
source</a>
1171-
1172-
```python
1173-
assertNestedListAlmostEqual(
1174-
a,
1175-
b,
1176-
places=7,
1177-
context='value'
1178-
)
1179-
```
1180-
11811166
<h3 id="assertNoCommonElements"><code>assertNoCommonElements</code></h3>
11821167

11831168
``` python
@@ -1428,33 +1413,6 @@ assertProtoEqualsVersion(
14281413
)
14291414
```
14301415

1431-
<h3 id="assertRaggedAlmostEqual"><code>assertRaggedAlmostEqual</code></h3>
1432-
1433-
<a target="_blank" href="https://github.com/tensorflow/datasets/tree/master/tensorflow_datasets/testing/ragged_test_util.py">View
1434-
source</a>
1435-
1436-
```python
1437-
assertRaggedAlmostEqual(
1438-
a,
1439-
b,
1440-
places=7
1441-
)
1442-
```
1443-
1444-
<h3 id="assertRaggedEqual"><code>assertRaggedEqual</code></h3>
1445-
1446-
<a target="_blank" href="https://github.com/tensorflow/datasets/tree/master/tensorflow_datasets/testing/ragged_test_util.py">View
1447-
source</a>
1448-
1449-
```python
1450-
assertRaggedEqual(
1451-
a,
1452-
b
1453-
)
1454-
```
1455-
1456-
Asserts that two potentially ragged tensors are equal.
1457-
14581416
<h3 id="assertRaises"><code>assertRaises</code></h3>
14591417

14601418
``` python
@@ -2224,15 +2182,6 @@ unittest runs tests.
22242182

22252183
* <b>`manager`</b>: The context manager to enter.
22262184

2227-
<h3 id="eval_to_list"><code>eval_to_list</code></h3>
2228-
2229-
<a target="_blank" href="https://github.com/tensorflow/datasets/tree/master/tensorflow_datasets/testing/ragged_test_util.py">View
2230-
source</a>
2231-
2232-
```python
2233-
eval_to_list(tensor)
2234-
```
2235-
22362185
<h3 id="evaluate"><code>evaluate</code></h3>
22372186

22382187
``` python

docs/api_docs/python/tfds/testing/FeatureExpectationsTestCase.md

Lines changed: 9 additions & 60 deletions
Original file line numberDiff line numberDiff line change
@@ -59,7 +59,6 @@
5959
<meta itemprop="property" content="assertMultiLineEqual"/>
6060
<meta itemprop="property" content="assertNDArrayNear"/>
6161
<meta itemprop="property" content="assertNear"/>
62-
<meta itemprop="property" content="assertNestedListAlmostEqual"/>
6362
<meta itemprop="property" content="assertNoCommonElements"/>
6463
<meta itemprop="property" content="assertNotAllClose"/>
6564
<meta itemprop="property" content="assertNotAllEqual"/>
@@ -76,8 +75,6 @@
7675
<meta itemprop="property" content="assertNotStartsWith"/>
7776
<meta itemprop="property" content="assertProtoEquals"/>
7877
<meta itemprop="property" content="assertProtoEqualsVersion"/>
79-
<meta itemprop="property" content="assertRaggedAlmostEqual"/>
80-
<meta itemprop="property" content="assertRaggedEqual"/>
8178
<meta itemprop="property" content="assertRaises"/>
8279
<meta itemprop="property" content="assertRaisesOpError"/>
8380
<meta itemprop="property" content="assertRaisesRegex"/>
@@ -111,7 +108,6 @@
111108
<meta itemprop="property" content="defaultTestResult"/>
112109
<meta itemprop="property" content="doCleanups"/>
113110
<meta itemprop="property" content="enter_context"/>
114-
<meta itemprop="property" content="eval_to_list"/>
115111
<meta itemprop="property" content="evaluate"/>
116112
<meta itemprop="property" content="fail"/>
117113
<meta itemprop="property" content="failIf"/>
@@ -322,17 +318,20 @@ one of the arguments is of type float16.
322318

323319
<h3 id="assertAllEqual"><code>assertAllEqual</code></h3>
324320

325-
<a target="_blank" href="https://github.com/tensorflow/datasets/tree/master/tensorflow_datasets/testing/test_utils.py">View
326-
source</a>
327-
328321
```python
329322
assertAllEqual(
330-
d1,
331-
d2
323+
*args,
324+
**kwds
332325
)
333326
```
334327

335-
Same as assertAllEqual but with RaggedTensor support.
328+
Asserts that two numpy arrays or Tensors have the same values.
329+
330+
#### Args:
331+
332+
* <b>`a`</b>: the expected numpy ndarray or anything can be converted to one.
333+
* <b>`b`</b>: the actual numpy ndarray or anything can be converted to one.
334+
* <b>`msg`</b>: Optional message to report on failure.
336335

337336
<h3 id="assertAllEqualNested"><code>assertAllEqualNested</code></h3>
338337

@@ -1142,20 +1141,6 @@ if not.
11421141
* <b>`err`</b>: A float value.
11431142
* <b>`msg`</b>: An optional string message to append to the failure message.
11441143

1145-
<h3 id="assertNestedListAlmostEqual"><code>assertNestedListAlmostEqual</code></h3>
1146-
1147-
<a target="_blank" href="https://github.com/tensorflow/datasets/tree/master/tensorflow_datasets/testing/ragged_test_util.py">View
1148-
source</a>
1149-
1150-
```python
1151-
assertNestedListAlmostEqual(
1152-
a,
1153-
b,
1154-
places=7,
1155-
context='value'
1156-
)
1157-
```
1158-
11591144
<h3 id="assertNoCommonElements"><code>assertNoCommonElements</code></h3>
11601145

11611146
``` python
@@ -1406,33 +1391,6 @@ assertProtoEqualsVersion(
14061391
)
14071392
```
14081393

1409-
<h3 id="assertRaggedAlmostEqual"><code>assertRaggedAlmostEqual</code></h3>
1410-
1411-
<a target="_blank" href="https://github.com/tensorflow/datasets/tree/master/tensorflow_datasets/testing/ragged_test_util.py">View
1412-
source</a>
1413-
1414-
```python
1415-
assertRaggedAlmostEqual(
1416-
a,
1417-
b,
1418-
places=7
1419-
)
1420-
```
1421-
1422-
<h3 id="assertRaggedEqual"><code>assertRaggedEqual</code></h3>
1423-
1424-
<a target="_blank" href="https://github.com/tensorflow/datasets/tree/master/tensorflow_datasets/testing/ragged_test_util.py">View
1425-
source</a>
1426-
1427-
```python
1428-
assertRaggedEqual(
1429-
a,
1430-
b
1431-
)
1432-
```
1433-
1434-
Asserts that two potentially ragged tensors are equal.
1435-
14361394
<h3 id="assertRaises"><code>assertRaises</code></h3>
14371395

14381396
``` python
@@ -2202,15 +2160,6 @@ unittest runs tests.
22022160

22032161
* <b>`manager`</b>: The context manager to enter.
22042162

2205-
<h3 id="eval_to_list"><code>eval_to_list</code></h3>
2206-
2207-
<a target="_blank" href="https://github.com/tensorflow/datasets/tree/master/tensorflow_datasets/testing/ragged_test_util.py">View
2208-
source</a>
2209-
2210-
```python
2211-
eval_to_list(tensor)
2212-
```
2213-
22142163
<h3 id="evaluate"><code>evaluate</code></h3>
22152164

22162165
``` python

0 commit comments

Comments
 (0)