Skip to content

Commit 555898a

Browse files
zhongyujiangamitgilad3
authored andcommitted
Type mismatch Avro zstd decompression (apache#2128)
<!-- Thanks for opening a pull request! --> <!-- In the case this PR will resolve an issue, please replace ${GITHUB_ISSUE_ID} below with the actual Github issue id. --> <!-- Closes #${GITHUB_ISSUE_ID} --> # Rationale for this change The return type of the decompress method in `ZStandardCodec` should be `bytes`, but it currently returns a `bytearray`, which causes an exception when reading Avro files compressed with zstd. ```text def new_decoder(b: bytes) -> BinaryDecoder: try: from pyiceberg.avro.decoder_fast import CythonBinaryDecoder > return CythonBinaryDecoder(b) E TypeError: Argument 'input_contents' has incorrect type (expected bytes, got bytearray) ``` # Are these changes tested? Yes, `test_write_manifest` # Are there any user-facing changes? No. <!-- In the case of user-facing changes, please add the changelog label. -->
1 parent 64d6241 commit 555898a

File tree

2 files changed

+2
-2
lines changed

2 files changed

+2
-2
lines changed

pyiceberg/avro/codecs/zstandard_codec.py

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -39,7 +39,7 @@ def decompress(data: bytes) -> bytes:
3939
if not chunk:
4040
break
4141
uncompressed.extend(chunk)
42-
return uncompressed
42+
return bytes(uncompressed)
4343

4444
except ImportError:
4545

tests/utils/test_manifest.py

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -358,7 +358,7 @@ def test_write_empty_manifest() -> None:
358358

359359

360360
@pytest.mark.parametrize("format_version", [1, 2])
361-
@pytest.mark.parametrize("compression", ["null", "deflate"])
361+
@pytest.mark.parametrize("compression", ["null", "deflate", "zstd"])
362362
def test_write_manifest(
363363
generated_manifest_file_file_v1: str,
364364
generated_manifest_file_file_v2: str,

0 commit comments

Comments
 (0)