Skip to content

typing: accept buffers in typing.IO.write #9084

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed
wants to merge 17 commits into from
Closed

typing: accept buffers in typing.IO.write #9084

Show file tree
Hide file tree
Changes from 7 commits
Commits
Show all changes
17 commits
Select commit Hold shift + click to select a range
2f87ea0
typing: accept buffers in typing.IO.write
JelleZijlstra Nov 4, 2022
e940cdf
type ignore
JelleZijlstra Nov 4, 2022
8d82f82
need this way
JelleZijlstra Nov 4, 2022
5aacf0a
fix arg name
JelleZijlstra Nov 4, 2022
9293f5e
Merge branch 'main' into typing
JelleZijlstra Nov 10, 2022
717e864
Merge branch 'main' into typing
AlexWaygood Feb 8, 2023
2894148
Update lzma.pyi
AlexWaygood Feb 8, 2023
6532cd4
Merge branch 'main' into typing
AlexWaygood Feb 9, 2023
642f843
Merge remote-tracking branch 'upstream/main' into typing
JelleZijlstra Mar 10, 2023
641511b
Remove references to closed mypy issue
JelleZijlstra Mar 10, 2023
590851b
Adjust overloads to work with pyright
JelleZijlstra Mar 10, 2023
b6174cb
[pre-commit.ci] auto fixes from pre-commit.com hooks
pre-commit-ci[bot] Mar 10, 2023
92c2606
ignore another
JelleZijlstra Mar 10, 2023
7f10fc8
no picklebuffer in 3.7
JelleZijlstra Mar 10, 2023
684b4ee
into the uncertain future
JelleZijlstra Mar 10, 2023
bd78b5b
maybe help mypy primer
JelleZijlstra Mar 10, 2023
37bb27e
try the other way
JelleZijlstra Mar 10, 2023
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
4 changes: 2 additions & 2 deletions stdlib/codecs.pyi
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -239,7 +239,7 @@ class StreamReaderWriter(TextIO):
def __next__(self) -> str: ...
def __iter__(self: Self) -> Self: ...
def write(self, data: str) -> None: ... # type: ignore[override]
def writelines(self, list: Iterable[str]) -> None: ...
def writelines(self, list: Iterable[str]) -> None: ... # type: ignore[override] # https://github.com/python/mypy/issues/14002
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We probably don't need this type: ignore anymore (as well as several others), with the latest mypy release.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Cleaned up all comments referencing that issue.

def reset(self) -> None: ...
def seek(self, offset: int, whence: int = 0) -> None: ... # type: ignore[override]
def __enter__(self: Self) -> Self: ...
Expand Down Expand Up @@ -273,7 +273,7 @@ class StreamRecoder(BinaryIO):
def __next__(self) -> bytes: ...
def __iter__(self: Self) -> Self: ...
def write(self, data: bytes) -> None: ... # type: ignore[override]
def writelines(self, list: Iterable[bytes]) -> None: ...
def writelines(self, list: Iterable[bytes]) -> None: ... # type: ignore[override] # https://github.com/python/mypy/issues/14002
def reset(self) -> None: ...
def __getattr__(self, name: str) -> Any: ...
def __enter__(self: Self) -> Self: ...
Expand Down
2 changes: 1 addition & 1 deletion stdlib/http/client.pyi
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -101,7 +101,7 @@ class HTTPMessage(email.message.Message):

def parse_headers(fp: io.BufferedIOBase, _class: Callable[[], email.message.Message] = ...) -> HTTPMessage: ...

class HTTPResponse(io.BufferedIOBase, BinaryIO):
class HTTPResponse(io.BufferedIOBase, BinaryIO): # type: ignore[misc] # https://github.com/python/mypy/issues/14002
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We do still need this type: ignore, but not because of the mypy issue linked (which is now closed) -- we need it because the definitions of writelines() in the two base classes are incompatible. The same goes for all the type: ignores in io.pyi and the one in lzma.pyi.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Updated these comments.

msg: HTTPMessage
headers: HTTPMessage
version: int
Expand Down
15 changes: 8 additions & 7 deletions stdlib/io.pyi
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -90,7 +90,7 @@ class BufferedIOBase(IOBase):
def read(self, __size: int | None = ...) -> bytes: ...
def read1(self, __size: int = ...) -> bytes: ...

class FileIO(RawIOBase, BinaryIO):
class FileIO(RawIOBase, BinaryIO): # type: ignore[misc] # https://github.com/python/mypy/issues/14002
mode: str
name: FileDescriptorOrPath # type: ignore[assignment]
def __init__(
Expand All @@ -102,7 +102,7 @@ class FileIO(RawIOBase, BinaryIO):
def read(self, __size: int = -1) -> bytes: ...
def __enter__(self: Self) -> Self: ...

class BytesIO(BufferedIOBase, BinaryIO):
class BytesIO(BufferedIOBase, BinaryIO): # type: ignore[misc] # https://github.com/python/mypy/issues/14002
def __init__(self, initial_bytes: ReadableBuffer = ...) -> None: ...
# BytesIO does not contain a "name" field. This workaround is necessary
# to allow BytesIO sub-classes to add this field, as it is defined
Expand All @@ -113,17 +113,18 @@ class BytesIO(BufferedIOBase, BinaryIO):
def getbuffer(self) -> memoryview: ...
def read1(self, __size: int | None = -1) -> bytes: ...

class BufferedReader(BufferedIOBase, BinaryIO):
class BufferedReader(BufferedIOBase, BinaryIO): # type: ignore[misc] # https://github.com/python/mypy/issues/14002
def __enter__(self: Self) -> Self: ...
def __init__(self, raw: RawIOBase, buffer_size: int = ...) -> None: ...
def peek(self, __size: int = 0) -> bytes: ...

class BufferedWriter(BufferedIOBase, BinaryIO):
class BufferedWriter(BufferedIOBase, BinaryIO): # type: ignore[misc] # https://github.com/python/mypy/issues/14002
def __enter__(self: Self) -> Self: ...
def __init__(self, raw: RawIOBase, buffer_size: int = ...) -> None: ...
def write(self, __buffer: ReadableBuffer) -> int: ...
# https://github.com/python/mypy/issues/14002
def write(self, __buffer: ReadableBuffer) -> int: ... # type: ignore[override]

class BufferedRandom(BufferedReader, BufferedWriter):
class BufferedRandom(BufferedReader, BufferedWriter): # type: ignore[misc] # https://github.com/python/mypy/issues/14002
def __enter__(self: Self) -> Self: ...
def seek(self, __target: int, __whence: int = 0) -> int: ... # stubtest needs this

Expand All @@ -144,7 +145,7 @@ class TextIOBase(IOBase):
def readlines(self, __hint: int = -1) -> list[str]: ... # type: ignore[override]
def read(self, __size: int | None = ...) -> str: ...

class TextIOWrapper(TextIOBase, TextIO):
class TextIOWrapper(TextIOBase, TextIO): # type: ignore[misc] # https://github.com/python/mypy/issues/14002
def __init__(
self,
buffer: IO[bytes],
Expand Down
2 changes: 1 addition & 1 deletion stdlib/lzma.pyi
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -104,7 +104,7 @@ class LZMACompressor:

class LZMAError(Exception): ...

class LZMAFile(io.BufferedIOBase, IO[bytes]):
class LZMAFile(io.BufferedIOBase, IO[bytes]): # type: ignore[misc] # incomptatible definitions of writelines in the 2 bases
def __init__(
self,
filename: _PathOrFile | None = None,
Expand Down
22 changes: 17 additions & 5 deletions stdlib/tempfile.pyi
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
import io
import sys
from _typeshed import BytesPath, GenericPath, Self, StrPath, WriteableBuffer
from _typeshed import BytesPath, GenericPath, ReadableBuffer, Self, StrPath, WriteableBuffer
from collections.abc import Iterable, Iterator
from types import TracebackType
from typing import IO, Any, AnyStr, Generic, overload
Expand Down Expand Up @@ -215,8 +215,14 @@ class _TemporaryFileWrapper(Generic[AnyStr], IO[AnyStr]):
def tell(self) -> int: ...
def truncate(self, size: int | None = ...) -> int: ...
def writable(self) -> bool: ...
def write(self, s: AnyStr) -> int: ...
def writelines(self, lines: Iterable[AnyStr]) -> None: ...
@overload
def write(self: _TemporaryFileWrapper[str], s: str) -> int: ...
@overload
def write(self: _TemporaryFileWrapper[bytes], s: ReadableBuffer) -> int: ...
@overload
def writelines(self: _TemporaryFileWrapper[str], lines: Iterable[str]) -> None: ...
@overload
def writelines(self: _TemporaryFileWrapper[bytes], lines: Iterable[ReadableBuffer]) -> None: ...
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

These overloads might not work very well with pyright, which has different handling of self types to mypy (#9591). The same goes for the overloads on lines 401-408, and the overloads in typing.pyi. You might want to instead do something similar to what I did with re.Pattern and re.Match in #9592.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Changed, and added some testcases for IO.write.


if sys.version_info >= (3, 11):
_SpooledTemporaryFileBase = io.IOBase
Expand Down Expand Up @@ -392,8 +398,14 @@ class SpooledTemporaryFile(IO[AnyStr], _SpooledTemporaryFileBase):
def seek(self, offset: int, whence: int = ...) -> int: ...
def tell(self) -> int: ...
def truncate(self, size: int | None = None) -> None: ... # type: ignore[override]
def write(self, s: AnyStr) -> int: ...
def writelines(self, iterable: Iterable[AnyStr]) -> None: ... # type: ignore[override]
@overload
def write(self: SpooledTemporaryFile[str], s: str) -> int: ...
@overload
def write(self: SpooledTemporaryFile[bytes], s: ReadableBuffer) -> int: ...
@overload
def writelines(self: SpooledTemporaryFile[str], iterable: Iterable[str]) -> None: ...
@overload
def writelines(self: SpooledTemporaryFile[bytes], iterable: Iterable[ReadableBuffer]) -> None: ...
def __iter__(self) -> Iterator[AnyStr]: ... # type: ignore[override]
# These exist at runtime only on 3.11+.
def readable(self) -> bool: ...
Expand Down
14 changes: 11 additions & 3 deletions stdlib/typing.pyi
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -3,7 +3,7 @@ import collections # Needed by aliases like DefaultDict, see mypy issue 2986
import sys
import typing_extensions
from _collections_abc import dict_items, dict_keys, dict_values
from _typeshed import IdentityFunction, Incomplete, SupportsKeysAndGetItem
from _typeshed import IdentityFunction, Incomplete, ReadableBuffer, SupportsKeysAndGetItem
from abc import ABCMeta, abstractmethod
from contextlib import AbstractAsyncContextManager, AbstractContextManager
from re import Match as Match, Pattern as Pattern
Expand Down Expand Up @@ -683,9 +683,17 @@ class IO(Iterator[AnyStr], Generic[AnyStr]):
@abstractmethod
def writable(self) -> bool: ...
@abstractmethod
def write(self, __s: AnyStr) -> int: ...
@overload
def write(self: IO[str], __s: str) -> int: ...
@abstractmethod
@overload
def write(self: IO[bytes], __s: ReadableBuffer) -> int: ...
@abstractmethod
def writelines(self, __lines: Iterable[AnyStr]) -> None: ...
@overload
def writelines(self: IO[str], __lines: Iterable[str]) -> None: ...
@abstractmethod
@overload
def writelines(self: IO[bytes], __lines: Iterable[ReadableBuffer]) -> None: ...
@abstractmethod
def __next__(self) -> AnyStr: ...
@abstractmethod
Expand Down
5 changes: 3 additions & 2 deletions stdlib/urllib/response.pyi
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -33,8 +33,9 @@ class addbase(BinaryIO):
def tell(self) -> int: ...
def truncate(self, size: int | None = ...) -> int: ...
def writable(self) -> bool: ...
def write(self, s: ReadableBuffer) -> int: ...
def writelines(self, lines: Iterable[ReadableBuffer]) -> None: ...
# https://github.com/python/mypy/issues/14002
def write(self, s: ReadableBuffer) -> int: ... # type: ignore[override]
def writelines(self, lines: Iterable[bytes]) -> None: ... # type: ignore[override]

class addclosehook(addbase):
closehook: Callable[..., object]
Expand Down