-
Notifications
You must be signed in to change notification settings - Fork 89
feat: retry and retry_async support streaming rpcs #495
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Merged
Merged
Changes from 1 commit
Commits
Show all changes
220 commits
Select commit
Hold shift + click to select a range
953106a
got retryable generators partially working
daniel-sanche 89aeb75
added retrun statement
daniel-sanche 27feb80
refactoring
daniel-sanche 0dffa6d
work for now deadline
daniel-sanche b330c3b
improved synchronous generator retry
daniel-sanche 67ceaa2
handle closing and returns
daniel-sanche ee2647a
got test to pass
daniel-sanche 5a5396c
restructured test
daniel-sanche 7afa76b
added tests
daniel-sanche 2d91ade
refactoring and comments in retry code
daniel-sanche 0cd384e
fixed helper; added is_generator flag
daniel-sanche f72bbec
got first test working
daniel-sanche 88eed5c
remove extra await in front of async generator
daniel-sanche 91f9cc4
implemented async generator retry test
daniel-sanche c3eb997
fixed is_generator
daniel-sanche f6c6201
added tests for aclose and athrow
daniel-sanche 57b0ee3
simplified close; don't support throws
daniel-sanche e814ce7
added tests
daniel-sanche 0ffb03f
have test that throw should retry
daniel-sanche a8024f3
improved aclose and athrow
daniel-sanche c76f641
added comments
daniel-sanche ee631e3
close synchronous generator
daniel-sanche 70eb78c
refactor async file
daniel-sanche 42ee132
ran blacken
daniel-sanche 102d83b
improved send test
daniel-sanche f029dbd
improved comments
daniel-sanche c83c62a
got send working
daniel-sanche 185826c
tested deadline handling
daniel-sanche c5f7bbe
changed timeout to only count time awaiting or sleeping
daniel-sanche 4242036
improved comments
daniel-sanche 9c4799c
added test for cancellation
daniel-sanche 0bd6cab
improved comments
daniel-sanche 67aeeaf
on_error can yield into the generator stream
daniel-sanche 985b13a
Merge branch 'main' into retry_generators
daniel-sanche 0ea8297
added filter_func to retryable generator
daniel-sanche b952652
fixed error in time budget calculation
daniel-sanche 6cb3e2d
added from field to raised timeout exception
daniel-sanche 99da116
removed filter_fn
daniel-sanche 7f862d0
ran blacken
daniel-sanche 04a4a69
removed generator auto-detection
daniel-sanche d20cf08
fixed tests and lint
daniel-sanche 183c221
changed comments
daniel-sanche d2217e4
fixed 3.11 failed test
daniel-sanche d4a9d30
added comments
daniel-sanche 06d45cc
made streaming retries into a custom generator object
daniel-sanche de41a14
added tests for iterators
daniel-sanche dcb3766
added test for non-awaitable target
daniel-sanche dd368e4
changed is_generator to is_stream
daniel-sanche 452b9bb
🦉 Updates from OwlBot post-processor
gcf-owl-bot[bot] 6879418
changed docstrings
daniel-sanche 847509f
removed back-ticks
daniel-sanche b5e3796
removed outdated comment
daniel-sanche 7a7d9ac
changed comments
daniel-sanche 6619895
moved streaming retries to new files
daniel-sanche 27fc930
reverted some style changes
daniel-sanche d6a23ea
changed comments
daniel-sanche 90ef834
added comments
daniel-sanche 6201db6
refactoring and commenting
daniel-sanche 61ce3a7
blacken/mypy fixes
daniel-sanche 69149a1
fixed issue with py37
daniel-sanche d63871e
added tests for bad sleep generators
daniel-sanche 773e033
improved test_retry coverage
daniel-sanche d1def5d
improved async test coverage
daniel-sanche cbaaa1d
added test for calling next on exhausted generator
daniel-sanche 21a863f
fixed lint issue
daniel-sanche 878ddfb
changed docstring
daniel-sanche 7b0a600
changed docstrings
daniel-sanche 0188228
updated comments
daniel-sanche 902a4ab
updated comments
daniel-sanche 74f3f3e
fixed send and asend retry logic
daniel-sanche e506aad
update test error string
daniel-sanche 5baa2aa
🦉 Updates from OwlBot post-processor
gcf-owl-bot[bot] 5c3805d
improved type hinting
daniel-sanche 265d998
improved test docs
daniel-sanche 0423ebe
fixed mypy issues
daniel-sanche c4049f5
Merge branch 'main' into retry_generators
daniel-sanche acd6546
remove wait_for in async streaming for perf reasons
daniel-sanche b1ad4b3
fixed style issues
daniel-sanche 8dcf67c
fixed callable type annotation
daniel-sanche 6104c59
change time calculations
daniel-sanche 43d0913
🦉 Updates from OwlBot post-processor
gcf-owl-bot[bot] 9ba7676
simplified retry_streaming_async to use wall time instead of cpu time
daniel-sanche 14c195c
Merge branch 'main' into retry_generators
daniel-sanche de7b51a
removed extra CancelledError handling
daniel-sanche 4cdee6b
improved docstrings
daniel-sanche a526d65
🦉 Updates from OwlBot post-processor
gcf-owl-bot[bot] ee2bbdd
Merge branch 'main' into retry_generators
daniel-sanche 5f82355
swapped out utcnow for more performant time.monotonic
daniel-sanche 9900c40
🦉 Updates from OwlBot post-processor
gcf-owl-bot[bot] 2c2dcbe
🦉 Updates from OwlBot post-processor
gcf-owl-bot[bot] 3340399
Merge branch 'retry_generators' of https://github.com/googleapis/pyth…
gcf-owl-bot[bot] de07714
Merge branch 'main' into retry_generators
parthea 67068ac
don't check timeout on each yield by default
daniel-sanche 54325bc
added exception building logic
daniel-sanche bafa18b
added type hint to check_timeout_on_yield
daniel-sanche 2ae2a32
simplified ensure_tareget; fixed mypy issues
daniel-sanche 9cadd63
don't check timeout on each yield by default
daniel-sanche c9ef1d5
added exception building logic
daniel-sanche 41c7868
added type hint to check_timeout_on_yield
daniel-sanche 30fccb9
simplified ensure_tareget; fixed mypy issues
daniel-sanche a2b0e6c
remove iteration helper
daniel-sanche 4aa1ab4
🦉 Updates from OwlBot post-processor
gcf-owl-bot[bot] 8349424
added test coverage for send/asend
daniel-sanche ece5cf8
Merge branch 'retry_generators_2' into retry_generators
daniel-sanche 5ddda24
added test for closing new generator
daniel-sanche 9e3ea92
improved test decorators
daniel-sanche 3b06b3a
swapped out generator object with generator function
daniel-sanche 8bb6b0c
support iterators, along with generators
daniel-sanche 37c64a0
got tests passing with new structure
daniel-sanche cee0028
replaces sync streaming retries object with generator function
daniel-sanche 3a7e5fa
removed timeout on yield functionality
daniel-sanche ba6dc9f
fixed comments
daniel-sanche 0500b8b
fixed mypy issues
daniel-sanche 1ccadb1
fixed issue with py310
daniel-sanche c312262
renamed streaming retry function
daniel-sanche 1fe57e0
removed unneeded functions
daniel-sanche 4f09f29
simplified some test functions
daniel-sanche 06824b9
🦉 Updates from OwlBot post-processor
gcf-owl-bot[bot] 343157b
removed unneeded test variable
daniel-sanche 93f82cc
improved documentation
daniel-sanche 0915ca0
Merge branch 'main' into retry_generators
parthea 61e5ab5
fixed type hinting issues
daniel-sanche 51c125b
fixed undefined name issues
daniel-sanche 02604bc
fixed lint issues
daniel-sanche 6269db2
update comment
daniel-sanche 0dcd0de
fix typo
daniel-sanche 54e9c81
Update google/api_core/retry_streaming.py
daniel-sanche 2342910
added comment to on_error
daniel-sanche eada0d7
fixed indentation
daniel-sanche ae2bf37
improved sample
daniel-sanche c8a4f26
improved default exception factory
daniel-sanche 2840b9f
added pylint disable line
daniel-sanche 82274a3
cleaned up async retry wrapping
daniel-sanche 1594a17
improved sample
daniel-sanche 9b0ddb0
🦉 Updates from OwlBot post-processor
gcf-owl-bot[bot] 8985127
remove extra generator close line
daniel-sanche 60b20ab
added missing test
daniel-sanche 237ca3d
AsyncRetry adds a coroutine in front of async streams
daniel-sanche a46c0f7
improved type checking
daniel-sanche 93727b7
Merge branch 'main' into retry_generators
daniel-sanche 796ae52
fixed typing issues
daniel-sanche 0688ffe
moved docstrings
daniel-sanche da048ab
use enum in exception builder
daniel-sanche 80e5eb0
🦉 Updates from OwlBot post-processor
gcf-owl-bot[bot] 562079b
fixed lint and docs issues
daniel-sanche a0fecc5
Merge branch 'main' into retry_generators
daniel-sanche 8cc6ea9
Update tests/unit/test_retry.py
daniel-sanche e7a5cd4
fixed comment line break
daniel-sanche 02c12cc
use kwargs map
daniel-sanche 03b1608
fixed on_error docstrings
daniel-sanche b05b11f
renamed example lists
daniel-sanche 0b5d3a2
removed ignore_sent
daniel-sanche 03f2af5
fixed lint issues
daniel-sanche 5fee888
fixed generator mock and added comments
daniel-sanche 239ed7d
Merge branch 'main' into retry_generators
daniel-sanche 94eb0f5
Merge branch 'main' into retry_generators
daniel-sanche 7d1e246
Merge branch 'main' into retry_generators
parthea b0faa2d
Apply suggestions from code review
daniel-sanche 6c44298
🦉 Updates from OwlBot post-processor
gcf-owl-bot[bot] 51df672
Update google/api_core/retry.py
daniel-sanche e207376
removed unneeded comments
daniel-sanche 39716a7
improved comments
daniel-sanche 2bbf33f
simplified generator detection
daniel-sanche 3b03bfa
renamed variables
daniel-sanche e63701d
improved comments
daniel-sanche c101ea6
renamed variable
daniel-sanche 3642d74
fixed tests
daniel-sanche 34cfa08
improved comments
daniel-sanche 583181d
Merge branch 'main' into retry_generators
daniel-sanche b311b87
fixed retry factory functionality
daniel-sanche 19a998d
created new objects for streaming retry config
daniel-sanche 5637e88
added typing to base retry
daniel-sanche c4be5f2
share base retry logic
daniel-sanche 4d9e762
🦉 Updates from OwlBot post-processor
gcf-owl-bot[bot] 2e9e84b
lint and mypy cleanup
daniel-sanche d183a7e
removed unneeded changes
daniel-sanche e2d9c9c
pass in initial args and kwargs to retry_target_stream
daniel-sanche 4543106
uncommented functools.wraps
daniel-sanche d791aad
Merge branch 'main' into retry_generators
daniel-sanche 638cc68
change enum encoding
daniel-sanche f7b1e14
moved base retry into own file
daniel-sanche 07db4c2
restructured files
daniel-sanche d448a52
expose other retry target functions in retry __init__
daniel-sanche 781426a
share a logger
daniel-sanche 4a05404
extracted shared error handling logic
daniel-sanche b221c8d
added type hints
daniel-sanche b5b4534
removed costly awaitable check
daniel-sanche 0f1145d
revised docstring
daniel-sanche 8408512
added exception_factory docstrings
daniel-sanche aa69c56
Revert "removed costly awaitable check"
daniel-sanche d1ac29d
renamed variable
daniel-sanche 3ab88fc
update docstring
daniel-sanche 382d0e2
add punctuation
daniel-sanche 4258823
punctuation
daniel-sanche 1bc9731
update docstrings
daniel-sanche aafe057
changed deadline to timeout
daniel-sanche 8095229
updated deadlien to timeout in docstrings
daniel-sanche de9f518
update docstring
daniel-sanche 7864667
update test comment
daniel-sanche 4c24322
update docstrings
daniel-sanche 7855513
removed unneeded comments
daniel-sanche f4bfb02
improved docstrings
daniel-sanche a88cf6f
use timeout in tests
daniel-sanche b5c62e1
🦉 Updates from OwlBot post-processor
gcf-owl-bot[bot] 852f4f8
moved test to proper place
daniel-sanche cd8323e
added test comments; fixed asserts
daniel-sanche ace61eb
use _build_retry_error as default param
daniel-sanche 1bbd1f0
ran blacken
daniel-sanche 35cc00a
added comment to clarify timeouts
daniel-sanche 89abfa4
removed timeout vs deadline explainer from retry_streaming
daniel-sanche 74ab817
remove duplicated test
daniel-sanche 85b3e02
fixed variable name
daniel-sanche 6dbe17d
made build_retry_error public
daniel-sanche 71e5888
changed docstring
daniel-sanche cbae3d3
import extra helper in retry_unary_async
daniel-sanche 61198b8
Merge branch 'main' into retry_generators
vchudnov-g acf9752
fix: address backwards compatibility warnings failing presubmits
vchudnov-g 7cf9fbf
fix: address mypy errors
vchudnov-g f62439a
fix: address coverage and lint issues failing presubmits
vchudnov-g b7abeca
chore: simplify resolution of backaward-compatibility warnings
vchudnov-g File filter
Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
There are no files selected for viewing
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Uh oh!
There was an error while loading. Please reload this page.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
it looks like we could just use sys.exc_info()[1] to be compatible with all versions?
sys.exception()
looks cleaner, but I don't think it's worth the extra noise for the next few years. (And there's a performance consideration to the version check too, although I don't thinkathrow
should be called often)There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Oh, that's a great idea. Let me try that.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
OK, thanks. That worked. Merging.