Skip to content

fix: first pass on making retry configuration more consistent#695

Merged
igorbernstein2 merged 10 commits intogoogleapis:v1from
igorbernstein2:readrows-timeouts
Nov 18, 2022
Merged

fix: first pass on making retry configuration more consistent#695
igorbernstein2 merged 10 commits intogoogleapis:v1from
igorbernstein2:readrows-timeouts

Conversation

@igorbernstein2
Copy link
Contributor

Currently ReadRows uses the Retry.deadline configuration inconsistently:

  • its used as the attempt timeout for the first retry attempt
  • its used as a limit for retry scheduling for reading a single row

Conceptually there are 3 timeouts that are relevant to ReadRows:

  • attempt timeout: how long a single RPC is allowed to run, this should map directly to a gRPC deadline
  • overall timeout: Limit how long we should wait across all of the retry attempts, possibly truncating the last attempt timeout.
  • read timeout: How long we are willing to wait for the next row in a stream

Ideally Retry.deadline would represent an operation deadline (since thats the primary concern of the end user). However there is no backwards compatible way to do this. Changing the behavior would cause existing application to start enforcing a very short deadline.

This PR tries to improve the situation in a backwards compatible way:

  • keep Retry.deadline as a read timeout
  • introduce a new parameter for overall timeout

This results in less than ideal api, but avoids breaking existing applications.

Thank you for opening a Pull Request! Before submitting your PR, there are a few things you can do to make sure it goes smoothly:

  • Make sure to open an issue as a bug/issue before writing your code! That way we can discuss the change, evaluate designs, and agree on the general idea
  • Ensure the tests and linter pass
  • Code coverage does not decrease (if any source code was changed)
  • Appropriate docs were updated (if necessary)

Fixes #<issue_number_goes_here> 🦕

Currently ReadRows uses the Retry.deadline configuration inconsistently:
- its used as the attempt timeout for the first retry attempt
- its used as a limit for retry scheduling for reading a single row

Conceptually there are 3 timeouts that are relevant to ReadRows:
- attempt timeout: how long a single RPC is allowed to run, this should map directly to a gRPC deadline
- overall timeout: Limit how long we should wait across all of the retry attempts, possibly truncating the last attempt timeout.
- read timeout: How long we are willing to wait for the next row in a stream

Ideally Retry.deadline would represent an operation deadline (since thats the primary concern of the end user). However there is no backwards compatible way to do this. Changing the behavior would cause existing application to start enforcing a very short deadline.

This PR tries to improve the situation in a backwards compatible way:
- keep Retry.deadline as a read timeout
- introduce a new parameter for overall timeout

This results in less than ideal api, but avoids breaking existing applications.
@product-auto-label product-auto-label bot added size: m Pull request size is medium. api: bigtable Issues related to the googleapis/python-bigtable API. labels Nov 17, 2022
@igorbernstein2 igorbernstein2 added the kokoro:force-run Add this label to force Kokoro to re-run the tests. label Nov 17, 2022
@yoshi-kokoro yoshi-kokoro removed the kokoro:force-run Add this label to force Kokoro to re-run the tests. label Nov 17, 2022
@igorbernstein2 igorbernstein2 marked this pull request as ready for review November 18, 2022 04:05
@igorbernstein2 igorbernstein2 requested a review from a team November 18, 2022 04:05
@igorbernstein2 igorbernstein2 requested a review from a team as a code owner November 18, 2022 04:05
@Mariatta Mariatta added the kokoro:force-run Add this label to force Kokoro to re-run the tests. label Nov 18, 2022
@yoshi-kokoro yoshi-kokoro removed the kokoro:force-run Add this label to force Kokoro to re-run the tests. label Nov 18, 2022
@igorbernstein2 igorbernstein2 merged commit c707c30 into googleapis:v1 Nov 18, 2022
@igorbernstein2 igorbernstein2 deleted the readrows-timeouts branch November 18, 2022 18:04
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

api: bigtable Issues related to the googleapis/python-bigtable API. size: m Pull request size is medium.

4 participants