Skip to content

chore: add cross_sync#999

Merged
daniel-sanche merged 270 commits intomainfrom
cross_sync2_pr1_architecture
Nov 8, 2024
Merged

chore: add cross_sync#999
daniel-sanche merged 270 commits intomainfrom
cross_sync2_pr1_architecture

Conversation

@daniel-sanche
Copy link
Contributor

@daniel-sanche daniel-sanche commented Jul 18, 2024

PR 1/x for adding the sync surface for the new data client

This class adds ast transformers for converting existing async classes into sync copies.

The conversion works through the following phases:

  • generate.py is called with a path, and finds all Python files in all subdirectories
  • CrossSyncClassDecoratorHandler is called on each file, looking for any classes decorated with @CrossSync.export_sync
  • For all classes found, they are passed through:
    • CrossSyncMethodDecoratorHandler to look for any methods annotated with CrossSync decorators (CrossSunc.convert, CrossSync.drop_method, CrossSync.pytest, etc).
    • RmAioFunctions to strip out any asyncio keywords annotated with CrossSync.rmaio()
    • SymbolReplacer to replace CrossSync with CrossSync._Sync_Impl in the codebase, and other user-specified symbol transformatioons

Next PR will add annotations to existing code to guide transformations

self.tree = ast_tree
self.header = header or ""

def render(self, with_black=True, save_to_disk: bool = False) -> str:
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

should we default save_to_disk to true?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I said no, because we also use this function as part of tests later, to compare to make sure the current outputs are up to date. In that case, we'd want to see the output without writing it. But I'm fine either way

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think a user of the library will want to set this to true, and we can override it for testing, let's make it default to true?

@daniel-sanche daniel-sanche force-pushed the cross_sync2_pr1_architecture branch from eda59d4 to 54e3007 Compare October 24, 2024 23:51
if new_mapping:
CrossSync.add_mapping(new_mapping, cls)
if self.async_docstring_format_vars:
cls.__doc__ = cls.__doc__.format(**self.async_docstring_format_vars)
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

what does this do? why is this only formatting async_docstring? How about sync_docstring?

Copy link
Contributor Author

@daniel-sanche daniel-sanche Oct 25, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

CrossSync allows you to add variables into docstrings, and then it will replace them with a different string for sync vs async. This came up because some async methods raise exceptions when run outside of an event loop context, but that doesn't apply to sync versions. So the docstrings are mostly the same, but with an extra little note for async

On the sync side, we can replace the variables when generating the sync file. On the async side, we update the __doc__ attribute of the class as part of the class decoration process. The code itself will have template variables when you read it directly, but docs generators will read from class.__doc__ when generating content, so it will render as expected

@daniel-sanche daniel-sanche added the kokoro:force-run Add this label to force Kokoro to re-run the tests. label Nov 5, 2024
@yoshi-kokoro yoshi-kokoro removed the kokoro:force-run Add this label to force Kokoro to re-run the tests. label Nov 5, 2024
@daniel-sanche daniel-sanche added the kokoro:force-run Add this label to force Kokoro to re-run the tests. label Nov 8, 2024
@yoshi-kokoro yoshi-kokoro removed the kokoro:force-run Add this label to force Kokoro to re-run the tests. label Nov 8, 2024
@daniel-sanche daniel-sanche merged commit 2057c20 into main Nov 8, 2024
@daniel-sanche daniel-sanche deleted the cross_sync2_pr1_architecture branch November 8, 2024 18:37
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

api: bigtable Issues related to the googleapis/python-bigtable API. size: xl Pull request size is extra large.

5 participants