Skip to content

Refactor async generator #226

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 1 commit into from
Jul 26, 2025
Merged

Refactor async generator #226

Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
14 changes: 9 additions & 5 deletions script/generate_async.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -99,17 +99,21 @@ def process(source: Path, target: Path):
with Path.open(module_path) as fp:
module_dump = fp.read()

module_dump = re.sub(r"( {4}def )(?!_)", r" async def ", module_dump)

module_dump = re.sub(r"self\.client\.(.+)\(", r"await self.client.\1(", module_dump)

# Adjust imports.
for relative_import in [".base", "..client", "..knowledge", "..model"]:
module_dump = module_dump.replace(f"from {relative_import}", f"from .{relative_import}")

module_dump = module_dump.replace("self.api.version", "await self.api.version")
# Modify function definitions.
module_dump = re.sub(r"( {4}def )(?!_)", r" async def ", module_dump)

Comment on lines +106 to +108
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🛠️ Refactor suggestion

Function-definition regex misses indents ≠ 4 & may hit code snippets

  1. It only matches exactly four leading spaces – methods nested deeper (e.g. inside an inner class) stay synchronous.
  2. The pattern isn’t anchored to the line start so a " def foo" string in a doc-string would also be changed.
-module_dump = re.sub(r"( {4}def )(?!_)", r"    async def ", module_dump)
+module_dump = re.sub(
+    r'^(\s{4,})def (?!_)',    # any indent ≥4 at BOL, next char not “_”
+    r'\1async def ',
+    module_dump,
+    flags=re.MULTILINE,
+)
📝 Committable suggestion

‼️ IMPORTANT
Carefully review the code before committing. Ensure that it accurately replaces the highlighted code, contains no missing lines, and has no issues with indentation. Thoroughly test & benchmark the code to ensure it meets the requirements.

Suggested change
# Modify function definitions.
module_dump = re.sub(r"( {4}def )(?!_)", r" async def ", module_dump)
# Modify function definitions.
module_dump = re.sub(
r'^(\s{4,})def (?!_)', # any indent ≥4 at BOL, next char not “_”
r'\1async def ',
module_dump,
flags=re.MULTILINE,
)
🤖 Prompt for AI Agents 
In script/generate_async.py around lines 104 to 106, the regex for modifying
function definitions only matches lines with exactly four leading spaces and is
not anchored to the start of the line, causing it to miss nested methods and
potentially alter strings in doc-strings. Update the regex to anchor it to the
start of the line and allow matching any number of leading spaces before "def"
to correctly identify all function definitions regardless of indentation level,
while avoiding changes inside strings or comments.

# Modify function calls.
module_dump = re.sub(r"self\.client\.(.+)\(", r"await self.client.\1(", module_dump)
module_dump = re.sub(r"= self\.(.+)\(", r"= await self.\1(", module_dump)
module_dump = re.sub(r"send_request\(", r"await send_request(", module_dump)

Comment on lines +110 to 113
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🛠️ Refactor suggestion

Greedy .+ may over-capture; tighten the call-site patterns

The current patterns can unexpectedly swallow dots or spaces until the next “(”.
Limiting the capture to identifier characters removes that ambiguity and avoids matching across comments.

-module_dump = re.sub(r"self\.client\.(.+)\(", r"await self.client.\1(", module_dump)
-module_dump = re.sub(r"= self\.(.+)\(", r"= await self.\1(", module_dump)
+module_dump = re.sub(r"self\.client\.([A-Za-z_][A-Za-z0-9_]*)\(", r"await self.client.\1(", module_dump)
+module_dump = re.sub(r"= (\s*)self\.([A-Za-z_][A-Za-z0-9_]*)\(", r"= \1await self.\2(", module_dump)
📝 Committable suggestion

‼️ IMPORTANT
Carefully review the code before committing. Ensure that it accurately replaces the highlighted code, contains no missing lines, and has no issues with indentation. Thoroughly test & benchmark the code to ensure it meets the requirements.

Suggested change
module_dump = re.sub(r"self\.client\.(.+)\(", r"await self.client.\1(", module_dump)
module_dump = re.sub(r"= self\.(.+)\(", r"= await self.\1(", module_dump)
module_dump = re.sub(r"send_request\(", r"await send_request(", module_dump)
module_dump = re.sub(r"self\.client\.([A-Za-z_][A-Za-z0-9_]*)\(", r"await self.client.\1(", module_dump)
module_dump = re.sub(r"= (\s*)self\.([A-Za-z_][A-Za-z0-9_]*)\(", r"= \1await self.\2(", module_dump)
module_dump = re.sub(r"send_request\(", r"await send_request(", module_dump)
🤖 Prompt for AI Agents 
In script/generate_async.py around lines 108 to 111, the regex patterns use
greedy .+ which can over-capture and match unintended characters. Replace .+
with a more precise pattern that matches only valid identifier characters (e.g.,
\w+) to ensure the substitutions only target method names without including
dots, spaces, or other characters. Update all three re.sub calls accordingly to
use this tightened pattern.

# Modify property accesses.
module_dump = module_dump.replace("self.api.version", "await self.api.version")

module_processed.append(module_path)
target_path = Path(str(module_path).replace(str(source), str(target)))
with Path.open(target_path, "w") as fp:
Expand Down
Loading