Skip to content

Create rule S7524 async for should be used with asynchronous iterators in async functions #5116

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
wants to merge 3 commits into
base: master
Choose a base branch
from
Open
Show file tree
Hide file tree
Changes from 1 commit
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 2 additions & 0 deletions rules/S7524/metadata.json
Original file line number Diff line number Diff line change
@@ -0,0 +1,2 @@
{
}
25 changes: 25 additions & 0 deletions rules/S7524/python/metadata.json
Original file line number Diff line number Diff line change
@@ -0,0 +1,25 @@
{
"title": "`async for` should be used with asynchronous iterators in `async` functions",
"type": "BUG",
"status": "ready",
"remediation": {
"func": "Constant/Issue",
"constantCost": "5min"
},
"tags": [
"async"
],
"defaultSeverity": "Major",
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think with Low the severity should be Minor

"ruleSpecification": "RSPEC-7524",
"sqKey": "S7524",
"scope": "All",
"defaultQualityProfiles": ["Sonar way"],
"quickfix": "covered",
"code": {
"impacts": {
"MAINTAINABILITY": "LOW",
"RELIABILITY": "LOW"
},
"attribute": "CONVENTIONAL"
}
}
112 changes: 112 additions & 0 deletions rules/S7524/python/rule.adoc
Original file line number Diff line number Diff line change
@@ -0,0 +1,112 @@
This rule raises an issue when a synchronous `for` loop is used with an iterator implementing both synchronous and asynchronous iteration protocols.
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm a bit confused why here (and throughout the rule description), we are focusing on iterators implementing both sync/async protocols.
I kind of expected a similar wording as S7515, where the message would be "if the iterator is async and we're in an async function, you should use the async protocol", no matter whether both protocols are implemented or not.

Here it seems we are saying that async for should be used even outside of async functions, which doesn't feel right (to be fair, we mention async functions in the "How to fix it" tab, but it's a bit late IMO).


== Why is this an issue?

Some iterators implement both synchronous and asynchronous iteration protocols, meaning they provide both:
- `+__iter__+`/`+__next__+` methods for synchronous iteration
- `+__aiter__+`/`+__anext__+` methods for asynchronous iteration

When working with such iterators inside `async` functions, a standard `for` loop bypasses their asynchronous capabilities. This can lead to missed opportunities for concurrent execution and inconsistent async/await patterns within your asynchronous code.

Using `async for` ensures that the asynchronous capabilities of the iterator are utilized, maintaining consistency with the asynchronous programming model and potentially allowing for better performance through concurrent operations.

=== What is the potential impact?

Using a standard `for` loop with these iterators in async functions can lead to:

* **Missed Concurrency**: The asynchronous iteration protocol may allow for concurrent operations or non-blocking I/O that are bypassed when using synchronous iteration.
* **Inconsistent Async Patterns**: Mixing synchronous iteration with asynchronous code makes the codebase less consistent and harder to reason about.
* **Performance Degradation**: Potential performance benefits from asynchronous iteration (such as yielding control to the event loop during I/O operations) are lost.
* **Code Clarity Issues**: Other developers may be confused about why an object with async capabilities is being used synchronously in an async context.

== How to fix it

Use the `async for` statement when iterating over such iterators within `async` functions. This ensures that the asynchronous iteration protocol is used, maintaining consistency with the async programming model.

=== Code examples

==== Noncompliant code example

[source,python,diff-id=1,diff-type=noncompliant]
----
import asyncio

class DualProtocolIterator:
def __init__(self):
...

# Synchronous iteration protocol
def __iter__(self):
...

def __next__(self):
...

# Asynchronous iteration protocol
def __aiter__(self):
...

async def __anext__(self):
...

async def process_data():
iterator = DualProtocolIterator()

for item in iterator: # Noncompliant
...
----

==== Compliant solution

[source,python,diff-id=1,diff-type=compliant]
----
import asyncio

class DualProtocolIterator:
def __init__(self):
...

# Synchronous iteration protocol
def __iter__(self):
...

def __next__(self):
...

# Asynchronous iteration protocol
def __aiter__(self):
...

async def __anext__(self):
...

async def process_data():
iterator = DualProtocolIterator()

async for item in iterator: # Compliant
...
----

ifdef::env-github,rspecator-view[]

== Implementation Specification
(visible only on this page)

=== Message

Use 'async for' to iterate over dual-protocol iterators in 'async' functions.
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Do we really need to focus on dual protocol iterators as well?
I guess using a simple for on an async iterator in an async function should be enough to raise, no? No need to ensure the iterator is dual? (I guess the "dual" part might influence whether we have a bug or a code smell, but that's it?).

Quickfix should be considered for implementation

=== Highlighting

* Primary location: The `for` keyword of the loop when used with a dual-protocol iterator inside an `async` function.
* Secondary locations: The `async` keyword of the enclosing function
endif::env-github,rspecator-view[]

== Resources

=== Documentation

* Python Documentation - https://docs.python.org/3/reference/compound_stmts.html#the-async-for-statement[The async for statement]
* Python Documentation - https://docs.python.org/3/reference/datamodel.html#the-iterator-protocol[The iterator protocol]
* Python Documentation - https://docs.python.org/3/reference/datamodel.html#asynchronous-iterators[Asynchronous Iterators]