Skip to content

Commit 0a73ced

Browse files
authored
Consumer docstring cleanup (#394)
1 parent 0acc9b7 commit 0a73ced

File tree

9 files changed

+755
-742
lines changed

9 files changed

+755
-742
lines changed

docs/api-reference/application.md

Lines changed: 10 additions & 10 deletions
Original file line numberDiff line numberDiff line change
@@ -10,7 +10,7 @@
1010
class Application()
1111
```
1212

13-
[[VIEW SOURCE]](https://github.com/quixio/quix-streams/blob/209d74b1262aa6d90dddd1a01473c83d9d10fdce/quixstreams/app.py#L56)
13+
[[VIEW SOURCE]](https://github.com/quixio/quix-streams/blob/d1c312905abaa551ea99b4db5afe2a25d428e445/quixstreams/app.py#L56)
1414

1515
The main Application class.
1616

@@ -83,7 +83,7 @@ def __init__(broker_address: Optional[Union[str, ConnectionConfig]] = None,
8383
processing_guarantee: ProcessingGuarantee = "at-least-once")
8484
```
8585

86-
[[VIEW SOURCE]](https://github.com/quixio/quix-streams/blob/209d74b1262aa6d90dddd1a01473c83d9d10fdce/quixstreams/app.py#L94)
86+
[[VIEW SOURCE]](https://github.com/quixio/quix-streams/blob/d1c312905abaa551ea99b4db5afe2a25d428e445/quixstreams/app.py#L94)
8787

8888

8989
<br>
@@ -186,7 +186,7 @@ def Quix(
186186
) -> Self
187187
```
188188

189-
[[VIEW SOURCE]](https://github.com/quixio/quix-streams/blob/209d74b1262aa6d90dddd1a01473c83d9d10fdce/quixstreams/app.py#L330)
189+
[[VIEW SOURCE]](https://github.com/quixio/quix-streams/blob/d1c312905abaa551ea99b4db5afe2a25d428e445/quixstreams/app.py#L330)
190190

191191
>***NOTE:*** DEPRECATED: use Application with `quix_sdk_token` argument instead.
192192

@@ -297,7 +297,7 @@ def topic(name: str,
297297
timestamp_extractor: Optional[TimestampExtractor] = None) -> Topic
298298
```
299299

300-
[[VIEW SOURCE]](https://github.com/quixio/quix-streams/blob/209d74b1262aa6d90dddd1a01473c83d9d10fdce/quixstreams/app.py#L471)
300+
[[VIEW SOURCE]](https://github.com/quixio/quix-streams/blob/d1c312905abaa551ea99b4db5afe2a25d428e445/quixstreams/app.py#L471)
301301

302302
Create a topic definition.
303303

@@ -378,7 +378,7 @@ topic = app.topic("input-topic", timestamp_extractor=custom_ts_extractor)
378378
def dataframe(topic: Topic) -> StreamingDataFrame
379379
```
380380

381-
[[VIEW SOURCE]](https://github.com/quixio/quix-streams/blob/209d74b1262aa6d90dddd1a01473c83d9d10fdce/quixstreams/app.py#L551)
381+
[[VIEW SOURCE]](https://github.com/quixio/quix-streams/blob/d1c312905abaa551ea99b4db5afe2a25d428e445/quixstreams/app.py#L551)
382382

383383
A simple helper method that generates a `StreamingDataFrame`, which is used
384384

@@ -428,7 +428,7 @@ to be used as an input topic.
428428
def stop(fail: bool = False)
429429
```
430430

431-
[[VIEW SOURCE]](https://github.com/quixio/quix-streams/blob/209d74b1262aa6d90dddd1a01473c83d9d10fdce/quixstreams/app.py#L590)
431+
[[VIEW SOURCE]](https://github.com/quixio/quix-streams/blob/d1c312905abaa551ea99b4db5afe2a25d428e445/quixstreams/app.py#L590)
432432

433433
Stop the internal poll loop and the message processing.
434434

@@ -455,7 +455,7 @@ to unhandled exception, and it shouldn't commit the current checkpoint.
455455
def get_producer() -> Producer
456456
```
457457

458-
[[VIEW SOURCE]](https://github.com/quixio/quix-streams/blob/209d74b1262aa6d90dddd1a01473c83d9d10fdce/quixstreams/app.py#L613)
458+
[[VIEW SOURCE]](https://github.com/quixio/quix-streams/blob/d1c312905abaa551ea99b4db5afe2a25d428e445/quixstreams/app.py#L613)
459459

460460
Create and return a pre-configured Producer instance.
461461
The Producer is initialized with params passed to Application.
@@ -490,7 +490,7 @@ with app.get_producer() as producer:
490490
def get_consumer(auto_commit_enable: bool = True) -> Consumer
491491
```
492492

493-
[[VIEW SOURCE]](https://github.com/quixio/quix-streams/blob/209d74b1262aa6d90dddd1a01473c83d9d10fdce/quixstreams/app.py#L643)
493+
[[VIEW SOURCE]](https://github.com/quixio/quix-streams/blob/d1c312905abaa551ea99b4db5afe2a25d428e445/quixstreams/app.py#L643)
494494

495495
Create and return a pre-configured Consumer instance.
496496
The Consumer is initialized with params passed to Application.
@@ -539,7 +539,7 @@ with app.get_consumer() as consumer:
539539
def clear_state()
540540
```
541541

542-
[[VIEW SOURCE]](https://github.com/quixio/quix-streams/blob/209d74b1262aa6d90dddd1a01473c83d9d10fdce/quixstreams/app.py#L690)
542+
[[VIEW SOURCE]](https://github.com/quixio/quix-streams/blob/d1c312905abaa551ea99b4db5afe2a25d428e445/quixstreams/app.py#L690)
543543

544544
Clear the state of the application.
545545

@@ -553,7 +553,7 @@ Clear the state of the application.
553553
def run(dataframe: StreamingDataFrame)
554554
```
555555

556-
[[VIEW SOURCE]](https://github.com/quixio/quix-streams/blob/209d74b1262aa6d90dddd1a01473c83d9d10fdce/quixstreams/app.py#L696)
556+
[[VIEW SOURCE]](https://github.com/quixio/quix-streams/blob/d1c312905abaa551ea99b4db5afe2a25d428e445/quixstreams/app.py#L696)
557557

558558
Start processing data from Kafka using provided `StreamingDataFrame`
559559

docs/api-reference/context.md

Lines changed: 2 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -12,7 +12,7 @@
1212
def set_message_context(context: Optional[MessageContext])
1313
```
1414

15-
[[VIEW SOURCE]](https://github.com/quixio/quix-streams/blob/209d74b1262aa6d90dddd1a01473c83d9d10fdce/quixstreams/context.py#L20)
15+
[[VIEW SOURCE]](https://github.com/quixio/quix-streams/blob/d1c312905abaa551ea99b4db5afe2a25d428e445/quixstreams/context.py#L20)
1616

1717
Set a MessageContext for the current message in the given `contextvars.Context`
1818

@@ -55,7 +55,7 @@ sdf = sdf.update(lambda value: alter_context(value))
5555
def message_context() -> MessageContext
5656
```
5757

58-
[[VIEW SOURCE]](https://github.com/quixio/quix-streams/blob/209d74b1262aa6d90dddd1a01473c83d9d10fdce/quixstreams/context.py#L51)
58+
[[VIEW SOURCE]](https://github.com/quixio/quix-streams/blob/d1c312905abaa551ea99b4db5afe2a25d428e445/quixstreams/context.py#L51)
5959

6060
Get a MessageContext for the current message, which houses most of the message
6161

docs/api-reference/dataframe.md

Lines changed: 26 additions & 26 deletions
Original file line numberDiff line numberDiff line change
@@ -10,7 +10,7 @@
1010
class StreamingDataFrame(BaseStreaming)
1111
```
1212

13-
[[VIEW SOURCE]](https://github.com/quixio/quix-streams/blob/209d74b1262aa6d90dddd1a01473c83d9d10fdce/quixstreams/dataframe/dataframe.py#L62)
13+
[[VIEW SOURCE]](https://github.com/quixio/quix-streams/blob/d1c312905abaa551ea99b4db5afe2a25d428e445/quixstreams/dataframe/dataframe.py#L62)
1414

1515
`StreamingDataFrame` is the main object you will use for ETL work.
1616

@@ -81,7 +81,7 @@ def apply(func: Union[
8181
metadata: bool = False) -> Self
8282
```
8383

84-
[[VIEW SOURCE]](https://github.com/quixio/quix-streams/blob/209d74b1262aa6d90dddd1a01473c83d9d10fdce/quixstreams/dataframe/dataframe.py#L177)
84+
[[VIEW SOURCE]](https://github.com/quixio/quix-streams/blob/d1c312905abaa551ea99b4db5afe2a25d428e445/quixstreams/dataframe/dataframe.py#L177)
8585

8686
Apply a function to transform the value and return a new value.
8787

@@ -139,7 +139,7 @@ def update(func: Union[
139139
metadata: bool = False) -> Self
140140
```
141141

142-
[[VIEW SOURCE]](https://github.com/quixio/quix-streams/blob/209d74b1262aa6d90dddd1a01473c83d9d10fdce/quixstreams/dataframe/dataframe.py#L266)
142+
[[VIEW SOURCE]](https://github.com/quixio/quix-streams/blob/d1c312905abaa551ea99b4db5afe2a25d428e445/quixstreams/dataframe/dataframe.py#L266)
143143

144144
Apply a function to mutate value in-place or to perform a side effect
145145

@@ -197,7 +197,7 @@ def filter(func: Union[
197197
metadata: bool = False) -> Self
198198
```
199199

200-
[[VIEW SOURCE]](https://github.com/quixio/quix-streams/blob/209d74b1262aa6d90dddd1a01473c83d9d10fdce/quixstreams/dataframe/dataframe.py#L354)
200+
[[VIEW SOURCE]](https://github.com/quixio/quix-streams/blob/d1c312905abaa551ea99b4db5afe2a25d428e445/quixstreams/dataframe/dataframe.py#L354)
201201

202202
Filter value using provided function.
203203

@@ -249,7 +249,7 @@ def group_by(key: Union[str, Callable[[Any], Any]],
249249
key_serializer: Optional[SerializerType] = "json") -> Self
250250
```
251251

252-
[[VIEW SOURCE]](https://github.com/quixio/quix-streams/blob/209d74b1262aa6d90dddd1a01473c83d9d10fdce/quixstreams/dataframe/dataframe.py#L440)
252+
[[VIEW SOURCE]](https://github.com/quixio/quix-streams/blob/d1c312905abaa551ea99b4db5afe2a25d428e445/quixstreams/dataframe/dataframe.py#L440)
253253

254254
"Groups" messages by re-keying them via the provided group_by operation
255255

@@ -314,7 +314,7 @@ a clone with this operation added (assign to keep its effect).
314314
def contains(key: str) -> StreamingSeries
315315
```
316316

317-
[[VIEW SOURCE]](https://github.com/quixio/quix-streams/blob/209d74b1262aa6d90dddd1a01473c83d9d10fdce/quixstreams/dataframe/dataframe.py#L518)
317+
[[VIEW SOURCE]](https://github.com/quixio/quix-streams/blob/d1c312905abaa551ea99b4db5afe2a25d428e445/quixstreams/dataframe/dataframe.py#L518)
318318

319319
Check if the key is present in the Row value.
320320

@@ -353,7 +353,7 @@ or False otherwise.
353353
def to_topic(topic: Topic, key: Optional[Callable[[Any], Any]] = None) -> Self
354354
```
355355

356-
[[VIEW SOURCE]](https://github.com/quixio/quix-streams/blob/209d74b1262aa6d90dddd1a01473c83d9d10fdce/quixstreams/dataframe/dataframe.py#L543)
356+
[[VIEW SOURCE]](https://github.com/quixio/quix-streams/blob/d1c312905abaa551ea99b4db5afe2a25d428e445/quixstreams/dataframe/dataframe.py#L543)
357357

358358
Produce current value to a topic. You can optionally specify a new key.
359359

@@ -396,7 +396,7 @@ By default, the current message key will be used.
396396
def set_timestamp(func: Callable[[Any, Any, int, Any], int]) -> Self
397397
```
398398

399-
[[VIEW SOURCE]](https://github.com/quixio/quix-streams/blob/209d74b1262aa6d90dddd1a01473c83d9d10fdce/quixstreams/dataframe/dataframe.py#L584)
399+
[[VIEW SOURCE]](https://github.com/quixio/quix-streams/blob/d1c312905abaa551ea99b4db5afe2a25d428e445/quixstreams/dataframe/dataframe.py#L584)
400400

401401
Set a new timestamp based on the current message value and its metadata.
402402

@@ -449,7 +449,7 @@ def set_headers(
449449
) -> Self
450450
```
451451

452-
[[VIEW SOURCE]](https://github.com/quixio/quix-streams/blob/209d74b1262aa6d90dddd1a01473c83d9d10fdce/quixstreams/dataframe/dataframe.py#L625)
452+
[[VIEW SOURCE]](https://github.com/quixio/quix-streams/blob/d1c312905abaa551ea99b4db5afe2a25d428e445/quixstreams/dataframe/dataframe.py#L625)
453453

454454
Set new message headers based on the current message value and metadata.
455455

@@ -500,7 +500,7 @@ def compose(
500500
) -> Dict[str, VoidExecutor]
501501
```
502502

503-
[[VIEW SOURCE]](https://github.com/quixio/quix-streams/blob/209d74b1262aa6d90dddd1a01473c83d9d10fdce/quixstreams/dataframe/dataframe.py#L676)
503+
[[VIEW SOURCE]](https://github.com/quixio/quix-streams/blob/d1c312905abaa551ea99b4db5afe2a25d428e445/quixstreams/dataframe/dataframe.py#L676)
504504

505505
Compose all functions of this StreamingDataFrame into one big closure.
506506

@@ -554,7 +554,7 @@ def test(value: Any,
554554
topic: Optional[Topic] = None) -> List[Any]
555555
```
556556

557-
[[VIEW SOURCE]](https://github.com/quixio/quix-streams/blob/209d74b1262aa6d90dddd1a01473c83d9d10fdce/quixstreams/dataframe/dataframe.py#L713)
557+
[[VIEW SOURCE]](https://github.com/quixio/quix-streams/blob/d1c312905abaa551ea99b4db5afe2a25d428e445/quixstreams/dataframe/dataframe.py#L713)
558558

559559
A shorthand to test `StreamingDataFrame` with provided value
560560

@@ -591,7 +591,7 @@ def tumbling_window(duration_ms: Union[int, timedelta],
591591
name: Optional[str] = None) -> TumblingWindowDefinition
592592
```
593593

594-
[[VIEW SOURCE]](https://github.com/quixio/quix-streams/blob/209d74b1262aa6d90dddd1a01473c83d9d10fdce/quixstreams/dataframe/dataframe.py#L750)
594+
[[VIEW SOURCE]](https://github.com/quixio/quix-streams/blob/d1c312905abaa551ea99b4db5afe2a25d428e445/quixstreams/dataframe/dataframe.py#L750)
595595

596596
Create a tumbling window transformation on this StreamingDataFrame.
597597

@@ -677,7 +677,7 @@ def hopping_window(duration_ms: Union[int, timedelta],
677677
name: Optional[str] = None) -> HoppingWindowDefinition
678678
```
679679

680-
[[VIEW SOURCE]](https://github.com/quixio/quix-streams/blob/209d74b1262aa6d90dddd1a01473c83d9d10fdce/quixstreams/dataframe/dataframe.py#L826)
680+
[[VIEW SOURCE]](https://github.com/quixio/quix-streams/blob/d1c312905abaa551ea99b4db5afe2a25d428e445/quixstreams/dataframe/dataframe.py#L826)
681681

682682
Create a hopping window transformation on this StreamingDataFrame.
683683

@@ -771,7 +771,7 @@ sdf = (
771771
class StreamingSeries(BaseStreaming)
772772
```
773773

774-
[[VIEW SOURCE]](https://github.com/quixio/quix-streams/blob/209d74b1262aa6d90dddd1a01473c83d9d10fdce/quixstreams/dataframe/series.py#L47)
774+
[[VIEW SOURCE]](https://github.com/quixio/quix-streams/blob/d1c312905abaa551ea99b4db5afe2a25d428e445/quixstreams/dataframe/series.py#L47)
775775

776776
`StreamingSeries` are typically generated by `StreamingDataframes` when getting
777777
elements from, or performing certain operations on, a `StreamingDataframe`,
@@ -837,7 +837,7 @@ sdf = sdf[["column_a"] & (sdf["new_sum_field"] >= 10)]
837837
def from_apply_callback(cls, func: ApplyWithMetadataCallback) -> Self
838838
```
839839

840-
[[VIEW SOURCE]](https://github.com/quixio/quix-streams/blob/209d74b1262aa6d90dddd1a01473c83d9d10fdce/quixstreams/dataframe/series.py#L107)
840+
[[VIEW SOURCE]](https://github.com/quixio/quix-streams/blob/d1c312905abaa551ea99b4db5afe2a25d428e445/quixstreams/dataframe/series.py#L107)
841841

842842
Create a StreamingSeries from a function.
843843

@@ -865,7 +865,7 @@ instance of `StreamingSeries`
865865
def apply(func: ApplyCallback) -> Self
866866
```
867867

868-
[[VIEW SOURCE]](https://github.com/quixio/quix-streams/blob/209d74b1262aa6d90dddd1a01473c83d9d10fdce/quixstreams/dataframe/series.py#L121)
868+
[[VIEW SOURCE]](https://github.com/quixio/quix-streams/blob/d1c312905abaa551ea99b4db5afe2a25d428e445/quixstreams/dataframe/series.py#L121)
869869

870870
Add a callable to the execution list for this series.
871871

@@ -917,7 +917,7 @@ a new `StreamingSeries` with the new callable added
917917
def compose_returning() -> ReturningExecutor
918918
```
919919

920-
[[VIEW SOURCE]](https://github.com/quixio/quix-streams/blob/209d74b1262aa6d90dddd1a01473c83d9d10fdce/quixstreams/dataframe/series.py#L155)
920+
[[VIEW SOURCE]](https://github.com/quixio/quix-streams/blob/d1c312905abaa551ea99b4db5afe2a25d428e445/quixstreams/dataframe/series.py#L155)
921921

922922
Compose a list of functions from this StreamingSeries and its parents into one
923923

@@ -948,7 +948,7 @@ def compose(
948948
None]] = None) -> VoidExecutor
949949
```
950950

951-
[[VIEW SOURCE]](https://github.com/quixio/quix-streams/blob/209d74b1262aa6d90dddd1a01473c83d9d10fdce/quixstreams/dataframe/series.py#L170)
951+
[[VIEW SOURCE]](https://github.com/quixio/quix-streams/blob/d1c312905abaa551ea99b4db5afe2a25d428e445/quixstreams/dataframe/series.py#L170)
952952

953953
Compose all functions of this StreamingSeries into one big closure.
954954

@@ -1006,7 +1006,7 @@ def test(value: Any,
10061006
ctx: Optional[MessageContext] = None) -> Any
10071007
```
10081008

1009-
[[VIEW SOURCE]](https://github.com/quixio/quix-streams/blob/209d74b1262aa6d90dddd1a01473c83d9d10fdce/quixstreams/dataframe/series.py#L214)
1009+
[[VIEW SOURCE]](https://github.com/quixio/quix-streams/blob/d1c312905abaa551ea99b4db5afe2a25d428e445/quixstreams/dataframe/series.py#L214)
10101010

10111011
A shorthand to test `StreamingSeries` with provided value
10121012

@@ -1038,7 +1038,7 @@ result of `StreamingSeries`
10381038
def isin(other: Container) -> Self
10391039
```
10401040

1041-
[[VIEW SOURCE]](https://github.com/quixio/quix-streams/blob/209d74b1262aa6d90dddd1a01473c83d9d10fdce/quixstreams/dataframe/series.py#L269)
1041+
[[VIEW SOURCE]](https://github.com/quixio/quix-streams/blob/d1c312905abaa551ea99b4db5afe2a25d428e445/quixstreams/dataframe/series.py#L269)
10421042

10431043
Check if series value is in "other".
10441044

@@ -1083,7 +1083,7 @@ new StreamingSeries
10831083
def contains(other: Union[Self, object]) -> Self
10841084
```
10851085

1086-
[[VIEW SOURCE]](https://github.com/quixio/quix-streams/blob/209d74b1262aa6d90dddd1a01473c83d9d10fdce/quixstreams/dataframe/series.py#L296)
1086+
[[VIEW SOURCE]](https://github.com/quixio/quix-streams/blob/d1c312905abaa551ea99b4db5afe2a25d428e445/quixstreams/dataframe/series.py#L296)
10871087

10881088
Check if series value contains "other"
10891089

@@ -1128,7 +1128,7 @@ new StreamingSeries
11281128
def is_(other: Union[Self, object]) -> Self
11291129
```
11301130

1131-
[[VIEW SOURCE]](https://github.com/quixio/quix-streams/blob/209d74b1262aa6d90dddd1a01473c83d9d10fdce/quixstreams/dataframe/series.py#L321)
1131+
[[VIEW SOURCE]](https://github.com/quixio/quix-streams/blob/d1c312905abaa551ea99b4db5afe2a25d428e445/quixstreams/dataframe/series.py#L321)
11321132

11331133
Check if series value refers to the same object as `other`
11341134

@@ -1170,7 +1170,7 @@ new StreamingSeries
11701170
def isnot(other: Union[Self, object]) -> Self
11711171
```
11721172

1173-
[[VIEW SOURCE]](https://github.com/quixio/quix-streams/blob/209d74b1262aa6d90dddd1a01473c83d9d10fdce/quixstreams/dataframe/series.py#L344)
1173+
[[VIEW SOURCE]](https://github.com/quixio/quix-streams/blob/d1c312905abaa551ea99b4db5afe2a25d428e445/quixstreams/dataframe/series.py#L344)
11741174

11751175
Check if series value does not refer to the same object as `other`
11761176

@@ -1213,7 +1213,7 @@ new StreamingSeries
12131213
def isnull() -> Self
12141214
```
12151215

1216-
[[VIEW SOURCE]](https://github.com/quixio/quix-streams/blob/209d74b1262aa6d90dddd1a01473c83d9d10fdce/quixstreams/dataframe/series.py#L368)
1216+
[[VIEW SOURCE]](https://github.com/quixio/quix-streams/blob/d1c312905abaa551ea99b4db5afe2a25d428e445/quixstreams/dataframe/series.py#L368)
12171217

12181218
Check if series value is None.
12191219

@@ -1250,7 +1250,7 @@ new StreamingSeries
12501250
def notnull() -> Self
12511251
```
12521252

1253-
[[VIEW SOURCE]](https://github.com/quixio/quix-streams/blob/209d74b1262aa6d90dddd1a01473c83d9d10fdce/quixstreams/dataframe/series.py#L391)
1253+
[[VIEW SOURCE]](https://github.com/quixio/quix-streams/blob/d1c312905abaa551ea99b4db5afe2a25d428e445/quixstreams/dataframe/series.py#L391)
12541254

12551255
Check if series value is not None.
12561256

@@ -1287,7 +1287,7 @@ new StreamingSeries
12871287
def abs() -> Self
12881288
```
12891289

1290-
[[VIEW SOURCE]](https://github.com/quixio/quix-streams/blob/209d74b1262aa6d90dddd1a01473c83d9d10fdce/quixstreams/dataframe/series.py#L414)
1290+
[[VIEW SOURCE]](https://github.com/quixio/quix-streams/blob/d1c312905abaa551ea99b4db5afe2a25d428e445/quixstreams/dataframe/series.py#L414)
12911291

12921292
Get absolute value of the series value.
12931293

0 commit comments

Comments
 (0)