Skip to content

Commit 010b595

Browse files
Update documentation (#598)
Co-authored-by: daniil-quix <133032822+daniil-quix@users.noreply.github.com>
1 parent c27172c commit 010b595

File tree

9 files changed

+508
-471
lines changed

9 files changed

+508
-471
lines changed

docs/api-reference/application.md

Lines changed: 5 additions & 5 deletions
Original file line numberDiff line numberDiff line change
@@ -494,7 +494,7 @@ Default: the source default
494494
def run(dataframe: Optional[StreamingDataFrame] = None)
495495
```
496496

497-
[[VIEW SOURCE]](https://github.com/quixio/quix-streams/blob/main/quixstreams/app.py#L658)
497+
[[VIEW SOURCE]](https://github.com/quixio/quix-streams/blob/main/quixstreams/app.py#L657)
498498

499499
Start processing data from Kafka using provided `StreamingDataFrame`
500500

@@ -530,7 +530,7 @@ app.run()
530530
def setup_topics()
531531
```
532532

533-
[[VIEW SOURCE]](https://github.com/quixio/quix-streams/blob/main/quixstreams/app.py#L780)
533+
[[VIEW SOURCE]](https://github.com/quixio/quix-streams/blob/main/quixstreams/app.py#L779)
534534

535535
Validate and create the topics
536536

@@ -542,7 +542,7 @@ Validate and create the topics
542542
class ApplicationConfig(BaseSettings)
543543
```
544544

545-
[[VIEW SOURCE]](https://github.com/quixio/quix-streams/blob/main/quixstreams/app.py#L956)
545+
[[VIEW SOURCE]](https://github.com/quixio/quix-streams/blob/main/quixstreams/app.py#L955)
546546

547547
Immutable object holding the application configuration
548548

@@ -565,7 +565,7 @@ def settings_customise_sources(
565565
) -> Tuple[PydanticBaseSettingsSource, ...]
566566
```
567567

568-
[[VIEW SOURCE]](https://github.com/quixio/quix-streams/blob/main/quixstreams/app.py#L991)
568+
[[VIEW SOURCE]](https://github.com/quixio/quix-streams/blob/main/quixstreams/app.py#L990)
569569

570570
Included to ignore reading/setting values from the environment
571571

@@ -579,7 +579,7 @@ Included to ignore reading/setting values from the environment
579579
def copy(**kwargs) -> Self
580580
```
581581

582-
[[VIEW SOURCE]](https://github.com/quixio/quix-streams/blob/main/quixstreams/app.py#L1004)
582+
[[VIEW SOURCE]](https://github.com/quixio/quix-streams/blob/main/quixstreams/app.py#L1003)
583583

584584
Update the application config and return a copy
585585

docs/api-reference/dataframe.md

Lines changed: 29 additions & 29 deletions
Original file line numberDiff line numberDiff line change
@@ -10,7 +10,7 @@
1010
class StreamingDataFrame(BaseStreaming)
1111
```
1212

13-
[[VIEW SOURCE]](https://github.com/quixio/quix-streams/blob/main/quixstreams/dataframe/dataframe.py#L64)
13+
[[VIEW SOURCE]](https://github.com/quixio/quix-streams/blob/main/quixstreams/dataframe/dataframe.py#L65)
1414

1515
`StreamingDataFrame` is the main object you will use for ETL work.
1616

@@ -81,7 +81,7 @@ def apply(func: Union[
8181
metadata: bool = False) -> Self
8282
```
8383

84-
[[VIEW SOURCE]](https://github.com/quixio/quix-streams/blob/main/quixstreams/dataframe/dataframe.py#L170)
84+
[[VIEW SOURCE]](https://github.com/quixio/quix-streams/blob/main/quixstreams/dataframe/dataframe.py#L171)
8585

8686
Apply a function to transform the value and return a new value.
8787

@@ -139,7 +139,7 @@ def update(func: Union[
139139
metadata: bool = False) -> Self
140140
```
141141

142-
[[VIEW SOURCE]](https://github.com/quixio/quix-streams/blob/main/quixstreams/dataframe/dataframe.py#L259)
142+
[[VIEW SOURCE]](https://github.com/quixio/quix-streams/blob/main/quixstreams/dataframe/dataframe.py#L260)
143143

144144
Apply a function to mutate value in-place or to perform a side effect
145145

@@ -207,7 +207,7 @@ def filter(func: Union[
207207
metadata: bool = False) -> Self
208208
```
209209

210-
[[VIEW SOURCE]](https://github.com/quixio/quix-streams/blob/main/quixstreams/dataframe/dataframe.py#L351)
210+
[[VIEW SOURCE]](https://github.com/quixio/quix-streams/blob/main/quixstreams/dataframe/dataframe.py#L352)
211211

212212
Filter value using provided function.
213213

@@ -259,7 +259,7 @@ def group_by(key: Union[str, Callable[[Any], Any]],
259259
key_serializer: Optional[SerializerType] = "json") -> Self
260260
```
261261

262-
[[VIEW SOURCE]](https://github.com/quixio/quix-streams/blob/main/quixstreams/dataframe/dataframe.py#L437)
262+
[[VIEW SOURCE]](https://github.com/quixio/quix-streams/blob/main/quixstreams/dataframe/dataframe.py#L438)
263263

264264
"Groups" messages by re-keying them via the provided group_by operation
265265

@@ -323,7 +323,7 @@ a clone with this operation added (assign to keep its effect).
323323
def contains(key: str) -> StreamingSeries
324324
```
325325

326-
[[VIEW SOURCE]](https://github.com/quixio/quix-streams/blob/main/quixstreams/dataframe/dataframe.py#L510)
326+
[[VIEW SOURCE]](https://github.com/quixio/quix-streams/blob/main/quixstreams/dataframe/dataframe.py#L511)
327327

328328
Check if the key is present in the Row value.
329329

@@ -362,7 +362,7 @@ or False otherwise.
362362
def to_topic(topic: Topic, key: Optional[Callable[[Any], Any]] = None) -> Self
363363
```
364364

365-
[[VIEW SOURCE]](https://github.com/quixio/quix-streams/blob/main/quixstreams/dataframe/dataframe.py#L535)
365+
[[VIEW SOURCE]](https://github.com/quixio/quix-streams/blob/main/quixstreams/dataframe/dataframe.py#L536)
366366

367367
Produce current value to a topic. You can optionally specify a new key.
368368

@@ -415,7 +415,7 @@ the updated StreamingDataFrame instance (reassignment NOT required).
415415
def set_timestamp(func: Callable[[Any, Any, int, Any], int]) -> Self
416416
```
417417

418-
[[VIEW SOURCE]](https://github.com/quixio/quix-streams/blob/main/quixstreams/dataframe/dataframe.py#L580)
418+
[[VIEW SOURCE]](https://github.com/quixio/quix-streams/blob/main/quixstreams/dataframe/dataframe.py#L581)
419419

420420
Set a new timestamp based on the current message value and its metadata.
421421

@@ -468,7 +468,7 @@ def set_headers(
468468
) -> Self
469469
```
470470

471-
[[VIEW SOURCE]](https://github.com/quixio/quix-streams/blob/main/quixstreams/dataframe/dataframe.py#L621)
471+
[[VIEW SOURCE]](https://github.com/quixio/quix-streams/blob/main/quixstreams/dataframe/dataframe.py#L622)
472472

473473
Set new message headers based on the current message value and metadata.
474474

@@ -517,7 +517,7 @@ a new StreamingDataFrame instance
517517
def print(pretty: bool = True, metadata: bool = False) -> Self
518518
```
519519

520-
[[VIEW SOURCE]](https://github.com/quixio/quix-streams/blob/main/quixstreams/dataframe/dataframe.py#L672)
520+
[[VIEW SOURCE]](https://github.com/quixio/quix-streams/blob/main/quixstreams/dataframe/dataframe.py#L673)
521521

522522
Print out the current message value (and optionally, the message metadata) to
523523

@@ -573,7 +573,7 @@ def compose(
573573
) -> Dict[str, VoidExecutor]
574574
```
575575

576-
[[VIEW SOURCE]](https://github.com/quixio/quix-streams/blob/main/quixstreams/dataframe/dataframe.py#L714)
576+
[[VIEW SOURCE]](https://github.com/quixio/quix-streams/blob/main/quixstreams/dataframe/dataframe.py#L715)
577577

578578
Compose all functions of this StreamingDataFrame into one big closure.
579579

@@ -627,7 +627,7 @@ def test(value: Any,
627627
topic: Optional[Topic] = None) -> List[Any]
628628
```
629629

630-
[[VIEW SOURCE]](https://github.com/quixio/quix-streams/blob/main/quixstreams/dataframe/dataframe.py#L748)
630+
[[VIEW SOURCE]](https://github.com/quixio/quix-streams/blob/main/quixstreams/dataframe/dataframe.py#L749)
631631

632632
A shorthand to test `StreamingDataFrame` with provided value
633633

@@ -664,7 +664,7 @@ def tumbling_window(duration_ms: Union[int, timedelta],
664664
name: Optional[str] = None) -> TumblingWindowDefinition
665665
```
666666

667-
[[VIEW SOURCE]](https://github.com/quixio/quix-streams/blob/main/quixstreams/dataframe/dataframe.py#L785)
667+
[[VIEW SOURCE]](https://github.com/quixio/quix-streams/blob/main/quixstreams/dataframe/dataframe.py#L786)
668668

669669
Create a tumbling window transformation on this StreamingDataFrame.
670670

@@ -750,7 +750,7 @@ def hopping_window(duration_ms: Union[int, timedelta],
750750
name: Optional[str] = None) -> HoppingWindowDefinition
751751
```
752752

753-
[[VIEW SOURCE]](https://github.com/quixio/quix-streams/blob/main/quixstreams/dataframe/dataframe.py#L861)
753+
[[VIEW SOURCE]](https://github.com/quixio/quix-streams/blob/main/quixstreams/dataframe/dataframe.py#L862)
754754

755755
Create a hopping window transformation on this StreamingDataFrame.
756756

@@ -843,7 +843,7 @@ def drop(columns: Union[str, List[str]],
843843
errors: Literal["ignore", "raise"] = "raise") -> Self
844844
```
845845

846-
[[VIEW SOURCE]](https://github.com/quixio/quix-streams/blob/main/quixstreams/dataframe/dataframe.py#L953)
846+
[[VIEW SOURCE]](https://github.com/quixio/quix-streams/blob/main/quixstreams/dataframe/dataframe.py#L954)
847847

848848
Drop column(s) from the message value (value must support `del`, like a dict).
849849

@@ -887,7 +887,7 @@ a new StreamingDataFrame instance
887887
def sink(sink: BaseSink)
888888
```
889889

890-
[[VIEW SOURCE]](https://github.com/quixio/quix-streams/blob/main/quixstreams/dataframe/dataframe.py#L997)
890+
[[VIEW SOURCE]](https://github.com/quixio/quix-streams/blob/main/quixstreams/dataframe/dataframe.py#L998)
891891

892892
Sink the processed data to the specified destination.
893893

@@ -917,7 +917,7 @@ operations, but branches can still be generated from its originating SDF.
917917
class StreamingSeries(BaseStreaming)
918918
```
919919

920-
[[VIEW SOURCE]](https://github.com/quixio/quix-streams/blob/main/quixstreams/dataframe/series.py#L69)
920+
[[VIEW SOURCE]](https://github.com/quixio/quix-streams/blob/main/quixstreams/dataframe/series.py#L70)
921921

922922
`StreamingSeries` are typically generated by `StreamingDataframes` when getting
923923
elements from, or performing certain operations on, a `StreamingDataframe`,
@@ -984,7 +984,7 @@ def from_apply_callback(cls, func: ApplyWithMetadataCallback,
984984
sdf_id: int) -> Self
985985
```
986986

987-
[[VIEW SOURCE]](https://github.com/quixio/quix-streams/blob/main/quixstreams/dataframe/series.py#L131)
987+
[[VIEW SOURCE]](https://github.com/quixio/quix-streams/blob/main/quixstreams/dataframe/series.py#L132)
988988

989989
Create a StreamingSeries from a function.
990990

@@ -1013,7 +1013,7 @@ instance of `StreamingSeries`
10131013
def apply(func: ApplyCallback) -> Self
10141014
```
10151015

1016-
[[VIEW SOURCE]](https://github.com/quixio/quix-streams/blob/main/quixstreams/dataframe/series.py#L154)
1016+
[[VIEW SOURCE]](https://github.com/quixio/quix-streams/blob/main/quixstreams/dataframe/series.py#L155)
10171017

10181018
Add a callable to the execution list for this series.
10191019

@@ -1065,7 +1065,7 @@ a new `StreamingSeries` with the new callable added
10651065
def compose_returning() -> ReturningExecutor
10661066
```
10671067

1068-
[[VIEW SOURCE]](https://github.com/quixio/quix-streams/blob/main/quixstreams/dataframe/series.py#L188)
1068+
[[VIEW SOURCE]](https://github.com/quixio/quix-streams/blob/main/quixstreams/dataframe/series.py#L189)
10691069

10701070
Compose a list of functions from this StreamingSeries and its parents into one
10711071

@@ -1096,7 +1096,7 @@ def compose(
10961096
None]] = None) -> VoidExecutor
10971097
```
10981098

1099-
[[VIEW SOURCE]](https://github.com/quixio/quix-streams/blob/main/quixstreams/dataframe/series.py#L203)
1099+
[[VIEW SOURCE]](https://github.com/quixio/quix-streams/blob/main/quixstreams/dataframe/series.py#L204)
11001100

11011101
Compose all functions of this StreamingSeries into one big closure.
11021102

@@ -1154,7 +1154,7 @@ def test(value: Any,
11541154
ctx: Optional[MessageContext] = None) -> Any
11551155
```
11561156

1157-
[[VIEW SOURCE]](https://github.com/quixio/quix-streams/blob/main/quixstreams/dataframe/series.py#L247)
1157+
[[VIEW SOURCE]](https://github.com/quixio/quix-streams/blob/main/quixstreams/dataframe/series.py#L248)
11581158

11591159
A shorthand to test `StreamingSeries` with provided value
11601160

@@ -1186,7 +1186,7 @@ result of `StreamingSeries`
11861186
def isin(other: Container) -> Self
11871187
```
11881188

1189-
[[VIEW SOURCE]](https://github.com/quixio/quix-streams/blob/main/quixstreams/dataframe/series.py#L303)
1189+
[[VIEW SOURCE]](https://github.com/quixio/quix-streams/blob/main/quixstreams/dataframe/series.py#L304)
11901190

11911191
Check if series value is in "other".
11921192

@@ -1231,7 +1231,7 @@ new StreamingSeries
12311231
def contains(other: Union[Self, object]) -> Self
12321232
```
12331233

1234-
[[VIEW SOURCE]](https://github.com/quixio/quix-streams/blob/main/quixstreams/dataframe/series.py#L330)
1234+
[[VIEW SOURCE]](https://github.com/quixio/quix-streams/blob/main/quixstreams/dataframe/series.py#L331)
12351235

12361236
Check if series value contains "other"
12371237

@@ -1276,7 +1276,7 @@ new StreamingSeries
12761276
def is_(other: Union[Self, object]) -> Self
12771277
```
12781278

1279-
[[VIEW SOURCE]](https://github.com/quixio/quix-streams/blob/main/quixstreams/dataframe/series.py#L355)
1279+
[[VIEW SOURCE]](https://github.com/quixio/quix-streams/blob/main/quixstreams/dataframe/series.py#L356)
12801280

12811281
Check if series value refers to the same object as `other`
12821282

@@ -1318,7 +1318,7 @@ new StreamingSeries
13181318
def isnot(other: Union[Self, object]) -> Self
13191319
```
13201320

1321-
[[VIEW SOURCE]](https://github.com/quixio/quix-streams/blob/main/quixstreams/dataframe/series.py#L378)
1321+
[[VIEW SOURCE]](https://github.com/quixio/quix-streams/blob/main/quixstreams/dataframe/series.py#L379)
13221322

13231323
Check if series value does not refer to the same object as `other`
13241324

@@ -1361,7 +1361,7 @@ new StreamingSeries
13611361
def isnull() -> Self
13621362
```
13631363

1364-
[[VIEW SOURCE]](https://github.com/quixio/quix-streams/blob/main/quixstreams/dataframe/series.py#L402)
1364+
[[VIEW SOURCE]](https://github.com/quixio/quix-streams/blob/main/quixstreams/dataframe/series.py#L403)
13651365

13661366
Check if series value is None.
13671367

@@ -1398,7 +1398,7 @@ new StreamingSeries
13981398
def notnull() -> Self
13991399
```
14001400

1401-
[[VIEW SOURCE]](https://github.com/quixio/quix-streams/blob/main/quixstreams/dataframe/series.py#L425)
1401+
[[VIEW SOURCE]](https://github.com/quixio/quix-streams/blob/main/quixstreams/dataframe/series.py#L426)
14021402

14031403
Check if series value is not None.
14041404

@@ -1435,7 +1435,7 @@ new StreamingSeries
14351435
def abs() -> Self
14361436
```
14371437

1438-
[[VIEW SOURCE]](https://github.com/quixio/quix-streams/blob/main/quixstreams/dataframe/series.py#L448)
1438+
[[VIEW SOURCE]](https://github.com/quixio/quix-streams/blob/main/quixstreams/dataframe/series.py#L449)
14391439

14401440
Get absolute value of the series value.
14411441

0 commit comments

Comments
 (0)