1010class Application ()
1111```
1212
13- [[VIEW SOURCE ]](https :// github .com / quixio / quix - streams / blob / main / quixstreams / app .py # L80 )
13+ [[VIEW SOURCE ]](https :// github .com / quixio / quix - streams / blob / main / quixstreams / app .py # L90 )
1414
1515The main Application class .
1616
@@ -85,7 +85,7 @@ def __init__(broker_address: Optional[Union[str, ConnectionConfig]] = None,
8585 processing_guarantee: ProcessingGuarantee = " at-least-once" )
8686```
8787
88- [[VIEW SOURCE ]](https:// github.com/ quixio/ quix- streams/ blob/ main/ quixstreams/ app.py# L118 )
88+ [[VIEW SOURCE ]](https:// github.com/ quixio/ quix- streams/ blob/ main/ quixstreams/ app.py# L128 )
8989
9090
9191< br>
@@ -174,7 +174,7 @@ instead of the default one.
174174def Quix(cls , * args, ** kwargs)
175175```
176176
177- [[VIEW SOURCE ]](https:// github.com/ quixio/ quix- streams/ blob/ main/ quixstreams/ app.py# L364 )
177+ [[VIEW SOURCE ]](https:// github.com/ quixio/ quix- streams/ blob/ main/ quixstreams/ app.py# L375 )
178178
179179RAISES EXCEPTION : DEPRECATED .
180180
@@ -197,7 +197,7 @@ def topic(name: str,
197197 timestamp_extractor: Optional[TimestampExtractor] = None ) -> Topic
198198```
199199
200- [[VIEW SOURCE ]](https:// github.com/ quixio/ quix- streams/ blob/ main/ quixstreams/ app.py# L396 )
200+ [[VIEW SOURCE ]](https:// github.com/ quixio/ quix- streams/ blob/ main/ quixstreams/ app.py# L407 )
201201
202202Create a topic definition.
203203
@@ -279,7 +279,7 @@ def dataframe(topic: Optional[Topic] = None,
279279 source: Optional[BaseSource] = None ) -> StreamingDataFrame
280280```
281281
282- [[VIEW SOURCE ]](https:// github.com/ quixio/ quix- streams/ blob/ main/ quixstreams/ app.py# L476 )
282+ [[VIEW SOURCE ]](https:// github.com/ quixio/ quix- streams/ blob/ main/ quixstreams/ app.py# L487 )
283283
284284A simple helper method that generates a `StreamingDataFrame` , which is used
285285
@@ -335,7 +335,7 @@ to be used as an input topic.
335335def stop(fail: bool = False )
336336```
337337
338- [[VIEW SOURCE ]](https:// github.com/ quixio/ quix- streams/ blob/ main/ quixstreams/ app.py# L532 )
338+ [[VIEW SOURCE ]](https:// github.com/ quixio/ quix- streams/ blob/ main/ quixstreams/ app.py# L543 )
339339
340340Stop the internal poll loop and the message processing.
341341
@@ -362,7 +362,7 @@ to unhandled exception, and it shouldn't commit the current checkpoint.
362362def get_producer() -> Producer
363363```
364364
365- [[VIEW SOURCE ]](https:// github.com/ quixio/ quix- streams/ blob/ main/ quixstreams/ app.py# L577 )
365+ [[VIEW SOURCE ]](https:// github.com/ quixio/ quix- streams/ blob/ main/ quixstreams/ app.py# L588 )
366366
367367Create and return a pre- configured Producer instance.
368368The Producer is initialized with params passed to Application.
@@ -397,7 +397,7 @@ with app.get_producer() as producer:
397397def get_consumer(auto_commit_enable: bool = True ) -> Consumer
398398```
399399
400- [[VIEW SOURCE ]](https:// github.com/ quixio/ quix- streams/ blob/ main/ quixstreams/ app.py# L632 )
400+ [[VIEW SOURCE ]](https:// github.com/ quixio/ quix- streams/ blob/ main/ quixstreams/ app.py# L643 )
401401
402402Create and return a pre- configured Consumer instance.
403403
@@ -454,7 +454,7 @@ with app.get_consumer() as consumer:
454454def clear_state()
455455```
456456
457- [[VIEW SOURCE ]](https:// github.com/ quixio/ quix- streams/ blob/ main/ quixstreams/ app.py# L682 )
457+ [[VIEW SOURCE ]](https:// github.com/ quixio/ quix- streams/ blob/ main/ quixstreams/ app.py# L693 )
458458
459459Clear the state of the application.
460460
@@ -468,7 +468,7 @@ Clear the state of the application.
468468def add_source(source: BaseSource, topic: Optional[Topic] = None ) -> Topic
469469```
470470
471- [[VIEW SOURCE ]](https:// github.com/ quixio/ quix- streams/ blob/ main/ quixstreams/ app.py# L688 )
471+ [[VIEW SOURCE ]](https:// github.com/ quixio/ quix- streams/ blob/ main/ quixstreams/ app.py# L699 )
472472
473473Add a source to the application.
474474
@@ -492,16 +492,41 @@ Note: the names of default topics are prefixed with "source__".
492492# ### Application.run
493493
494494```python
495- def run(dataframe: Optional[StreamingDataFrame] = None )
495+ def run(dataframe: Optional[StreamingDataFrame] = None ,
496+ timeout: float = 0.0 ,
497+ count: int = 0 )
496498```
497499
498- [[VIEW SOURCE ]](https:// github.com/ quixio/ quix- streams/ blob/ main/ quixstreams/ app.py# L721 )
500+ [[VIEW SOURCE ]](https:// github.com/ quixio/ quix- streams/ blob/ main/ quixstreams/ app.py# L732 )
499501
500502Start processing data from Kafka using provided `StreamingDataFrame`
501503
502504Once started, it can be safely terminated with a `SIGTERM ` signal
503505(like Kubernetes does) or a typical `KeyboardInterrupt ` (`Ctrl+ C` ).
504506
507+ Alternatively, stop conditions can be set (typically for debugging purposes);
508+ has the option of stopping after a number of messages, timeout, or both.
509+
510+ Not setting a timeout or count limit will result in the Application running
511+ indefinitely (expected production behavior).
512+
513+
514+ Stop Condition Details:
515+
516+ A timeout will immediately stop an Application once no new messages have
517+ been consumed after T seconds (after rebalance and recovery).
518+
519+ A count will process N total records from ANY input / SDF topics (so
520+ multiple input topics will very likely differ in their consume total!) after
521+ an initial rebalance and recovery.
522+ THEN , any remaining processes from things such as groupby (which uses internal
523+ topics) will also be validated to ensure the results of said messages are
524+ fully processed (this does NOT count towards the process total).
525+ Note that without a timeout, the Application runs until the count is hit.
526+
527+ If timeout and count are used together (which is the recommended pattern for
528+ debugging), either condition will trigger a stop.
529+
505530
506531
507532< br>
@@ -518,9 +543,19 @@ topic = app.topic('test-topic')
518543df = app.dataframe(topic)
519544df.apply(lambda value , context : print (' New message' , value)
520545
521- app.run()
546+ app.run() # could pass `timeout=5` here, for example
522547```
523548
549+
550+ < br>
551+ ** * Arguments:***
552+
553+ - `dataframe` : DEPRECATED - do not use; sdfs are now automatically tracked.
554+ - `timeout` : maximum time to wait for a new message.
555+ Default = 0.0 (infinite)
556+ - `count` : how many input topic messages to process before stopping.
557+ Default = 0 (infinite)
558+
524559< a id = " quixstreams.app.Application.setup_topics" >< / a>
525560
526561< br>< br>
@@ -531,7 +566,7 @@ app.run()
531566def setup_topics()
532567```
533568
534- [[VIEW SOURCE ]](https:// github.com/ quixio/ quix- streams/ blob/ main/ quixstreams/ app.py# L846 )
569+ [[VIEW SOURCE ]](https:// github.com/ quixio/ quix- streams/ blob/ main/ quixstreams/ app.py# L910 )
535570
536571Validate the application topics
537572
@@ -543,7 +578,7 @@ Validate the application topics
543578class ApplicationConfig(BaseSettings)
544579```
545580
546- [[VIEW SOURCE ]](https:// github.com/ quixio/ quix- streams/ blob/ main/ quixstreams/ app.py# L1014 )
581+ [[VIEW SOURCE ]](https:// github.com/ quixio/ quix- streams/ blob/ main/ quixstreams/ app.py# L1081 )
547582
548583Immutable object holding the application configuration
549584
@@ -566,7 +601,7 @@ def settings_customise_sources(
566601) -> Tuple[PydanticBaseSettingsSource, ... ]
567602```
568603
569- [[VIEW SOURCE ]](https:// github.com/ quixio/ quix- streams/ blob/ main/ quixstreams/ app.py# L1049 )
604+ [[VIEW SOURCE ]](https:// github.com/ quixio/ quix- streams/ blob/ main/ quixstreams/ app.py# L1116 )
570605
571606Included to ignore reading/ setting values from the environment
572607
@@ -580,7 +615,7 @@ Included to ignore reading/setting values from the environment
580615def copy(** kwargs) -> Self
581616```
582617
583- [[VIEW SOURCE ]](https:// github.com/ quixio/ quix- streams/ blob/ main/ quixstreams/ app.py# L1062 )
618+ [[VIEW SOURCE ]](https:// github.com/ quixio/ quix- streams/ blob/ main/ quixstreams/ app.py# L1129 )
584619
585620Update the application config and return a copy
586621
0 commit comments