Skip to content

Commit a6fb0fe

Browse files
authored
tweak Source to have a defined setup method for easier simple usecases (#783)
1 parent be32e53 commit a6fb0fe

File tree

7 files changed

+15
-23
lines changed

7 files changed

+15
-23
lines changed

docs/connectors/sources/README.md

Lines changed: 5 additions & 5 deletions
Original file line numberDiff line numberDiff line change
@@ -136,10 +136,10 @@ if __name__ == "__main__":
136136

137137
## Connection Callbacks
138138

139-
Assuming a given connector is set up to utilize them, there are two callbacks that can
140-
be set for when a client connects/authenticates successfully or not, named
141-
`on_client_connect_success` and `on_client_connect_failure`, respectfully.
139+
Assuming a given connector has a proper `setup` method to utilize them, there are two
140+
callbacks that can be set for when a client connects/authenticates successfully or not,
141+
named `on_client_connect_success` and `on_client_connect_failure`, respectfully.
142142

143-
Though having a `setup` method is required, it is not guaranteed that it is implemented or
144-
utilized fully with `community` sources; you can inspect a given Source's `setup` method
143+
Though implementing an applicable `setup` method is strongly encouraged, it is not an
144+
enforced guarantee with `community` sources; you can inspect a given Source's `setup` method
145145
to confirm whether it tests the client connection there (and that the callbacks are then applicable).

docs/connectors/sources/custom-sources.md

Lines changed: 3 additions & 6 deletions
Original file line numberDiff line numberDiff line change
@@ -13,19 +13,16 @@ The recommended parent class to create a new source. It handles configuring, sta
1313
To get started, implement the [`run`](../../api-reference/sources.md#sourcerun) method
1414
which loops while `self.running` is `True` (or until it's done).
1515

16-
When you also have a client pattern, it is recommended to use the required `setup`
17-
method to establish the initial connection/authentication so that the built-in callbacks
18-
of `on_client_connect_success` and `on_client_connect_failure` can be utilized.
19-
Otherwise, just set it to return.
16+
>**NOTE**: With client-based sources, it is recommended to also implement the `setup`
17+
method for establishing initial connection/authentication so that the built-in callbacks
18+
of `on_client_connect_success` and `on_client_connect_failure` can be utilized.
2019

2120
Example subclass:
2221

2322
```python
2423
from quixstreams.sources.base import Source
2524

2625
class MySource(Source):
27-
def setup(self):
28-
return
2926

3027
def run(self):
3128
with open("file.txt", "r") as f:

docs/tutorials/anomaly-detection/tutorial_app.py

Lines changed: 0 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -50,9 +50,6 @@ def __init__(self):
5050
}
5151
super().__init__(name="temperature-event-generator")
5252

53-
def setup(self):
54-
return
55-
5653
def update_machine_temp(self, machine_id):
5754
"""
5855
Updates the temperature for a machine by -1, 0, or 1 based on its current temp.

docs/tutorials/purchase-filtering/tutorial_app.py

Lines changed: 0 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -66,9 +66,6 @@ class PurchaseGenerator(Source):
6666
def __init__(self):
6767
super().__init__(name="customer-purchases")
6868

69-
def setup(self):
70-
return
71-
7269
def run(self):
7370
for cid, purchase_info in enumerate(self._purchases_data):
7471
event = self.serialize(key=f"CUSTOMER_ID{cid}", value=purchase_info)

docs/tutorials/websocket-source/tutorial_app.py

Lines changed: 0 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -36,9 +36,6 @@ def __init__(
3636
self._url = url
3737
self._product_ids = product_ids
3838

39-
def setup(self):
40-
return
41-
4239
def run(self) -> None:
4340
"""
4441
The main method of the source with the main logic.

docs/tutorials/word-count/tutorial_app.py

Lines changed: 0 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -27,9 +27,6 @@ class ReviewGenerator(Source):
2727
def __init__(self):
2828
super().__init__(name="customer-reviews")
2929

30-
def setup(self):
31-
return
32-
3330
def run(self):
3431
for review in self._review_list:
3532
event = self.serialize(key=choice(self._product_list), value=review)

quixstreams/sources/base/source.py

Lines changed: 7 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -256,6 +256,13 @@ def running(self) -> bool:
256256
"""
257257
return self._running
258258

259+
def setup(self):
260+
"""
261+
Though implemented for the sake of making simple sources easier to use,
262+
it is still recommended to override this when using a client-based pattern.
263+
"""
264+
return
265+
259266
def cleanup(self, failed: bool) -> None:
260267
"""
261268
This method is triggered once the `run` method completes.

0 commit comments

Comments
 (0)