Skip to content

Commit 8d134c4

Browse files
github-actions[bot]daniil-quix
github-actions[bot]
and
daniil-quix
authored
Update documentation (#852)
Co-authored-by: daniil-quix <133032822+daniil-quix@users.noreply.github.com>
1 parent c4362dc commit 8d134c4

File tree

2 files changed

+46
-32
lines changed

2 files changed

+46
-32
lines changed

‎docs/api-reference/quixstreams.md

Lines changed: 23 additions & 16 deletions
Original file line numberDiff line numberDiff line change
@@ -7420,7 +7420,7 @@ a list of (key, value) tuples.
74207420
class TopicConfig()
74217421
```
74227422

7423-
[[VIEW SOURCE]](https://github.com/quixio/quix-streams/blob/main/quixstreams/models/topics/topic.py#L44)
7423+
[[VIEW SOURCE]](https://github.com/quixio/quix-streams/blob/main/quixstreams/models/topics/topic.py#L45)
74247424

74257425
Represents all kafka-level configuration for a kafka topic.
74267426

@@ -7434,7 +7434,7 @@ Generally used by Topic and any topic creation procedures.
74347434
class Topic()
74357435
```
74367436

7437-
[[VIEW SOURCE]](https://github.com/quixio/quix-streams/blob/main/quixstreams/models/topics/topic.py#L85)
7437+
[[VIEW SOURCE]](https://github.com/quixio/quix-streams/blob/main/quixstreams/models/topics/topic.py#L92)
74387438

74397439
A definition of a Kafka topic.
74407440

@@ -7449,6 +7449,7 @@ instance.
74497449
```python
74507450
def __init__(
74517451
name: str,
7452+
topic_type: TopicType = TopicType.REGULAR,
74527453
create_config: Optional[TopicConfig] = None,
74537454
value_deserializer: Optional[DeserializerType] = None,
74547455
key_deserializer: Optional[DeserializerType] = BytesDeserializer(),
@@ -7458,11 +7459,17 @@ def __init__(
74587459
quix_name: str = "")
74597460
```
74607461

7461-
[[VIEW SOURCE]](https://github.com/quixio/quix-streams/blob/main/quixstreams/models/topics/topic.py#L94)
7462+
[[VIEW SOURCE]](https://github.com/quixio/quix-streams/blob/main/quixstreams/models/topics/topic.py#L101)
74627463

74637464
**Arguments**:
74647465

74657466
- `name`: topic name
7467+
- `topic_type`: a type of the topic, can be one of:
7468+
- `TopicType.REGULAR` - the regular input and output topics
7469+
- `TopicType.REPARTITION` - a repartition topic used for re-keying the data
7470+
- `TopicType.CHANGELOG` - a changelog topic to back up the state stores.
7471+
7472+
Default - `TopicType.REGULAR`.
74667473
- `create_config`: a `TopicConfig` to create a new topic if it does not exist
74677474
- `value_deserializer`: a deserializer type for values
74687475
- `key_deserializer`: a deserializer type for keys
@@ -7482,7 +7489,7 @@ It is set only by `QuixTopicManager`.
74827489
def create_config() -> Optional[TopicConfig]
74837490
```
74847491

7485-
[[VIEW SOURCE]](https://github.com/quixio/quix-streams/blob/main/quixstreams/models/topics/topic.py#L144)
7492+
[[VIEW SOURCE]](https://github.com/quixio/quix-streams/blob/main/quixstreams/models/topics/topic.py#L161)
74867493

74877494
A config to create the topic
74887495

@@ -7495,7 +7502,7 @@ A config to create the topic
74957502
def broker_config() -> TopicConfig
74967503
```
74977504

7498-
[[VIEW SOURCE]](https://github.com/quixio/quix-streams/blob/main/quixstreams/models/topics/topic.py#L155)
7505+
[[VIEW SOURCE]](https://github.com/quixio/quix-streams/blob/main/quixstreams/models/topics/topic.py#L172)
74997506

75007507
A topic config obtained from the Kafka broker
75017508

@@ -7507,7 +7514,7 @@ A topic config obtained from the Kafka broker
75077514
def row_serialize(row: Row, key: Any) -> KafkaMessage
75087515
```
75097516

7510-
[[VIEW SOURCE]](https://github.com/quixio/quix-streams/blob/main/quixstreams/models/topics/topic.py#L169)
7517+
[[VIEW SOURCE]](https://github.com/quixio/quix-streams/blob/main/quixstreams/models/topics/topic.py#L198)
75117518

75127519
Serialize Row to a Kafka message structure
75137520

@@ -7530,7 +7537,7 @@ def row_deserialize(
75307537
) -> Union[Row, List[Row], None]
75317538
```
75327539

7533-
[[VIEW SOURCE]](https://github.com/quixio/quix-streams/blob/main/quixstreams/models/topics/topic.py#L209)
7540+
[[VIEW SOURCE]](https://github.com/quixio/quix-streams/blob/main/quixstreams/models/topics/topic.py#L238)
75347541

75357542
Deserialize incoming Kafka message to a Row.
75367543

@@ -7596,7 +7603,7 @@ def __init__(topic_admin: TopicAdmin,
75967603
def changelog_topics() -> Dict[Optional[str], Dict[str, Topic]]
75977604
```
75987605

7599-
[[VIEW SOURCE]](https://github.com/quixio/quix-streams/blob/main/quixstreams/models/topics/manager.py#L83)
7606+
[[VIEW SOURCE]](https://github.com/quixio/quix-streams/blob/main/quixstreams/models/topics/manager.py#L75)
76007607

76017608
Note: `Topic`s are the changelogs.
76027609

@@ -7611,7 +7618,7 @@ returns: the changelog topic dict, {topic_name: {suffix: Topic}}
76117618
def changelog_topics_list() -> List[Topic]
76127619
```
76137620

7614-
[[VIEW SOURCE]](https://github.com/quixio/quix-streams/blob/main/quixstreams/models/topics/manager.py#L92)
7621+
[[VIEW SOURCE]](https://github.com/quixio/quix-streams/blob/main/quixstreams/models/topics/manager.py#L84)
76157622

76167623
Returns a list of changelog topics
76177624

@@ -7626,7 +7633,7 @@ returns: the changelog topic dict, {topic_name: {suffix: Topic}}
76267633
def non_changelog_topics() -> Dict[str, Topic]
76277634
```
76287635

7629-
[[VIEW SOURCE]](https://github.com/quixio/quix-streams/blob/main/quixstreams/models/topics/manager.py#L101)
7636+
[[VIEW SOURCE]](https://github.com/quixio/quix-streams/blob/main/quixstreams/models/topics/manager.py#L93)
76307637

76317638
Returns a dict with normal and repartition topics
76327639

@@ -7639,7 +7646,7 @@ Returns a dict with normal and repartition topics
76397646
def all_topics() -> Dict[str, Topic]
76407647
```
76417648

7642-
[[VIEW SOURCE]](https://github.com/quixio/quix-streams/blob/main/quixstreams/models/topics/manager.py#L108)
7649+
[[VIEW SOURCE]](https://github.com/quixio/quix-streams/blob/main/quixstreams/models/topics/manager.py#L100)
76437650

76447651
Every registered topic name mapped to its respective `Topic`.
76457652

@@ -7655,7 +7662,7 @@ def topic_config(num_partitions: Optional[int] = None,
76557662
extra_config: Optional[dict] = None) -> TopicConfig
76567663
```
76577664

7658-
[[VIEW SOURCE]](https://github.com/quixio/quix-streams/blob/main/quixstreams/models/topics/manager.py#L116)
7665+
[[VIEW SOURCE]](https://github.com/quixio/quix-streams/blob/main/quixstreams/models/topics/manager.py#L113)
76597666

76607667
Convenience method for generating a `TopicConfig` with default settings
76617668

@@ -7683,7 +7690,7 @@ def topic(name: str,
76837690
timestamp_extractor: Optional[TimestampExtractor] = None) -> Topic
76847691
```
76857692

7686-
[[VIEW SOURCE]](https://github.com/quixio/quix-streams/blob/main/quixstreams/models/topics/manager.py#L138)
7693+
[[VIEW SOURCE]](https://github.com/quixio/quix-streams/blob/main/quixstreams/models/topics/manager.py#L135)
76877694

76887695
A convenience method for generating a `Topic`. Will use default config options
76897696

@@ -7712,7 +7719,7 @@ Topic object with creation configs
77127719
def register(topic: Topic) -> Topic
77137720
```
77147721

7715-
[[VIEW SOURCE]](https://github.com/quixio/quix-streams/blob/main/quixstreams/models/topics/manager.py#L184)
7722+
[[VIEW SOURCE]](https://github.com/quixio/quix-streams/blob/main/quixstreams/models/topics/manager.py#L182)
77167723

77177724
Register an already generated :class:`quixstreams.models.topics.Topic` to the topic manager.
77187725

@@ -7737,7 +7744,7 @@ def repartition_topic(
77377744
key_serializer: Optional[SerializerType] = "json") -> Topic
77387745
```
77397746

7740-
[[VIEW SOURCE]](https://github.com/quixio/quix-streams/blob/main/quixstreams/models/topics/manager.py#L203)
7747+
[[VIEW SOURCE]](https://github.com/quixio/quix-streams/blob/main/quixstreams/models/topics/manager.py#L201)
77417748

77427749
Create an internal repartition topic.
77437750

@@ -7764,7 +7771,7 @@ def changelog_topic(stream_id: Optional[str], store_name: str,
77647771
config: TopicConfig) -> Topic
77657772
```
77667773

7767-
[[VIEW SOURCE]](https://github.com/quixio/quix-streams/blob/main/quixstreams/models/topics/manager.py#L239)
7774+
[[VIEW SOURCE]](https://github.com/quixio/quix-streams/blob/main/quixstreams/models/topics/manager.py#L238)
77687775

77697776
Create and register a changelog topic for the given "stream_id" and store name.
77707777

‎docs/api-reference/topics.md

Lines changed: 23 additions & 16 deletions
Original file line numberDiff line numberDiff line change
@@ -142,7 +142,7 @@ fail (it ignores issues for a topic already existing).
142142
class TopicConfig()
143143
```
144144

145-
[[VIEW SOURCE]](https://github.com/quixio/quix-streams/blob/main/quixstreams/models/topics/topic.py#L44)
145+
[[VIEW SOURCE]](https://github.com/quixio/quix-streams/blob/main/quixstreams/models/topics/topic.py#L45)
146146

147147
Represents all kafka-level configuration for a kafka topic.
148148

@@ -156,7 +156,7 @@ Generally used by Topic and any topic creation procedures.
156156
class Topic()
157157
```
158158

159-
[[VIEW SOURCE]](https://github.com/quixio/quix-streams/blob/main/quixstreams/models/topics/topic.py#L85)
159+
[[VIEW SOURCE]](https://github.com/quixio/quix-streams/blob/main/quixstreams/models/topics/topic.py#L92)
160160

161161
A definition of a Kafka topic.
162162

@@ -173,6 +173,7 @@ instance.
173173
```python
174174
def __init__(
175175
name: str,
176+
topic_type: TopicType = TopicType.REGULAR,
176177
create_config: Optional[TopicConfig] = None,
177178
value_deserializer: Optional[DeserializerType] = None,
178179
key_deserializer: Optional[DeserializerType] = BytesDeserializer(),
@@ -182,13 +183,19 @@ def __init__(
182183
quix_name: str = "")
183184
```
184185

185-
[[VIEW SOURCE]](https://github.com/quixio/quix-streams/blob/main/quixstreams/models/topics/topic.py#L94)
186+
[[VIEW SOURCE]](https://github.com/quixio/quix-streams/blob/main/quixstreams/models/topics/topic.py#L101)
186187

187188

188189
<br>
189190
***Arguments:***
190191

191192
- `name`: topic name
193+
- `topic_type`: a type of the topic, can be one of:
194+
- `TopicType.REGULAR` - the regular input and output topics
195+
- `TopicType.REPARTITION` - a repartition topic used for re-keying the data
196+
- `TopicType.CHANGELOG` - a changelog topic to back up the state stores.
197+
198+
Default - `TopicType.REGULAR`.
192199
- `create_config`: a `TopicConfig` to create a new topic if it does not exist
193200
- `value_deserializer`: a deserializer type for values
194201
- `key_deserializer`: a deserializer type for keys
@@ -210,7 +217,7 @@ It is set only by `QuixTopicManager`.
210217
def create_config() -> Optional[TopicConfig]
211218
```
212219

213-
[[VIEW SOURCE]](https://github.com/quixio/quix-streams/blob/main/quixstreams/models/topics/topic.py#L144)
220+
[[VIEW SOURCE]](https://github.com/quixio/quix-streams/blob/main/quixstreams/models/topics/topic.py#L161)
214221

215222
A config to create the topic
216223

@@ -225,7 +232,7 @@ A config to create the topic
225232
def broker_config() -> TopicConfig
226233
```
227234

228-
[[VIEW SOURCE]](https://github.com/quixio/quix-streams/blob/main/quixstreams/models/topics/topic.py#L155)
235+
[[VIEW SOURCE]](https://github.com/quixio/quix-streams/blob/main/quixstreams/models/topics/topic.py#L172)
229236

230237
A topic config obtained from the Kafka broker
231238

@@ -239,7 +246,7 @@ A topic config obtained from the Kafka broker
239246
def row_serialize(row: Row, key: Any) -> KafkaMessage
240247
```
241248

242-
[[VIEW SOURCE]](https://github.com/quixio/quix-streams/blob/main/quixstreams/models/topics/topic.py#L169)
249+
[[VIEW SOURCE]](https://github.com/quixio/quix-streams/blob/main/quixstreams/models/topics/topic.py#L198)
243250

244251
Serialize Row to a Kafka message structure
245252

@@ -268,7 +275,7 @@ def row_deserialize(
268275
) -> Union[Row, List[Row], None]
269276
```
270277

271-
[[VIEW SOURCE]](https://github.com/quixio/quix-streams/blob/main/quixstreams/models/topics/topic.py#L209)
278+
[[VIEW SOURCE]](https://github.com/quixio/quix-streams/blob/main/quixstreams/models/topics/topic.py#L238)
272279

273280
Deserialize incoming Kafka message to a Row.
274281

@@ -340,7 +347,7 @@ def __init__(topic_admin: TopicAdmin,
340347
def changelog_topics() -> Dict[Optional[str], Dict[str, Topic]]
341348
```
342349

343-
[[VIEW SOURCE]](https://github.com/quixio/quix-streams/blob/main/quixstreams/models/topics/manager.py#L83)
350+
[[VIEW SOURCE]](https://github.com/quixio/quix-streams/blob/main/quixstreams/models/topics/manager.py#L75)
344351

345352
Note: `Topic`s are the changelogs.
346353

@@ -357,7 +364,7 @@ returns: the changelog topic dict, {topic_name: {suffix: Topic}}
357364
def changelog_topics_list() -> List[Topic]
358365
```
359366

360-
[[VIEW SOURCE]](https://github.com/quixio/quix-streams/blob/main/quixstreams/models/topics/manager.py#L92)
367+
[[VIEW SOURCE]](https://github.com/quixio/quix-streams/blob/main/quixstreams/models/topics/manager.py#L84)
361368

362369
Returns a list of changelog topics
363370

@@ -374,7 +381,7 @@ returns: the changelog topic dict, {topic_name: {suffix: Topic}}
374381
def non_changelog_topics() -> Dict[str, Topic]
375382
```
376383

377-
[[VIEW SOURCE]](https://github.com/quixio/quix-streams/blob/main/quixstreams/models/topics/manager.py#L101)
384+
[[VIEW SOURCE]](https://github.com/quixio/quix-streams/blob/main/quixstreams/models/topics/manager.py#L93)
378385

379386
Returns a dict with normal and repartition topics
380387

@@ -389,7 +396,7 @@ Returns a dict with normal and repartition topics
389396
def all_topics() -> Dict[str, Topic]
390397
```
391398

392-
[[VIEW SOURCE]](https://github.com/quixio/quix-streams/blob/main/quixstreams/models/topics/manager.py#L108)
399+
[[VIEW SOURCE]](https://github.com/quixio/quix-streams/blob/main/quixstreams/models/topics/manager.py#L100)
393400

394401
Every registered topic name mapped to its respective `Topic`.
395402

@@ -407,7 +414,7 @@ def topic_config(num_partitions: Optional[int] = None,
407414
extra_config: Optional[dict] = None) -> TopicConfig
408415
```
409416

410-
[[VIEW SOURCE]](https://github.com/quixio/quix-streams/blob/main/quixstreams/models/topics/manager.py#L116)
417+
[[VIEW SOURCE]](https://github.com/quixio/quix-streams/blob/main/quixstreams/models/topics/manager.py#L113)
411418

412419
Convenience method for generating a `TopicConfig` with default settings
413420

@@ -441,7 +448,7 @@ def topic(name: str,
441448
timestamp_extractor: Optional[TimestampExtractor] = None) -> Topic
442449
```
443450

444-
[[VIEW SOURCE]](https://github.com/quixio/quix-streams/blob/main/quixstreams/models/topics/manager.py#L138)
451+
[[VIEW SOURCE]](https://github.com/quixio/quix-streams/blob/main/quixstreams/models/topics/manager.py#L135)
445452

446453
A convenience method for generating a `Topic`. Will use default config options
447454

@@ -476,7 +483,7 @@ Topic object with creation configs
476483
def register(topic: Topic) -> Topic
477484
```
478485

479-
[[VIEW SOURCE]](https://github.com/quixio/quix-streams/blob/main/quixstreams/models/topics/manager.py#L184)
486+
[[VIEW SOURCE]](https://github.com/quixio/quix-streams/blob/main/quixstreams/models/topics/manager.py#L182)
480487

481488
Register an already generated :class:`quixstreams.models.topics.Topic` to the topic manager.
482489

@@ -505,7 +512,7 @@ def repartition_topic(
505512
key_serializer: Optional[SerializerType] = "json") -> Topic
506513
```
507514

508-
[[VIEW SOURCE]](https://github.com/quixio/quix-streams/blob/main/quixstreams/models/topics/manager.py#L203)
515+
[[VIEW SOURCE]](https://github.com/quixio/quix-streams/blob/main/quixstreams/models/topics/manager.py#L201)
509516

510517
Create an internal repartition topic.
511518

@@ -538,7 +545,7 @@ def changelog_topic(stream_id: Optional[str], store_name: str,
538545
config: TopicConfig) -> Topic
539546
```
540547

541-
[[VIEW SOURCE]](https://github.com/quixio/quix-streams/blob/main/quixstreams/models/topics/manager.py#L239)
548+
[[VIEW SOURCE]](https://github.com/quixio/quix-streams/blob/main/quixstreams/models/topics/manager.py#L238)
542549

543550
Create and register a changelog topic for the given "stream_id" and store name.
544551

0 commit comments

Comments
 (0)