New README (#401)

Author: daniil-quix

README.md

@@ -1,33 +1,28 @@
-![Quix - React to data, fast](https://github.com/quixio/quix-streams/blob/main/images/quixstreams-banner.jpg)
+![Quix - React to data, fast](./images/quixstreams-banner.png)
 
-[![Docs](https://img.shields.io/badge/-Docs-red?logo=read-the-docs)](https://quix.io/docs/quix-streams/introduction.html)
+ [![GitHub Version](https://img.shields.io/github/tag-pre/quixio/quix-streams.svg?label=Version&color=008dff)](https://github.com/quixio/quix-streams/releases)
+![PyPI License](https://img.shields.io/pypi/l/quixstreams?label=Licence&color=008dff)
+[![Docs](https://img.shields.io/badge/docs-quix.io-0345b2?label=Docs&color=008dff)](https://quix.io/docs/quix-streams/introduction.html) \
 [![Community Slack](https://img.shields.io/badge/Community%20Slack-blueviolet?logo=slack)](https://quix.io/slack-invite)
-[![Linkedin](https://img.shields.io/badge/LinkedIn-0A66C2.svg?logo=linkedin)](https://www.linkedin.com/company/70925173/)
-[![Quix on Twitter](https://img.shields.io/twitter/url?label=Twitter&style=social&url=https%3A%2F%2Ftwitter.com%2Fquix_io)](https://twitter.com/quix_io)
+[![YouTube](https://img.shields.io/badge/-YouTube-FF0000?logo=youtube)](https://www.youtube.com/@QuixStreams)
+[![LinkedIn](https://img.shields.io/badge/LinkedIn-0A66C2.svg?logo=linkedin)](https://www.linkedin.com/company/70925173/)
+[![X](https://img.shields.io/twitter/url?label=X&style=social&url=https%3A%2F%2Ftwitter.com%2Fquix_io)](https://twitter.com/quix_io)
 
-# Quix Streams
+# 100% Python Stream Processing for Kafka
 
-Quix Streams is a cloud native library for processing data in Kafka using pure Python. It’s designed to give you the power of a distributed system in a lightweight library by combining the low-level scalability and resiliency features of Kafka with an easy to use Python interface (to ease newcomers to stream processing).
+Quix Streams is a cloud-native library for processing data in Kafka using pure Python. It’s designed to give you the power of a distributed system in a lightweight library by combining Kafka's low-level scalability and resiliency features with an easy-to-use Python interface (to ease newcomers to stream processing). 
 
-Quix Streams has the following benefits:
-
-- Pure Python (no JVM, no wrappers, no cross-language debugging). 
-- No orchestrator, no server-side engine.
+It has the following benefits:
 - Streaming DataFrame API (similar to pandas DataFrame) for tabular data transformations.
-- Easily integrates with the entire Python ecosystem (pandas, scikit-learn, TensorFlow, PyTorch etc).
-- Support for many serialization formats, including JSON (and Quix-specific).
-- Support for stateful operations using RocksDB.
-- Support for aggregations over tumbling and hopping time windows.
-- "At-least-once" and "exactly-once" Kafka processing guarantees.
-- Designed to run and scale resiliently via container orchestration (like Kubernetes).
-- Easily runs locally and in Jupyter Notebook for convenient development and debugging.
-- Seamless integration with the fully managed [Quix Cloud](https://quix.io/product) platform.
-
-Use Quix Streams to build event-driven, machine learning/AI or physics-based applications that depend on real-time data from Kafka.
+- Custom stateful operations via a state object.
+- Custom reducing and aggregating over tumbling and hopping time windows.
+- Exactly-once processing semantics via Kafka transactions.
+- Pure Python with no need for a server-side engine.  
 
+Use Quix Streams to build simple Kafka producer/consumer applications or leverage stream processing to build complex event-driven systems, real-time data pipelines and AI/ML products.
 
-## Getting started 🏄
 
+## Getting Started 🏄
 
 ### Install Quix Streams
 
@@ -40,90 +35,44 @@ Python 3.8+, Apache Kafka 0.10+
 
 See [requirements.txt](https://github.com/quixio/quix-streams/blob/main/requirements.txt) for the full list of requirements
 
-## Documentation
+### Documentation
 [Quix Streams Docs](https://quix.io/docs/quix-streams/introduction.html)
 
-### Example Application
+### Example
 
 Here's an example of how to <b>process</b> data from a Kafka Topic with Quix Streams:
 
 ```python
-from quixstreams import Application, State
+from quixstreams import Application
 
-# Define an application
+# A minimal application reading temperature data in Celsius from the Kafka topic,
+# converting it to Fahrenheit and producing alerts to another topic.
+
+# Define an application that will connect to Kafka
 app = Application(
     broker_address="localhost:9092",  # Kafka broker address
-    consumer_group="consumer-group-name",  # Kafka consumer group
 )
 
-# Define the input and output topics. By default, "json" serialization will be used
-input_topic = app.topic("my_input_topic")
-output_topic = app.topic("my_output_topic")
-
-
-def count(data: dict, state: State):
-    # Get a value from state for the current Kafka message key
-    total = state.get('total', default=0)
-    total += 1
-    # Set a value back to the state
-    state.set('total', total)
-    # Update your message data with a value from the state
-    data['total'] = total
-
-
-# Create a StreamingDataFrame instance
-# StreamingDataFrame is a primary interface to define the message processing pipeline
-sdf = app.dataframe(topic=input_topic)
-
-# Print the incoming messages
-sdf = sdf.update(lambda value: print('Received a message:', value))
+# Define the Kafka topics
+temperature_topic = app.topic("temperature-celsius", value_deserializer="json")
+alerts_topic = app.topic("temperature-alerts", value_serializer="json")
 
-# Select fields from incoming messages
-sdf = sdf[["field_1", "field_2", "field_3"]]
+# Create a Streaming DataFrame connected to the input Kafka topic
+sdf = app.dataframe(topic=temperature_topic)
 
-# Filter only messages with "field_0" > 10 and "field_2" != "test"
-sdf = sdf[(sdf["field_1"] > 10) & (sdf["field_2"] != "test")]
+# Convert temperature to Fahrenheit by transforming the input message (with an anonymous or user-defined function)
+sdf = sdf.apply(lambda value: {"temperature_F": (value["temperature"] * 9/5) + 32})
 
-# Filter messages using custom functions
-sdf = sdf[sdf.apply(lambda value: 0 < (value['field_1'] + value['field_3']) < 1000)]
+# Filter values above the threshold
+sdf = sdf[sdf["temperature_F"] > 150]
 
-# Generate a new value based on the current one
-sdf = sdf.apply(lambda value: {**value, 'new_field': 'new_value'})
-
-# Update a value based on the entire message content
-sdf['field_4'] = sdf.apply(lambda value: value['field_1'] + value['field_3'])
-
-# Use a stateful function to persist data to the state store and update the value in place
-sdf = sdf.update(count, stateful=True)
-
-# Print the result before producing it
-sdf = sdf.update(lambda value, ctx: print('Producing a message:', value))
-
-# Produce the result to the output topic 
-sdf = sdf.to_topic(output_topic)
-
-if __name__ == "__main__":
-    # Run the streaming application 
-    app.run(sdf)
+# Produce alerts to the output topic
+sdf = sdf.to_topic(alerts_topic)
 
+# Run the streaming application 
+app.run(sdf)
 ```
 
-
-### How It Works
-There are two primary components:
-- `StreamingDataFrame` - a predefined declarative pipeline to process and transform incoming messages.
-- `Application` - to manage the Kafka-related setup & teardown and message lifecycle (consuming, committing). It processes each message with the dataframe you provide it.
-
-Under the hood, the `Application` will:
-- Consume a message.
-- Deserialize it.
-- Process it with your `StreamingDataFrame`.
-- Produce it to the output topic.
-- Automatically commit the topic offset and state updates after the message is processed.
-- React to Kafka rebalancing updates and manage the topic partitions.
-- Manage the State store.
-- Handle OS signals and gracefully exit the application.
-
 ### Tutorials
 
 To see Quix Streams in action, check out the Quickstart and Tutorials in the docs: 
@@ -134,64 +83,54 @@ To see Quix Streams in action, check out the Quickstart and Tutorials in the doc
 - [**Tutorial - Purchase Filtering**](https://quix.io/docs/quix-streams/tutorials/purchase-filtering/tutorial.html)
 
 
-### Using the [Quix Cloud](https://quix.io/)
+### Key Concepts
+There are two primary objects:
+- `StreamingDataFrame` - a predefined declarative pipeline to process and transform incoming messages.
+- `Application` - to manage the Kafka-related setup, teardown and message lifecycle (consuming, committing). It processes each message with the dataframe you provide for it to run.
+
+Under the hood, the `Application` will:
+- Consume and deserialize messages.
+- Process them with your `StreamingDataFrame`.
+- Produce it to the output topic.
+- Automatically checkpoint processed messages and state for resiliency.
+- Scale using Kafka's built-in consumer groups mechanism.
 
-This library doesn't have any dependency on any commercial products, but if you use it together with Quix Cloud you will get some advantages out of the box during your development process such as:
-- Auto-configuration.
-- Monitoring.
-- Data explorer.
-- Data persistence.
-- Pipeline visualization.
-- Metrics.
 
-and more.
+### Deployment
+You can run Quix Streams pipelines anywhere Python is installed.
 
-Quix Streams provides a seamless integration with Quix Cloud, and it can automatically configure the `Application` using Quix SDK Token.
+Deploy to your own infrastructure or to [Quix Cloud](https://quix.io/product) on AWS, Azure, GCP or on-premise for a fully managed platform.  
+You'll get self-service DevOps, CI/CD and monitoring, all built with best in class engineering practices learned from Formula 1 Racing.
 
 Please see the [**Connecting to Quix Cloud**](https://quix.io/docs/quix-streams/quix-platform.html) page 
 to learn how to use Quix Streams and Quix Cloud together.
 
-### What's Next
+## Roadmap 📍
 
-This library is being actively developed. 
+This library is being actively developed by a full-time team.
 
 Here are some of the planned improvements:
 
 - [x] [Windowed aggregations over Tumbling & Hopping windows](https://quix.io/docs/quix-streams/v2-0-latest/windowing.html)
-- [x] [State recovery based on Kafka changelog topics](https://quix.io/docs/quix-streams/advanced/stateful-processing.html#fault-tolerance-recovery)
+- [x] [Stateful operations and recovery based on Kafka changelog topics](https://quix.io/docs/quix-streams/advanced/stateful-processing.html)
 - [x] [Group-by operation](https://quix.io/docs/quix-streams/groupby.html)
-- [X] ["Exactly Once" delivery guarantees for Kafka message processing (AKA transactions)](https://quix.io/docs/quix-streams/configuration.html#processing-guarantees)
+- [x] ["Exactly Once" delivery guarantees for Kafka message processing (AKA transactions)](https://quix.io/docs/quix-streams/configuration.html#processing-guarantees)
 - [ ] Joins
 - [ ] Windowed aggregations over Sliding windows
 - [ ] Support for Avro and Protobuf formats
 - [ ] Schema Registry support
 
 
-To find out when the next version is ready, make sure you watch this repo 
-and join our [Quix Community on Slack](https://quix.io/slack-invite)! 
-
-## Contribution Guide
-
-Contributing is a great way to learn and we especially welcome those who haven't contributed to an OSS project before.
-<br>
-We're very open to any feedback or code contributions to this OSS project ❤️. 
-
-Before contributing, please read our [Contributing](https://github.com/quixio/quix-streams/blob/main/CONTRIBUTING.md) file for how you can best give feedback and contribute. 
-
-## Need help?
-
-If you run into any problems, please create an [issue](https://github.com/quixio/quix-streams/issues) or ask in #quix-help in our [Quix Community on Slack](https://quix.io/slack-invite).  
-
-## Community 👭
-
-Join the [Quix Community on Slack](https://quix.io/slack-invite), a vibrant group of Python developers, data enthusiasts and newcomers to Apache Kafka, who are learning and leveraging Quix Streams for real-time data processing.
-
-## License
+## Get Involved 🤝
 
-Quix Streams is licensed under the Apache 2.0 license. View a copy of the License file [here](https://github.com/quixio/quix-streams/blob/main/LICENSE).
+- Please use [GitHub issues](https://github.com/quixio/quix-streams/issues) to report bugs and suggest new features.
+- Join the [Quix Community on Slack](https://quix.io/slack-invite), a vibrant group of Kafka Python developers, data engineers and newcomers to Apache Kafka, who are learning and leveraging Quix Streams for real-time data processing.
+- Watch and subscribe to [@QuixStreams on YouTube](https://www.youtube.com/@QuixStreams) for code-along tutorials from scratch and interesting community highlights.
+- Follow us on [X](https://x.com/Quix_io) and [LinkedIn](https://www.linkedin.com/company/70925173) where we share our latest tutorials, forthcoming community events and the occasional meme.
+- If you have any questions or feedback - write to us at support@quix.io!
 
-## Stay in touch 👋
 
-You can follow us on [Twitter](https://twitter.com/quix_io) and [Linkedin](https://www.linkedin.com/company/70925173) where we share our latest tutorials, forthcoming community events and the occasional meme.  
+## License 📗
 
-If you have any questions or feedback - write to us at support@quix.io!
+Quix Streams is licensed under the Apache 2.0 license.  
+View a copy of the License file [here](https://github.com/quixio/quix-streams/blob/main/LICENSE).