You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
DataSQRL is a data streaming framework for incremental and real-time data processing applications. Ingest data from various sources, integrate, transform, store, and serve the result as data APIs, LLM tooling, or iceberg views - with the simplicity of SQL.
10
+
DataSQRL is a development framework for incremental and real-time data processing applications. Ingest data from various sources, integrate, transform, store, and serve the result as data APIs, LLM tooling, or iceberg views - with the simplicity of SQL.
13
11
14
12
Data Engineers use DataSQRL to quickly build production-ready data pipelines that:
15
13
* Create realtime **data APIs**,
16
-
*Expose enriched data for**LLM tooling**,
17
-
*Materialize data into **Iceberg tables** and catalog views for querying in Snowflake, DuckDB, AWS Athena, etc.
14
+
*Serve accurate data as**tooling** for LLMs and agents,
15
+
*Populate **data lakehouses** with **Iceberg tables** and catalog views.
You define the data processing in SQL and DataSQRL compiles the deployment artifacts for Apache Kafka, Flink, Postgres, Iceberg, GraphQL API, and LLM tooling. It generates the glue code, schemas, and mappings to automatically integrate and configure these components into a coherent data pipeline that is highly available, consistent, scalable, observable, and fast. DataSQRL supports quick local iteration, end-to-end pipeline testing, and deployment to Kubernetes or cloud-managed services.
19
+
You define the data processing in SQL and DataSQRL compiles the entire data infrastructure with Apache Kafka, Flink, Postgres, Iceberg, GraphQL API, and LLM tooling. It generates the glue code, schemas, mappings, and deployment artifacts to automatically integrate and configure these components into a coherent data stack that is highly available, consistent, scalable, observable, and fast. DataSQRL supports quick local iteration, end-to-end pipeline testing, and deployment to Kubernetes or cloud-managed services.
22
20
23
21
## DataSQRL Features
24
22
25
-
* 🔗 **Eliminate glue code:** DataSQRL generates connectors, schemas, data mappings, SQL dialect translation, and configurations. Do more with less.
23
+
* 🔗 **Eliminate Infrastructure Glue:** DataSQRL generates connectors, schemas, data mappings, SQL dialect translation, and configurations. Do more with less.
26
24
* 🚀 **Develop faster:** Local development, CI/CD support, logging framework, reusable components, and composable architecture for quick iteration cycles.
27
-
* 🛡️ **Reliable Data:** Consistent data processing with exactly or at-least once guarantees, testing framework, and data lineage.
25
+
* 🛡️ **Accurate Data:** Consistent data processing with exactly or at-least once guarantees, testing framework, and data lineage.
28
26
* 🔒 **Production-grade:** Robust, highly available, scalable, observable, and executed by trusted OSS technologies (Kafka, Flink, Postgres, DuckDB).
29
27
* 🤖 **AI-native:** Support for vector embeddings, LLM invocation, and ML model inference, and LLM tooling interfaces.
Copy file name to clipboardExpand all lines: documentation/blog/2025-05-07-datasqrl-0.6.md
+1-1Lines changed: 1 addition & 1 deletion
Original file line number
Diff line number
Diff line change
@@ -59,6 +59,6 @@ To help you transition, we’ve provided updated examples and migration guidance
59
59
60
60
This release wouldn’t have been possible without the contributions, bug reports, and thoughtful feedback from our growing community. Whether you opened a pull request, filed an issue, or joined a discussion, thank you. Your support drives this project forward.
61
61
62
-
We’re excited to see what you build with DataSQRL 0.6. If you haven’t joined the community yet, now’s a great time to get involved: star us on [GitHub](https://github.com/DataSQRL/sqrl), try out the latest release, and share your thoughts.
62
+
We’re excited to see what you build with DataSQRL 0.6. If you haven’t joined the [community](/community) yet, now’s a great time to get involved: star us on [GitHub](https://github.com/DataSQRL/sqrl), try out the latest release, and share your thoughts.
If you want to talk to the community, ask questions, brainstorm on your problem or tune into the development process behind DataSQRL,
33
+
join us <Linkto="https://join.slack.com/t/datasqrlcommunity/shared_invite/zt-2l3rl1g6o-im6YXYCqU7t55CNaHqz_Kg">on Slack</Link>. Get help and share your thoughts while watching how the sausage gets made.
34
+
</>
35
+
),
36
+
},
37
+
{
38
+
title: 'GitHub',
39
+
image: '/img/logos/github.svg',
40
+
link: 'https://github.com/DataSQRL/sqrl',
41
+
linkText: 'Contribute to DataSQRL',
42
+
description: (
43
+
<>
44
+
<Linkto="https://github.com/DataSQRL/sqrl">GitHub</Link> is where all open-source development on DataSQRL takes place.
45
+
<Linkto="https://github.com/DataSQRL/sqrl/issues">File a bug</Link>, star DataSQRL, or contribute to the codebase. That's the beauty of open-source: when
46
+
everybody contributes a little, something great can happen.
47
+
</>
48
+
),
49
+
},
50
+
{
51
+
title: 'Blog',
52
+
image: '/img/undraw/blog.svg',
53
+
link: '/blog',
54
+
linkText: 'Read the Blog',
55
+
description: (
56
+
<>
57
+
The <Linkto="/blog">DataSQRL blog</Link> regularly publishes articles on the development of
58
+
DataSQRL, how to implement data products, and lessons we learned along the way. Great morning reading.
0 commit comments