Skip to content

Commit 1d8eb10

Browse files
authored
Flink SQL hopping windows tutorial (#1630)
* feat: add Flink SQL hopping windows tutorial * chore: wording tweaks and typos fixes These are largely copy/paste issues noticed during hopping windows tutorial dev. * add note about readings affecting multiple hopping windows
1 parent dfdfd03 commit 1d8eb10

File tree

100 files changed

+1229
-50
lines changed

Some content is hidden

Large Commits have some content hidden by default. Use the searchbox below for content that may be hidden.

100 files changed

+1229
-50
lines changed

.semaphore/semaphore.yml

Lines changed: 3 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -502,6 +502,9 @@ blocks:
502502
- name: Flink SQL tumbling windows tests
503503
commands:
504504
- make -C _includes/tutorials/tumbling-windows/flinksql/code tutorial
505+
- name: Flink SQL hopping windows tests
506+
commands:
507+
- make -C _includes/tutorials/hopping-windows/flinksql/code tutorial
505508
- name: Flink SQL test for deduplication
506509
commands:
507510
- make -C _includes/tutorials/finding-distinct/flinksql/code tutorial

_data/harnesses/aggregating-count/flinksql.yml

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -13,7 +13,7 @@ dev:
1313
render:
1414
file: tutorials/aggregating-count/flinksql/markup/dev/init.adoc
1515

16-
- title: Get Confluent Platform
16+
- title: Get Confluent Platform and Apache Flink®
1717
content:
1818
- action: make_file
1919
file: docker-compose.yml

_data/harnesses/aggregating-minmax/flinksql.yml

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -13,7 +13,7 @@ dev:
1313
render:
1414
file: tutorials/aggregating-minmax/flinksql/markup/dev/init.adoc
1515

16-
- title: Get Confluent Platform
16+
- title: Get Confluent Platform and Apache Flink®
1717
content:
1818
- action: make_file
1919
file: docker-compose.yml

_data/harnesses/filtering/flinksql.yml

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -13,7 +13,7 @@ dev:
1313
render:
1414
file: tutorials/filtering/flinksql/markup/dev/init.adoc
1515

16-
- title: Get Confluent Platform
16+
- title: Get Confluent Platform and Apache Flink®
1717
content:
1818
- action: make_file
1919
file: docker-compose.yml

_data/harnesses/finding-distinct/flinksql.yml

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -13,7 +13,7 @@ dev:
1313
render:
1414
file: tutorials/finding-distinct/flinksql/markup/dev/init.adoc
1515

16-
- title: Get Confluent Platform
16+
- title: Get Confluent Platform and Apache Flink®
1717
content:
1818
- action: make_file
1919
file: docker-compose.yml
Lines changed: 152 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,152 @@
1+
dev:
2+
steps:
3+
- title: Prerequisites
4+
content:
5+
- action: skip
6+
render:
7+
file: shared/markup/dev/docker-prerequisite.adoc
8+
9+
- title: Initialize the project
10+
content:
11+
- action: execute
12+
file: tutorial-steps/dev/init.sh
13+
render:
14+
file: tutorials/hopping-windows/flinksql/markup/dev/init.adoc
15+
16+
- title: Get Confluent Platform and Apache Flink®
17+
content:
18+
- action: make_file
19+
file: docker-compose.yml
20+
render:
21+
file: tutorials/hopping-windows/flinksql/markup/dev/make-docker-compose.adoc
22+
23+
- action: execute_async
24+
file: tutorial-steps/dev/docker-compose-up.sh
25+
render:
26+
file: tutorials/hopping-windows/flinksql/markup/dev/start-compose.adoc
27+
28+
- action: execute
29+
file: tutorial-steps/dev/wait-for-containers.sh
30+
render:
31+
skip: true
32+
33+
- title: Write the program interactively using the CLI
34+
content:
35+
- action: docker_flinksql_cli_session
36+
container: flink-sql-client
37+
docker_bootup_file: tutorial-steps/dev/start-cli.sh
38+
column_width: 20
39+
render:
40+
file: tutorials/hopping-windows/flinksql/markup/dev/start-cli.adoc
41+
stdin:
42+
- file: tutorial-steps/dev/create-temperature-readings.sql
43+
render:
44+
file: tutorials/hopping-windows/flinksql/markup/dev/create-temperature-readings.adoc
45+
46+
- file: tutorial-steps/dev/populate-temperature-readings.sql
47+
render:
48+
file: tutorials/hopping-windows/flinksql/markup/dev/populate-temperature-readings.adoc
49+
50+
- file: tutorial-steps/dev/transient-query-temperature-by-10min-window.sql
51+
render:
52+
file: tutorials/hopping-windows/flinksql/markup/dev/transient-query-temperature-by-10min-window.adoc
53+
54+
- file: tutorial-steps/dev/create-temperature-by-10min-window.sql
55+
render:
56+
file: tutorials/hopping-windows/flinksql/markup/dev/create-temperature-by-10min-window.adoc
57+
58+
stdout:
59+
directory: tutorial-steps/dev/outputs
60+
61+
- name: wait for table execution to complete
62+
action: sleep
63+
ms: 10000
64+
render:
65+
skip: true
66+
67+
- title: Validate output
68+
content:
69+
- action: execute
70+
file: tutorial-steps/dev/validate-temperature-by-10min-window.sh
71+
stdout: tutorial-steps/dev/outputs/validate-temperature-by-10min-window.log
72+
render:
73+
file: tutorials/hopping-windows/flinksql/markup/dev/validate-temperature-by-10min-window.adoc
74+
75+
test:
76+
steps:
77+
- title: Decide what testing tools to use
78+
content:
79+
- action: skip
80+
render:
81+
file: tutorials/hopping-windows/flinksql/markup/test/test-architecture.adoc
82+
83+
- title: Create the test skeleton
84+
content:
85+
- action: execute
86+
file: tutorial-steps/test/make-test-dirs.sh
87+
render:
88+
file: tutorials/hopping-windows/flinksql/markup/test/make-test-dirs.adoc
89+
90+
- action: make_file
91+
file: build.gradle
92+
render:
93+
file: tutorials/hopping-windows/flinksql/markup/test/make-build-gradle.adoc
94+
95+
- action: execute
96+
file: tutorial-steps/test/gradle-wrapper.sh
97+
render:
98+
file: tutorials/hopping-windows/flinksql/markup/test/make-gradle-wrapper.adoc
99+
100+
- title: Create SQL resources
101+
content:
102+
- action: make_file
103+
file: src/test/resources/create-temperature-readings.sql.template
104+
render:
105+
file: tutorials/hopping-windows/flinksql/markup/test/create-resource-create-temperature-readings.sql.template.adoc
106+
107+
- action: make_file
108+
file: src/test/resources/populate-temperature-readings.sql
109+
render:
110+
file: tutorials/hopping-windows/flinksql/markup/test/create-resource-populate-temperature-readings.sql.adoc
111+
112+
- action: make_file
113+
file: src/test/resources/create-temperature-by-10min-window.sql.template
114+
render:
115+
file: tutorials/hopping-windows/flinksql/markup/test/create-resource-create-temperature-by-10min-window.sql.template.adoc
116+
117+
- action: make_file
118+
file: src/test/resources/query-temperature-by-10min-window.sql
119+
render:
120+
file: tutorials/hopping-windows/flinksql/markup/test/create-resource-query-temperature-by-10min-window.sql.adoc
121+
122+
- action: make_file
123+
file: src/test/resources/expected-temperature-by-10min-window.txt
124+
render:
125+
file: tutorials/hopping-windows/flinksql/markup/test/create-resource-expected-temperature-by-10min-window.txt.adoc
126+
127+
- title: Write a test
128+
content:
129+
- action: make_file
130+
file: src/test/java/io/confluent/developer/AbstractFlinkKafkaTest.java
131+
render:
132+
file: tutorials/hopping-windows/flinksql/markup/test/make-test-base.adoc
133+
134+
- action: make_file
135+
file: src/test/java/io/confluent/developer/FlinkSqlHoppingWindowTest.java
136+
render:
137+
file: tutorials/hopping-windows/flinksql/markup/test/make-test.adoc
138+
139+
- title: Invoke the test
140+
content:
141+
- action: execute
142+
file: tutorial-steps/test/invoke-test.sh
143+
render:
144+
file: tutorials/hopping-windows/flinksql/markup/test/invoke-test.adoc
145+
146+
ccloud:
147+
steps:
148+
- title: Run your app with Confluent Cloud
149+
content:
150+
- action: skip
151+
render:
152+
file: shared/markup/ccloud/try-ccloud.adoc

_data/harnesses/joining-stream-stream/flinksql.yml

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -13,7 +13,7 @@ dev:
1313
render:
1414
file: tutorials/joining-stream-stream/flinksql/markup/dev/init.adoc
1515

16-
- title: Get Confluent Platform
16+
- title: Get Confluent Platform and Apache Flink®
1717
content:
1818
- action: make_file
1919
file: docker-compose.yml

_data/harnesses/splitting/flinksql.yml

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -13,7 +13,7 @@ dev:
1313
render:
1414
file: tutorials/splitting/flinksql/markup/dev/init.adoc
1515

16-
- title: Get Confluent Platform
16+
- title: Get Confluent Platform and Apache Flink®
1717
content:
1818
- action: make_file
1919
file: docker-compose.yml

_data/harnesses/tumbling-windows/flinksql.yml

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -13,7 +13,7 @@ dev:
1313
render:
1414
file: tutorials/tumbling-windows/flinksql/markup/dev/init.adoc
1515

16-
- title: Get Confluent Platform
16+
- title: Get Confluent Platform and Apache Flink®
1717
content:
1818
- action: make_file
1919
file: docker-compose.yml

_data/tutorials.yaml

Lines changed: 3 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -174,11 +174,12 @@ hopping-windows:
174174
into fixed-size, possibly overlapping, contiguous time intervals to identify a
175175
specific scenario?
176176
introduction: You want to build an alerting system that automatically detects if
177-
the temperature of a room consistently drops. In this tutorial, we'll write a
177+
the temperature of a room drops below a threshold over a period of time. In this tutorial, we'll write a
178178
program that monitors a stream of temperature readings and detects when the temperature
179-
consistently drops below 45 degrees Fahrenheit for a period of 10 minutes.
179+
drops below 45 degrees Fahrenheit for a period of 10 minutes.
180180
status:
181181
ksql: enabled
182+
flinksql: enabled
182183
session-windows:
183184
title: Create session windows
184185
meta-description: create session windows for time-series events

0 commit comments

Comments
 (0)