流表使用窗口函数无结果输出 #1276
Unanswered
whitecloud6688
asked this question in
Q&A
流表使用窗口函数无结果输出
#1276
Replies: 0 comments
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Uh oh!
There was an error while loading. Please reload this page.
Uh oh!
There was an error while loading. Please reload this page.
-
请问在查流表的时候为啥没有结果输出(见语句1),查批表的时候有结果输出(见语句2)?hudi_orders_mor_s和hudi_orders_mor_b确定都是有数据的。将参数:read.streaming.check-BIGINTerval 变更为:read.streaming.check-interval ,效果一样,流表无结果,批表有结果。谢谢。
CREATE TABLE hudi_orders_mor_s (
order_id BIGINT comment '订单编号',
order_date TIMESTAMP(3) comment '订单时间',
customer_name STRING comment '客户名称',
price FLOAT comment '价格',
product_id BIGINT comment '产品编号',
order_status BOOLEAN comment '订单状态',
WATERMARK FOR order_date AS order_date - INTERVAL '10' MINUTES,
PRIMARY KEY (order_id) NOT ENFORCED
) COMMENT '订单表'
WITH (
'connector' = 'hudi',
'table.type' = 'MERGE_ON_READ',
'path' = '/hive/warehouse/test.db/hudi_orders_mor_s',
'read.streaming.enabled' = 'true',
'read.streaming.check-BIGINTerval' = '3',
'write.bucket_assign.tasks' = '1',
'write.tasks' = '1',
'hive_sync.enable' = 'true',
'hive_sync.mode' = 'hms',
'hive_sync.metastore.uris' = 'thrift://vm01:9083'
);
语句1 无结果
SELECT window_start, window_end, window_time, COUNT(1) AS cnt, SUM(price) AS price
FROM TABLE (TUMBLE(TABLE hudi_orders_mor_s, DESCRIPTOR(order_date), INTERVAL '1' HOUR))
GROUP BY window_start, window_end, window_time;
CREATE TABLE hudi_orders_mor_b (
order_id BIGINT comment '订单编号',
order_date TIMESTAMP(3) comment '订单时间',
customer_name STRING comment '客户名称',
price FLOAT comment '价格',
product_id BIGINT comment '产品编号',
order_status BOOLEAN comment '订单状态',
WATERMARK FOR order_date AS order_date - INTERVAL '10' MINUTES,
PRIMARY KEY (order_id) NOT ENFORCED
) COMMENT '订单表'
WITH (
'connector' = 'hudi',
'table.type' = 'MERGE_ON_READ',
'path' = '/hive/warehouse/test.db/hudi_orders_mor_b',
'read.streaming.enabled' = 'false',
'read.streaming.check-BIGINTerval' = '3',
'write.bucket_assign.tasks' = '1',
'write.tasks' = '1',
'hive_sync.enable' = 'true',
'hive_sync.mode' = 'hms',
'hive_sync.metastore.uris' = 'thrift://vm01:9083'
);
语句2 有结果
SELECT window_start, window_end, window_time, COUNT(1) AS cnt, SUM(price) AS price
FROM TABLE (TUMBLE(TABLE hudi_orders_mor_b, DESCRIPTOR(order_date), INTERVAL '1' HOUR))
GROUP BY window_start, window_end, window_time;
其中 hudi_orders_mor_s 和 hudi_orders_mor_b 的数据来源如下:
CREATE TABLE orders (
order_id INT COMMENT '订单id',
order_date TIMESTAMP(0) COMMENT '下单时间',
customer_name STRING COMMENT '客户名称',
price DECIMAL(10, 5) COMMENT '价格',
product_id INT COMMENT '产品id',
order_status BOOLEAN COMMENT '订单状态',
PRIMARY KEY (order_id) NOT ENFORCED
) WITH (
'connector' = 'mysql-cdc',
'hostname' = 'vm01',
'port' = '3306',
'username' = 'user',
'password' = 'passwd',
'database-name' = 'mydb',
'table-name' = 'orders'
);
insert into hudi_orders_mor_s select * from orders;
insert into hudi_orders_mor_b select * from orders;
版本:
flink-1.14.4
hudi-0.11.0
spark-3.2.1-bin-hadoop2.7
hadoop-2.10.1
Beta Was this translation helpful? Give feedback.
All reactions