Skip to content

Commit 611b04d

Browse files
committed
docs(weekly): update weekly 72
1 parent ea6cb4c commit 611b04d

File tree

1 file changed

+17
-23
lines changed

1 file changed

+17
-23
lines changed

website/blog/2022-12-14-databend-weekly-72.md

Lines changed: 17 additions & 23 deletions
Original file line numberDiff line numberDiff line change
@@ -88,22 +88,17 @@ Check out what we've done this week to make Databend even better for you.
8888

8989
Stay connected with the latest news about Databend.
9090

91-
#### Unifying Format Options and Removing `format_*` Settings
91+
#### Breaking Change: Unified File Format Options
9292

93-
> ***Breaking Change*** We plan to unify the formatting options and remove `format_*` settings in the future.
94-
> This means that we will **disable** the old format.
95-
> Work is in progress and support for `FILE_FORMAT` has now been completed in streming load.
93+
To simplify, we're rolling out a set of unified file format options as follows for the COPY INTO command, the Streaming Load API, and all the other cases where users need to describe their file formats:
9694

97-
##### Propose
98-
99-
In the new implementation, **only** use the snowflake style in any place (insert copy and "select from stage" later):
100-
101-
- `FILE_FORMAT = (type = 'CSV' some_option = '|')`
102-
- `FILE_FORMAT = (format_name = 'MyCustomCSV')`
103-
104-
`FORMAT CSV` is retained for the ClickHouse handler.
95+
```sql
96+
[ FILE_FORMAT = ( TYPE = { CSV | TSV | NDJSON | PARQUET | XML} [ formatTypeOptions ] ) ]
97+
```
10598

106-
MyCustomCSV is created using SQL like `CREATE FILE FORMAT`
99+
- Please note that the current format options starting with `format_*` will be deprecated.
100+
- `... FORMAT CSV ...` will still be accepted by the ClickHouse handler.
101+
- Support for customized formats created by `CREATE FILE FORMAT ...` will be added in a future release: `... FILE_FORMAT = (format_name = 'MyCustomCSV') ....` .
107102

108103
**Learn More**
109104

@@ -112,12 +107,11 @@ MyCustomCSV is created using SQL like `CREATE FILE FORMAT`
112107

113108
#### Open Sharing
114109

115-
Open Sharing is a cheap and secure data sharing protocol for databend query on multi-cloud environments.
110+
Open Sharing is a simple and secure data-sharing protocol designed for databend-query nodes running in a multi-cloud environment.
116111

117-
* **Cheap**: Open Sharing allow data sharing via simple RESTful API sharing protocol, which is cheap and easy to understand.
118-
* **Secure**: Open Sharing protocol would verify allow incoming requesters identity and access permission and provide audit log.
119-
* **Multi-cloud**: Open Sharing is designed to work with different cloud platforms, including AWS, Azure, GCP, etc.
120-
* **Open source**: Open Sharing is an open source project
112+
- **Simple & Free**: Open Sharing is open-source and basically a RESTful API implementation.
113+
- **Secure**: Open Sharing verifies incoming requesters' identities and access permissions, and provides an audit log.
114+
- **Multi-Cloud**: Open Sharing supports a variety of public cloud platforms, including AWS, Azure, GCP, etc.
121115

122116
**Learn More**
123117

@@ -130,13 +124,13 @@ We're always open to cutting-edge technologies and innovative ideas. You're more
130124

131125
#### Refactoring Stage-Related Tests
132126

133-
We should use streaming upload to put files directly in the stage instead of in s3.
127+
We're about to run stage-related tests again using the Streaming Load API to move files to a stage instead of an AWS command like this:
134128

135129
```bash
136130
aws --endpoint-url ${STORAGE_S3_ENDPOINT_URL} s3 cp s3://testbucket/admin/data/ontime_200.csv s3://testbucket/admin/stage/internal/s1/ontime_200.csv >/dev/null 2>&1
137131
```
138132

139-
This is wrong, and stage's path rule should not be leak.
133+
This is because Databend users do not need to take care of, or do not even know the stage paths that the AWS command requires.
140134

141135
[Issue 8528: refactor stage related tests](https://github.com/datafuselabs/databend/issues/8528)
142136

@@ -171,9 +165,9 @@ Thanks a lot to the contributors for their excellent work this week.
171165
:---: |:---: |:---: |:---: |:---: |:---: |
172166
[soyeric128](https://github.com/soyeric128) |[sundy-li](https://github.com/sundy-li) |[wubx](https://github.com/wubx) |[Xuanwo](https://github.com/Xuanwo) |[xudong963](https://github.com/xudong963) |[youngsofun](https://github.com/youngsofun) |
173167

174-
[<img alt="ZhiHanZ" src="https://avatars.githubusercontent.com/u/25170437?v=4&s=117" width="117" />](https://github.com/ZhiHanZ) |[<img alt="zhyass" src="https://avatars.githubusercontent.com/u/34016424?v=4&s=117" width="117" />](https://github.com/zhyass) |[<img alt="zzzdong" src="https://avatars.githubusercontent.com/u/5125482?v=4&s=117" width="117" />](https://github.com/zzzdong) | | | |
175-
:---: |:---: |:---: |:---: |:---: |:---: |
176-
[ZhiHanZ](https://github.com/ZhiHanZ) |[zhyass](https://github.com/zhyass) |[zzzdong](https://github.com/zzzdong) | | | |
168+
[<img alt="ZhiHanZ" src="https://avatars.githubusercontent.com/u/25170437?v=4&s=117" width="117" />](https://github.com/ZhiHanZ) |[<img alt="zhyass" src="https://avatars.githubusercontent.com/u/34016424?v=4&s=117" width="117" />](https://github.com/zhyass) |[<img alt="zzzdong" src="https://avatars.githubusercontent.com/u/5125482?v=4&s=117" width="117" />](https://github.com/zzzdong) |
169+
:---: |:---: |:---: |
170+
[ZhiHanZ](https://github.com/ZhiHanZ) |[zhyass](https://github.com/zhyass) |[zzzdong](https://github.com/zzzdong) |
177171

178172
## Connect With Us
179173

0 commit comments

Comments
 (0)