Skip to content

Commit 7253468

Browse files
MeelahMejstirnaman
authored andcommitted
Added an export command example
1 parent 24cf2cf commit 7253468

File tree

1 file changed

+48
-9
lines changed

1 file changed

+48
-9
lines changed

content/shared/influxdb3-query-guides/snapshots/snowflake.md

Lines changed: 48 additions & 9 deletions
Original file line numberDiff line numberDiff line change
@@ -62,18 +62,57 @@ For more information, refer to the [Snowflake documentation](https://docs.snowfl
6262

6363
Use InfluxData's Iceberg exporter to convert and export your time-series data from your {{% product-name omit="Clustered" %}} cluster into the Iceberg table format.
6464

65-
### Example: Export data using the Iceberg exporter
65+
### Example: Export data using Iceberg exporter
6666

67-
```sh
68-
# Clone the Iceberg exporter repository
69-
git clone https://github.com/influxdata/influxdb_iox.git
70-
cd influxdb_iox/iceberg_exporter
71-
```
67+
This example assumes:
68+
- You have followed the example for [writing and querying data in the IOx README](https://github.com/influxdata/influxdb_iox/blob/main/README.md#write-and-read-data)
69+
- You've configured compaction to trigger more quickly with these environment variables:
70+
71+
- `INFLUXDB_IOX_COMPACTION_MIN_NUM_L0_FILES_TO_COMPACT=1`
72+
- `INFLUXDB_IOX_COMPACTION_MIN_NUM_L1_FILES_TO_COMPACT=1`
73+
- You have a `config.json`
7274

73-
- Configure the exporter with your InfluxDB data source and target Iceberg table location.
74-
- Run the exporter to generate Iceberg-compatible Parquet files.
75+
#### Example `config.json`
7576

76-
For more details, refer to the [Iceberg Exporter README](https://github.com/influxdata/influxdb_iox/tree/main/iceberg_exporter).
77+
```json
78+
{
79+
"exports": [
80+
{
81+
"namespace": "company_sensors",
82+
"table_name": "cpu"
83+
}
84+
]
85+
}
86+
```
87+
#### Running the export command
88+
89+
```console
90+
$ influxdb_iox iceberg export \
91+
--catalog-dsn postgresql://postgres@localhost:5432/postgres \
92+
--source-object-store file
93+
--source-data-dir ~/.influxdb_iox/object_store \
94+
--sink-object-store file \
95+
--sink-data-dir /tmp/iceberg \
96+
--export-config-path config.json
97+
```
98+
The export command outputs an absolute path to an Iceberg metadata file:
99+
100+
`/tmp/iceberg/company_sensors/cpu/metadata/v1.metadata.json
101+
`
102+
#### Example: Querying the exported metadata using duckdb
103+
104+
```console
105+
$ duckdb
106+
D SELECT * FROM iceberg_scan('/tmp/iceberg/metadata/v1.metadata.json') LIMIT 1;
107+
┌───────────┬──────────────────────┬─────────────────────┬─────────────┬───┬────────────┬───────────────┬─────────────┬────────────────────┬────────────────────┐
108+
│ cpu │ host │ time │ usage_guest │ … │ usage_nice │ usage_softirq │ usage_steal │ usage_system │ usage_user │
109+
│ varchar │ varchar │ timestamp │ double │ │ double │ double │ double │ double │ double │
110+
├───────────┼──────────────────────┼─────────────────────┼─────────────┼───┼────────────┼───────────────┼─────────────┼────────────────────┼────────────────────┤
111+
│ cpu-total │ Andrews-MBP.hsd1.m… │ 2020-06-11 16:52:00 │ 0.0 │ … │ 0.0 │ 0.0 │ 0.0 │ 1.1173184357541899 │ 0.9435133457479826 │
112+
├───────────┴──────────────────────┴─────────────────────┴─────────────┴───┴────────────┴───────────────┴─────────────┴────────────────────┴────────────────────┤
113+
│ 1 rows 13 columns (9 shown) │
114+
└───────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────┘
115+
```
77116

78117
### Create an Iceberg table in Snowflake
79118

0 commit comments

Comments
 (0)