Iceberg Implementation in marts

[thehanggit]

Following #545 procedures, I'm trying to implement iceberg tables for large tables in marts. For example, hourly or 5-minute based tables.

[thehanggit]

@ian-r-rose I have trouble with altering the dbt-core version from 1.8 to 1.9, it shows I need to run poetry lock first. Does that mean I need to have a local repository to proceed it on my computer?

[thehanggit]

@ian-r-rose I have trouble with altering the dbt-core version from 1.8 to 1.9, it shows I need to run poetry lock first. Does that mean I need to have a local repository to proceed it on my computer?

@ian-r-rose Should I follow these procedures to install dependencies first Setup

[ian-r-rose]

@ian-r-rose I have trouble with altering the dbt-core version from 1.8 to 1.9, it shows I need to run poetry lock first. Does that mean I need to have a local repository to proceed it on my computer?

If you want to run it locally, then yes, you would need to run poetry lock to update the version of dbt-core (or incorporate the changes from my branch)

@ian-r-rose Should I follow these procedures to install dependencies first Setup

Yes, that's a great place to start!

[ian-r-rose]

Incremental iceberg marts won't work until the bugfix to the dbt Snowflake adapter is released (it was merged yesterday!)

[thehanggit]

@ian-r-rose I have trouble with altering the dbt-core version from 1.8 to 1.9, it shows I need to run poetry lock first. Does that mean I need to have a local repository to proceed it on my computer?

If you want to run it locally, then yes, you would need to run poetry lock to update the version of dbt-core (or incorporate the changes from my branch)

@ian-r-rose Should I follow these procedures to install dependencies first Setup

Yes, that's a great place to start!

Thanks! I'm working on the local repository. Currently when I have walked through all the Setup steps and reached to the final step. pre-commit run --all-files still has some bugs and may I reach out to you tomorrow to go over it if I can't fix it today? Really appreciate it!