Skip to content

Update Kedro-Ibis blog post #160

@deepyaman

Description

@deepyaman

(Sections correspond to sections in the blog post.)

The dev-prod dilemma

[Although that begs the question of how standardised is the SQL standard? Gil Forsyth's PyData NYC 2022 talk demonstrates challenges arising from differences between SQL dialects. Even the dbt-labs/jaffle_shop GitHub repository README disclaims, "If this steps fails, it might mean that you need to make small changes to the SQL in the models folder to adjust for the flavor of SQL of your target database"].

I think slightly better phrasing might be, as a separate paragraph in parentheses:

(Although that begs the question: how standardised is the SQL standard? Gil Forsyth's PyData NYC 2022 talk demonstrates challenges arising from differences between SQL dialects. Even the dbt-labs/jaffle_shop GitHub repository README disclaims, "If this steps fails, it might mean that you need to make small changes to the SQL in the models folder to adjust for the flavor of SQL of your target database.")

Creating a custom ibis.Table dataset

Can we put a callout here? Or however it works, basically we can replace

Find the complete dataset implemention on GitHub.

with

Note: Since this article was originally published, Ibis datasets have been contributed to the official Kedro-Datasets repository. Find the complete dataset implementations on GitHub.

Configuring backends with the OmegaConfigLoader using variable interpolation

I guess we can update the examples to simply replace
jaffle_shop.datasets.ibis.TableDataset
with
ibis.TableDataset

Building pipelines

I've significantly simplified the type hints by:

Remove from __future__ import annotations (Python 3.8 is EOL)
Remove TYPE_CHECKING import and TYPE_CHECKING block
Replace ir.Table with ibis.Table

Try it yourself

Nit: Change kedro viz run to kedro viz

Metadata

Metadata

Labels

No labels
No labels

Type

No type

Projects

Status

In Progress

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions