Replies: 1 comment
-
DagsterAI provided four examples, including the following. def create_assets_with_pattern(pattern: str):
|
Beta Was this translation helpful? Give feedback.
0 replies
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Uh oh!
There was an error while loading. Please reload this page.
Uh oh!
There was an error while loading. Please reload this page.
-
To build asset dependencies, the documentation states the following:
“By passing upstream assets as deps parameters to the @asset decorator of downstream assets,
you can define dependencies between two assets.”
Example:
@dg.asset
def sugary_cereals(): ...
@dg.asset(deps=[sugary_cereals])
def shopping_list(): ....
When building multiple assets that span multiple modules and
constructing dependencies across modules,
changing the dependency structure can become a complicated task.
Therefore, instead of specifying dependencies directly in the @asset decorator parameter,
we would like to create a module that describes only the dependencies.
(Something like a central dependency management module)
For example, is it possible to create a central dependency management module that only describes asset dependencies, as shown in the following example?
assets_sugary.py
@dg.asset
def sugary_cereals(): ...
assets_shopping.py
@dg.asset(deps=[sugary_cereals])
def shopping_list(): ....
dependencies.py
import assets_sugary.sugary_cereals
import assets_shopping.shopping_list
def assets_dependencies():
Describe the dependencies between “sugary_cereals” and “shopping_list” here
Even without using Dagster's features, you can create something like dependencies by arranging functions
in the order they are processed.
However, if possible, I would like to use Dagster's functionality to
construct dependencies and make the constructed relationships viewable in the UI.
dependencies.py
import assets_sugary.sugary_cereals
import assets_shopping.shopping_list
def assets_dependencies():
sugary_cereals()
assets_shopping()
Beta Was this translation helpful? Give feedback.
All reactions