Asset with multi_run backfill policy does not show run_time metadata on the asset details pane.
What's the issue?
Hi team,
I have a partitioned asset with backfill_policy=BackfillPolicy.multi_run(max_partitions_per_run=1=20).
For every partition that completes I want to capture metadata. I have tried the `MaterializeResult, but that has no partition arg and so it does not work for me. I ended up using AssetMaterialization as an event, or AssetObservation as an event.
The events are captured, but the asset details pane does not show a plot of the (numeric) results. The asset detail pane, under partitions has a button 'view all historical events (2)'. When I click that button it shows one observation and one materialization (when I used AssetObservation). this event does show me the metadata. But the overview asset detail page does not have that information and no plots.
What did you expect to happen?
I expect the Asset overview details pane to show me all the metadata plots for that asset. Just like it does for other assets.
How to reproduce?
I don't have a code example so I'm going to write the details:
- Create an partitioned asset or partitioned graph_asset with the 'experimental' BackfillPolicy.multi_run
- in the asset, loop through the partition keys
- for every partition key, do an assetObservation with numeric metadata
- Run a backfil of one partition
- observe the results
- the display of that asset does not show any run_time metadata
Dagster version
1.7.9
Deployment type
Other
Deployment details
I created my own kubernetes chart for dagster. It runs on k8s with a minio storage and a postgres database. But I do not think my deployment details are the issue here.
Additional information
I thought it was related to graph_asset but it happens on 'normal' partitioned assets too.
Could this be related to #24600 ?
Message from the maintainers
Impacted by this issue? Give it a ! We factor engagement into prioritization. By submitting this issue, you agree to follow Dagster's Code of Conduct.
Source: dagster-io/dagster