Skip to content

Manifest file not generated after successful FinOps Hub V12 deployment – ADF pipeline not triggering #1873

@brtklc

Description

@brtklc

Issue Description

I’ve successfully deployed FinOps Hub V12 into my Azure Resource Group.
Post-deployment, I performed the following steps:

  • Registered the Microsoft.EventGrid and Microsoft.CostManagementExports resource providers on the target subscription.
  • Granted the Cost Management Contributor RBAC role to the ADF Managed Identity on the target subscription.
  • Added the subscription to the scopes section in the settings.json file within the config container.

Observed Behavior

After some investigation, I noticed that:

  • No export is created on the Cost Management page.
  • As a result, the manifest file is not being created in the msexports container.
  • Consequently, the msexports_ExecuteETL ADF pipeline is not triggered, and no cost data is ingested into the ingestion container.

Investigation Details

I looked into the config_ConfigureExports pipeline, which is expected to automatically configure exports.
The Filter Invalid Scopes activity seems to have incorrect dependency logic.

  • The Save Scope activity keeps failing.
  • Even if Save Scopes as Array succeeds, the pipeline does not proceed to Filter Invalid Scopes because of the dependsOn configuration.

Current logic requires both conditions to be true:

  • Save Scopes as ArraySucceeded or Skipped
  • Save ScopesSucceeded

Here’s the current configuration:

{ // Filter Invalid Scopes
  "name": "Filter Invalid Scopes",
  "type": "Filter",
  "dependsOn": [
    {
      "activity": "Save Scopes",
      "dependencyConditions": [ "Succeeded" ]
    },
    {
      "activity": "Save Scopes as Array",
      "dependencyConditions": [ "Skipped", "Succeeded" ]
    }
  ],
  "userProperties": [],
  "typeProperties": {
    "items": {
      "value": "@variables('scopesArray')",
      "type": "Expression"
    },
    "condition": {
      "value": "@and(not(empty(item().scope)), not(equals(item().scope, '/')))",
      "type": "Expression"
    }
  }
}
Image

Analysis

This logic seems incorrect because it enforces an AND condition.
In my case, Save Scopes fails, but Save Scopes as Array succeeds, so the filter never runs.

Workaround

As a temporary fix, I changed the dependency for Save Scopes to Completed (i.e., any terminal state: Succeeded, Failed, or Skipped).
This allows the pipeline to continue even if Save Scopes fails, which works for my scenario.

{
  "name": "Filter Invalid Scopes",
  "type": "Filter",
  "dependsOn": [
    {
      "activity": "Save scopes as array",
      "dependencyConditions": [ "Succeeded", "Skipped" ]
    },
    {
      "activity": "Save scopes",
      "dependencyConditions": [ "Completed" ]
    }
  ]
}

So my question:

Could you please advise me on how to handle this issue better? Because if my observation is correct, many user could face the same issue.

Metadata

Metadata

Assignees

No one assigned

    Labels

    Needs: Triage 🔍Untriaged issue needs to be reviewed

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions