What happens during the subsequent applies with a btp resource that was created during the initial apply?

[ptesny]

I've recently started seeing deadlocks with btp entitlements.
For whatever reason with at minima 10 entitlements to be applied, sometimes, this is will go into a deadlock and eventually the entire tf configuration will fail.

Morever, reapplying will not help as the btp subaccount is in a failed state. It cannot be accessed at all from btp cocpit.
Thus, all I can do is to remove such a subaccount forcibly - and then start from scratch.

I was just wondering whether telling tf explicitly - just create a resource - and then do nothing (no updates, only reads) until it is supposed to be destroyed - would help a bit to alleviate the deadlock issue

On the other hand I have been experiencing the aforementioned deadlocks with the initial apply - so the subaccount was empty and these entitlements did not yet exist.

Using latest BTP TF provider version 1.9 and TF version 1.9.8

resource "btp_subaccount_entitlement" "cis" {
  subaccount_id = data.btp_subaccount.context.id
  service_name  = "cis"
  plan_name     = "local"

  lifecycle {
    ignore_changes = all
  }  
}

[lechnerc77]

First things first, I do not think that the lifecycle block with the ignore_changes should be used for this resource. The only "valid" scenario I can think of here (valid = a fair workaround) to exclude attributes of the resource configuration that might be changed by the platform (namely SAP BTP) to avoid unwanted "fixes" by Terraform.

Concerning the issue you are facing: we introduced a retry logic for assigning/creating entitlements as there are some locking issues when entitlements are provisioned in parallel (sometimes). While this should help concerning the rate limiting, if the API on the platform runs and puts the subaccount in an inconsistent state (not TF state), I do not see an option from the provider perspective to "heal" that situation on SAP BTP.

You could reduce the degree Terraform executes applications in parallel via the -parallelism parameter to a lower value than 10 (default) during apply to mitigate if this occurs due to too much parallel request (which "10" shouldn't be - but the API seems to have scaling issues).

[ptesny]

Got it. Thanks. There is one known to me use case where lifecycle block with the ignore_changes makes sense and is a life saver, namely

// SAPLaunchPad plan may morph into SAPLaunchPadSMS after the configuration change within SAP Workzone
// 
resource "btp_subaccount_entitlement" "build_workzone" {
  subaccount_id = data.btp_subaccount.context.id
  service_name  = local.service_name__build_workzone
  plan_name     = var.service_plan__build_workzone
  #amount        = var.service_plan__build_workzone == "free" ? 1 : null

  lifecycle {
    ignore_changes = all
  }  
}

# Create app subscription to SAP Build Workzone, standard edition (depends on entitlement)
resource "btp_subaccount_subscription" "build_workzone" {
  subaccount_id = data.btp_subaccount.context.id
  app_name      = local.service_name__build_workzone
  plan_name     = var.service_plan__build_workzone
  depends_on    = [btp_subaccount_entitlement.build_workzone]

  timeouts = {
    create = "25m"
    delete = "15m"
  }
  lifecycle {
    ignore_changes = all
  }  
}

Additional question, I've seen TF errors telling me that this resource -- the SAP Workzone subscription does not support updates.
But this does not happen all the time. Which is super confusing!

Thus, in order to reduce the risk of having TF to fail while re-applying I added the lifecycle block with the ignore_changes to prevent it.

Is it a rule of thumb for all BTP subscriptions ? That they do not support updates ?

thank you

[lechnerc77]

The resource subscription does not support updates in general (see https://github.com/SAP/terraform-provider-btp/blob/main/internal/provider/resource_subaccount_subscription.go#L272)

When the situation you describe appears, the Terraform execution plan wants to trigger an update on the resource. When this is triggered depends on the changes in your overall configuration and how Terraform interprets that change and propagates it down the chain of dependencies.

Thinking about it, maybe it makes sense to add RequiresReplace modifiers to the parameters, when one of the subscription parameters gets changed

[lechnerc77]

Opened feature request #1018 for resource replacement in case of updates

[ptesny]

As you have correctly assessed, depending on the overall configuration changes terraform may decide a subscription needs to updated which results in tf apply error.
Unfortunately when a subscription gets deleted this results in the loss of the subscription configuration.

Not sure if one could remove and recreate a subscription in place while retaining all the configuration ?
For instance roles and roles collections that a given subscription populates would disappear after its deletion.
Also many subscriptions allow for configurations which cannot be managed by terraform yet. Examples are SAP Workzone portals, AI Core Launchpad....

Thus, for the time being, with the btp subscriptions the only workaround is to add the lifecycle block with the ignore_changes.
This way a subscription gets created

  timeouts = {
    create = "25m"
    delete = "15m"
  } 

  lifecycle {
    ignore_changes = all
  }  

[lechnerc77]

At least with the recreation the process is clearer e.g. when a plan should be changed. However, this is an overall issue of the SAP BTP platform design imho. Too much stuff is happening somewhere under the hood implicitly. This is something that cannot be solved.
Your configuration might work for your setup, but it just cures the symptom. If you do not change the plan and the app name, the only thing that could trigger a change is data.btp_subaccount.context.id which shouldn't change per se but maybe some upstream dependency does some harm here. So I would recommend to further investigate in that direction instead of excluding the resource from any changes

[ptesny]

Your configuration might work for your setup, but it just cures the symptom. If you do not change the plan and the app name, the only thing that could trigger a change is data.btp_subaccount.context.id which shouldn't change per se but maybe some upstream dependency does some harm here.

I think this is a TF bug. I assumed that if all the subscription parameters remain the same, no update should ever be triggered.
So this must be down to how terrform generates and interprets the dependency graph.
And this is not the same with TF 1.9.3, 1.9.8 or 10.x.x (full of bugs)