Notification settings in Alert rule group not sync'ed

[chauvm]

Describe the bug
When creating a GrafanaAlertRuleGroup with association with a contact point, the association is not created.

Version
v5.12.0

To Reproduce
Steps to reproduce the behavior:

1. Go to Grafana and export a rule group that has a contact point as the notification policy
2. Use the rule to add to a GrafanaAlertRuleGroup manifest's spec.rules

apiVersion: grafana.integreatly.org/v1beta1
kind: GrafanaAlertRuleGroup
metadata: ...
spec:
  rules:
    - annotations: ...
      data:
        - datasourceUid:...
      notification_settings:
        receiver: my-contact-point
      title: ...
      uid: ...

3. From ArgoCD, see that the created alert rule group doesn't have the notification_settings.receiver, and from Grafana confirm that there's no such association.

Expected behavior
The alert rule should have linked correctly to the contact point.

Suspect component/Location where the bug might be occurring
Unknown. Not sure if this is related, but my contact point also has this bug #1644.

Screenshots
If applicable, add screenshots to help explain your problem.

Runtime (please complete the following information):

• OS: [e.g. Linux,Fedora,Mac]
• Grafana Operator Version [e.g. v5.0.0]: v5.12.0
• Environment: [e.g Openshift,Kubernetes,minikube etc. please specify versions]
• Deployment type: [e.g Openshift OLM/Helm/kustomize]
• Other: [Other variables/things that might be relevant to this bug, versions of other services e.g. operator-sdk]

Additional context
Add any other context about the problem here.

[theSuess]

To keep in line with kubernetes convention, the operator uses notificationSettings instead of notification_settings

[DrDJIng]

Could we add support for notification_settings? In our use case, it's very frustrating to either have to ask our users to change this when provisioning, or (what we chose to do) maintain an extra custom exporter where all it does is adjust notification_settings to notificationSettings.

I understand it goes against the convention, but when that is what grafana exports, I think it's important to minimise friction. So much good work has already been done to avoid diffs when using argocd, this would be the last piece for us to finally remove the extra step of the custom exporting tool.

I would be happy to make a PR for this.

[Baarsgaard]

Could it make sense for you to use Notification Policies?
The operator supports dynamically assembling the policy routes from GrafanaNotificationPolicyRoute objects in the cluster.
This would allow your users to add their own Contact points and configure the NotificationPolicy in-depth for the their Contact points
It's not a solution but an alternative approach.

[DrDJIng]

Thanks for reminding me that exists! This is a possibility, but I still think a longer workaround via notification policies and labels is a poor choice when there's the option of a single click when creating the alert to choose the contact point.

We have our workaround that works for us, but I just think it's rough to have to implement a workaround at all for the sake of not including an underscore.

At the end of the day, this isn't a big deal, it's just a "would be nice to have" thing.

[Baarsgaard]

The main reason we're hesitant is that it's a choice we made a long time ago and until we release v1 it's unlikely to change.