Virtual Node flagger-appmesh-gateway-appmesh-gateway not adding new backends

[jgordor]

Hi,

I have and exposed service and I can access it via the NLB -> Gateway:

apiVersion: appmesh.k8s.aws/v1beta1
kind: VirtualService
metadata:
  name: s1.apps.svc.cluster.local
  namespace: apps
  annotations:
    gateway.appmesh.k8s.aws/expose: "true"
    gateway.appmesh.k8s.aws/retries: "5"
    gateway.appmesh.k8s.aws/timeout: "25s"
    gateway.appmesh.k8s.aws/domain: "s1.com"

I see in the virtual node flagger-appmesh-gateway-appmesh-gateway that the backend s1.apps.svc.cluster.localis added .

But, when I added a second exposed service:

apiVersion: appmesh.k8s.aws/v1beta1
kind: VirtualService
metadata:
  name: s2.apps.svc.cluster.local
  namespace: apps
  annotations:
    gateway.appmesh.k8s.aws/expose: "true"
    gateway.appmesh.k8s.aws/retries: "5"
    gateway.appmesh.k8s.aws/timeout: "25s"
    gateway.appmesh.k8s.aws/domain: "s2.com"

This second exposed virtual service is not being added to the gateway virtual node and a 404 response is being returned.

Checking the logs I only see I0317 16:43:43.011647 1 virtualnode.go:114] virtual node flagger-appmesh-gateway updated with 1 backends How can I debug what's going on?

Thanks

[stefanprodan]

Is this a transient error or the backend is never added?

[jgordor]

The backend is never added

[stefanprodan]

I can't replicate this in CI. Do you see anything else in the logs? If you restart the gateway does it add it?