milvus connection failure after onboarding to service mesh / envoy

[eyang-2]

saw this error upstream connect error or disconnect/reset before headers. reset reason: connection failure after trying to hit 9091 - all request timed out with the error message

however if i do port forward everything works but port forward is not the ideal way.

this is my milvus proxy config, has anyone seen this problem before?

  proxy:
    enabled: true
    replicas: 1
    resources:
      requests:
        cpu: "4000m"
        memory: "8Gi"
      limits:
        cpu: "4000m"
        memory: "8Gi"

[xiaofan-luan]

saw this error upstream connect error or disconnect/reset before headers. reset reason: connection failure after trying to hit 9091 - all request timed out with the error message

however if i do port forward everything works but port forward is not the ideal way.

this is my milvus proxy config, has anyone seen this problem before?

  proxy:
    enabled: true
    replicas: 1
    resources:
      requests:
        cpu: "4000m"
        memory: "8Gi"
      limits:
        cpu: "4000m"
        memory: "8Gi"

What's the goal to connect 9091? the webui?
Milvus GRPC port is 19530 and 9091 is used for pprof and webui only

[eyang-2]

What's the goal to connect 9091? the webui?

The goal is to provide product users access to 19530 and 9091 through company's service discovery. Its going to be hard for product teams do the port forward method.

after adding service discovery labels to milvus service the user experiences this timeout error but port forwarding works. I did not see any error log on proxy neither.

does milvus proxy require tls by default?

[yhmo]

TLS is disabled by default in milvus.
I think it is a k8s-related question, you can search for more info in k8s doc or blogs.
https://thorstenalpers.github.io/Docs/how-to-expose-kubernetes-ports.html

[eyang-2]

it appears that service discovery can talk to proxy but proxy timed out on requests like
"/api/v1/_cluster/dependencies" "/api/v1/_cluster/clients"
probably its an internal infra issue.