"Why Are Spans from Multiple API Calls Appearing in a Single OpenTelemetry Trace?"

[NikunjPatel31]

Below is the trace generated by Opentelemetry. I hit two different APIs. i.e. "/api/greeting" and "/api/user/{username}". I received the below json. Now I want to understand why I am getting both API's spans into this one json.

{
  "resourceSpans": [
    {
      "resource": {
        "attributes": [
          {
            "key": "host.arch",
            "value": {
              "stringValue": "amd64"
            }
          },
          },
          {
            "key": "os.description",
            "value": {
              "stringValue": "Linux 5.15.0-136-generic"
            }
          },
          {
            "key": "os.type",
            "value": {
              "stringValue": "linux"
            }
          },
          {
            "key": "process.pid",
            "value": {
              "intValue": "2133971"
            }
          },
          {
            "key": "process.runtime.description",
            "value": {
              "stringValue": "Temurin OpenJDK 64-Bit Server VM 25.422-b05"
            }
          },
          {
            "key": "process.runtime.name",
            "value": {
              "stringValue": "OpenJDK Runtime Environment"
            }
          },
          {
            "key": "process.runtime.version",
            "value": {
              "stringValue": "1.8.0_422-b05"
            }
          },
          {
            "key": "service.instance.id",
            "value": {
              "stringValue": "2c2087ed-8e24-43b7-a477-2ca27264ba0c"
            }
          },
          {
            "key": "service.name",
            "value": {
              "stringValue": "Harsh"
            }
          },
          {
            "key": "telemetry.distro.name",
            "value": {
              "stringValue": "opentelemetry-java-instrumentation"
            }
          },
          {
            "key": "telemetry.distro.version",
            "value": {
              "stringValue": "2.12.0-SNAPSHOT"
            }
          },
          {
            "key": "telemetry.sdk.language",
            "value": {
              "stringValue": "java"
            }
          },
          {
            "key": "telemetry.sdk.name",
            "value": {
              "stringValue": "opentelemetry"
            }
          },
          {
            "key": "telemetry.sdk.version",
            "value": {
              "stringValue": "1.45.0"
            }
          }
        ]
      },
      "scopeSpans": [
        {
          "scope": {
            "name": "io.opentelemetry.tomcat-7.0",
            "version": "2.12.0-alpha-SNAPSHOT",
            "attributes": []
          },
          "spans": [
            {
              "traceId": "2da79c81d5b2a3c13bf5a63bd6f4d2cd",
              "spanId": "14ac59ec32f87670",
              "name": "GET /api/greeting",
              "kind": 2,
              "startTimeUnixNano": "1746706377450000000",
              "endTimeUnixNano": "1746706377451972395",
              "attributes": [
                {
                  "key": "url.path",
                  "value": {
                    "stringValue": "/api/greeting"
                  }
                },
                {
                  "key": "server.address",
                  "value": {
                    "stringValue": "172.16.12.127"
                  }
                },
                {
                  "key": "client.address",
                  "value": {
                    "stringValue": "10.20.40.143"
                  }
                },
                {
                  "key": "network.peer.address",
                  "value": {
                    "stringValue": "10.20.40.143"
                  }
                },
                {
                  "key": "network.peer.port",
                  "value": {
                    "intValue": "45952"
                  }
                },
                {
                  "key": "http.route",
                  "value": {
                    "stringValue": "/api/greeting"
                  }
                },
                {
                  "key": "http.request.method",
                  "value": {
                    "stringValue": "GET"
                  }
                },
                {
                  "key": "url.scheme",
                  "value": {
                    "stringValue": "http"
                  }
                },
                {
                  "key": "thread.name",
                  "value": {
                    "stringValue": "http-nio-9005-exec-5"
                  }
                },
                {
                  "key": "http.response.status_code",
                  "value": {
                    "intValue": "200"
                  }
                },
                {
                  "key": "user_agent.original",
                  "value": {
                    "stringValue": "curl/8.5.0"
                  }
                },
                {
                  "key": "thread.id",
                  "value": {
                    "intValue": "27"
                  }
                },
                {
                  "key": "network.protocol.version",
                  "value": {
                    "stringValue": "1.1"
                  }
                },
                {
                  "key": "server.port",
                  "value": {
                    "intValue": "9005"
                  }
                }
              ],
              "events": [],
              "links": [],
              "status": {},
              "flags": 257
            },
            {
              "traceId": "75ecc450523942b3e9a3e03d43346b6a",
              "spanId": "1246712f130a41f0",
              "name": "GET /api/greeting",
              "kind": 2,
              "startTimeUnixNano": "1746706377645000000",
              "endTimeUnixNano": "1746706377646643285",
              "attributes": [
                {
                  "key": "url.path",
                  "value": {
                    "stringValue": "/api/greeting"
                  }
                },
                {
                  "key": "server.address",
                  "value": {
                    "stringValue": "172.16.12.127"
                  }
                },
                {
                  "key": "client.address",
                  "value": {
                    "stringValue": "10.20.40.143"
                  }
                },
                {
                  "key": "network.peer.address",
                  "value": {
                    "stringValue": "10.20.40.143"
                  }
                },
                {
                  "key": "network.peer.port",
                  "value": {
                    "intValue": "45964"
                  }
                },
                {
                  "key": "http.route",
                  "value": {
                    "stringValue": "/api/greeting"
                  }
                },
                {
                  "key": "http.request.method",
                  "value": {
                    "stringValue": "GET"
                  }
                },
                {
                  "key": "url.scheme",
                  "value": {
                    "stringValue": "http"
                  }
                },
                {
                  "key": "thread.name",
                  "value": {
                    "stringValue": "http-nio-9005-exec-7"
                  }
                },
                {
                  "key": "http.response.status_code",
                  "value": {
                    "intValue": "200"
                  }
                },
                {
                  "key": "user_agent.original",
                  "value": {
                    "stringValue": "curl/8.5.0"
                  }
                },
                {
                  "key": "thread.id",
                  "value": {
                    "intValue": "29"
                  }
                },
                {
                  "key": "network.protocol.version",
                  "value": {
                    "stringValue": "1.1"
                  }
                },
                {
                  "key": "server.port",
                  "value": {
                    "intValue": "9005"
                  }
                }
              ],
              "events": [],
              "links": [],
              "status": {},
              "flags": 257
            },
            {
              "traceId": "e01dbb940ff869337e8c075f5660798b",
              "spanId": "64450abba20fbc7c",
              "name": "GET /api/greeting",
              "kind": 2,
              "startTimeUnixNano": "1746706377868000000",
              "endTimeUnixNano": "1746706377870112405",
              "attributes": [
                {
                  "key": "url.path",
                  "value": {
                    "stringValue": "/api/greeting"
                  }
                },
                {
                  "key": "server.address",
                  "value": {
                    "stringValue": "172.16.12.127"
                  }
                },
                {
                  "key": "client.address",
                  "value": {
                    "stringValue": "10.20.40.143"
                  }
                },
                {
                  "key": "network.peer.address",
                  "value": {
                    "stringValue": "10.20.40.143"
                  }
                },
                {
                  "key": "network.peer.port",
                  "value": {
                    "intValue": "45966"
                  }
                },
                {
                  "key": "http.route",
                  "value": {
                    "stringValue": "/api/greeting"
                  }
                },
                {
                  "key": "http.request.method",
                  "value": {
                    "stringValue": "GET"
                  }
                },
                {
                  "key": "url.scheme",
                  "value": {
                    "stringValue": "http"
                  }
                },
                {
                  "key": "thread.name",
                  "value": {
                    "stringValue": "http-nio-9005-exec-9"
                  }
                },
                {
                  "key": "http.response.status_code",
                  "value": {
                    "intValue": "200"
                  }
                },
                {
                  "key": "user_agent.original",
                  "value": {
                    "stringValue": "curl/8.5.0"
                  }
                },
                {
                  "key": "thread.id",
                  "value": {
                    "intValue": "31"
                  }
                },
                {
                  "key": "network.protocol.version",
                  "value": {
                    "stringValue": "1.1"
                  }
                },
                {
                  "key": "server.port",
                  "value": {
                    "intValue": "9005"
                  }
                }
              ],
              "events": [],
              "links": [],
              "status": {},
              "flags": 257
            },
            {
              "traceId": "7f84a7cea1de91c88cf20959e4e08987",
              "spanId": "751394a5254216ad",
              "name": "GET /api/greeting",
              "kind": 2,
              "startTimeUnixNano": "1746706378036000000",
              "endTimeUnixNano": "1746706378037493772",
              "attributes": [
                {
                  "key": "url.path",
                  "value": {
                    "stringValue": "/api/greeting"
                  }
                },
                {
                  "key": "server.address",
                  "value": {
                    "stringValue": "172.16.12.127"
                  }
                },
                {
                  "key": "client.address",
                  "value": {
                    "stringValue": "10.20.40.143"
                  }
                },
                {
                  "key": "network.peer.address",
                  "value": {
                    "stringValue": "10.20.40.143"
                  }
                },
                {
                  "key": "network.peer.port",
                  "value": {
                    "intValue": "45974"
                  }
                },
                {
                  "key": "http.route",
                  "value": {
                    "stringValue": "/api/greeting"
                  }
                },
                {
                  "key": "http.request.method",
                  "value": {
                    "stringValue": "GET"
                  }
                },
                {
                  "key": "url.scheme",
                  "value": {
                    "stringValue": "http"
                  }
                },
                {
                  "key": "thread.name",
                  "value": {
                    "stringValue": "http-nio-9005-exec-1"
                  }
                },
                {
                  "key": "http.response.status_code",
                  "value": {
                    "intValue": "200"
                  }
                },
                {
                  "key": "user_agent.original",
                  "value": {
                    "stringValue": "curl/8.5.0"
                  }
                },
                {
                  "key": "thread.id",
                  "value": {
                    "intValue": "23"
                  }
                },
                {
                  "key": "network.protocol.version",
                  "value": {
                    "stringValue": "1.1"
                  }
                },
                {
                  "key": "server.port",
                  "value": {
                    "intValue": "9005"
                  }
                }
              ],
              "events": [],
              "links": [],
              "status": {},
              "flags": 257
            },
            {
              "traceId": "d1ef101b62c75eb13637a16ad7439611",
              "spanId": "e579eb66e476545d",
              "name": "GET /api/user/{username}",
              "kind": 2,
              "startTimeUnixNano": "1746706378069000000",
              "endTimeUnixNano": "1746706378081146824",
              "attributes": [
                {
                  "key": "url.path",
                  "value": {
                    "stringValue": "/api/user/harsh"
                  }
                },
                {
                  "key": "server.address",
                  "value": {
                    "stringValue": "172.16.12.127"
                  }
                },
                {
                  "key": "client.address",
                  "value": {
                    "stringValue": "10.20.40.102"
                  }
                },
                {
                  "key": "network.peer.address",
                  "value": {
                    "stringValue": "10.20.40.102"
                  }
                },
                {
                  "key": "network.peer.port",
                  "value": {
                    "intValue": "42332"
                  }
                },
                {
                  "key": "http.route",
                  "value": {
                    "stringValue": "/api/user/{username}"
                  }
                },
                {
                  "key": "http.request.method",
                  "value": {
                    "stringValue": "GET"
                  }
                },
                {
                  "key": "url.scheme",
                  "value": {
                    "stringValue": "http"
                  }
                },
                {
                  "key": "thread.name",
                  "value": {
                    "stringValue": "http-nio-9005-exec-3"
                  }
                },
                {
                  "key": "http.response.status_code",
                  "value": {
                    "intValue": "200"
                  }
                },
                {
                  "key": "user_agent.original",
                  "value": {
                    "stringValue": "PostmanRuntime/7.43.4"
                  }
                },
                {
                  "key": "thread.id",
                  "value": {
                    "intValue": "25"
                  }
                },
                {
                  "key": "network.protocol.version",
                  "value": {
                    "stringValue": "1.1"
                  }
                },
                {
                  "key": "server.port",
                  "value": {
                    "intValue": "9005"
                  }
                }
              ],
              "events": [],
              "links": [],
              "status": {},
              "flags": 257
            },
            {
              "traceId": "d2122390f6b58b54e8385871104d6456",
              "spanId": "06719646f85cbc75",
              "name": "GET /api/greeting",
              "kind": 2,
              "startTimeUnixNano": "1746706378221000000",
              "endTimeUnixNano": "1746706378223156704",
              "attributes": [
                {
                  "key": "url.path",
                  "value": {
                    "stringValue": "/api/greeting"
                  }
                },
                {
                  "key": "server.address",
                  "value": {
                    "stringValue": "172.16.12.127"
                  }
                },
                {
                  "key": "client.address",
                  "value": {
                    "stringValue": "10.20.40.143"
                  }
                },
                {
                  "key": "network.peer.address",
                  "value": {
                    "stringValue": "10.20.40.143"
                  }
                },
                {
                  "key": "network.peer.port",
                  "value": {
                    "intValue": "45986"
                  }
                },
                {
                  "key": "http.route",
                  "value": {
                    "stringValue": "/api/greeting"
                  }
                },
                {
                  "key": "http.request.method",
                  "value": {
                    "stringValue": "GET"
                  }
                },
                {
                  "key": "url.scheme",
                  "value": {
                    "stringValue": "http"
                  }
                },
                {
                  "key": "thread.name",
                  "value": {
                    "stringValue": "http-nio-9005-exec-4"
                  }
                },
                {
                  "key": "http.response.status_code",
                  "value": {
                    "intValue": "200"
                  }
                },
                {
                  "key": "user_agent.original",
                  "value": {
                    "stringValue": "curl/8.5.0"
                  }
                },
                {
                  "key": "thread.id",
                  "value": {
                    "intValue": "26"
                  }
                },
                {
                  "key": "network.protocol.version",
                  "value": {
                    "stringValue": "1.1"
                  }
                },
                {
                  "key": "server.port",
                  "value": {
                    "intValue": "9005"
                  }
                }
              ],
              "events": [],
              "links": [],
              "status": {},
              "flags": 257
            },
            {
              "traceId": "a6ec33dfe36527f1d5e64a70559b2813",
              "spanId": "3e616fb0bb9d4505",
              "name": "GET /api/greeting",
              "kind": 2,
              "startTimeUnixNano": "1746706378392000000",
              "endTimeUnixNano": "1746706378394145466",
              "attributes": [
                {
                  "key": "url.path",
                  "value": {
                    "stringValue": "/api/greeting"
                  }
                },
                {
                  "key": "server.address",
                  "value": {
                    "stringValue": "172.16.12.127"
                  }
                },
                {
                  "key": "client.address",
                  "value": {
                    "stringValue": "10.20.40.143"
                  }
                },
                {
                  "key": "network.peer.address",
                  "value": {
                    "stringValue": "10.20.40.143"
                  }
                },
                {
                  "key": "network.peer.port",
                  "value": {
                    "intValue": "45992"
                  }
                },
                {
                  "key": "http.route",
                  "value": {
                    "stringValue": "/api/greeting"
                  }
                },
                {
                  "key": "http.request.method",
                  "value": {
                    "stringValue": "GET"
                  }
                },
                {
                  "key": "url.scheme",
                  "value": {
                    "stringValue": "http"
                  }
                },
                {
                  "key": "thread.name",
                  "value": {
                    "stringValue": "http-nio-9005-exec-6"
                  }
                },
                {
                  "key": "http.response.status_code",
                  "value": {
                    "intValue": "200"
                  }
                },
                {
                  "key": "user_agent.original",
                  "value": {
                    "stringValue": "curl/8.5.0"
                  }
                },
                {
                  "key": "thread.id",
                  "value": {
                    "intValue": "28"
                  }
                },
                {
                  "key": "network.protocol.version",
                  "value": {
                    "stringValue": "1.1"
                  }
                },
                {
                  "key": "server.port",
                  "value": {
                    "intValue": "9005"
                  }
                }
              ],
              "events": [],
              "links": [],
              "status": {},
              "flags": 257
            },
            {
              "traceId": "375cf0d13a4dad3b05a9940e035d3f77",
              "spanId": "5f8479ec27c953b6",
              "name": "GET /api/user/{username}",
              "kind": 2,
              "startTimeUnixNano": "1746706378519000000",
              "endTimeUnixNano": "1746706378521749171",
              "attributes": [
                {
                  "key": "url.path",
                  "value": {
                    "stringValue": "/api/user/harsh"
                  }
                },
                {
                  "key": "server.address",
                  "value": {
                    "stringValue": "172.16.12.127"
                  }
                },
                {
                  "key": "client.address",
                  "value": {
                    "stringValue": "10.20.40.102"
                  }
                },
                {
                  "key": "network.peer.address",
                  "value": {
                    "stringValue": "10.20.40.102"
                  }
                },
                {
                  "key": "network.peer.port",
                  "value": {
                    "intValue": "42332"
                  }
                },
                {
                  "key": "http.route",
                  "value": {
                    "stringValue": "/api/user/{username}"
                  }
                },
                {
                  "key": "http.request.method",
                  "value": {
                    "stringValue": "GET"
                  }
                },
                {
                  "key": "url.scheme",
                  "value": {
                    "stringValue": "http"
                  }
                },
                {
                  "key": "thread.name",
                  "value": {
                    "stringValue": "http-nio-9005-exec-8"
                  }
                },
                {
                  "key": "http.response.status_code",
                  "value": {
                    "intValue": "200"
                  }
                },
                {
                  "key": "user_agent.original",
                  "value": {
                    "stringValue": "PostmanRuntime/7.43.4"
                  }
                },
                {
                  "key": "thread.id",
                  "value": {
                    "intValue": "30"
                  }
                },
                {
                  "key": "network.protocol.version",
                  "value": {
                    "stringValue": "1.1"
                  }
                },
                {
                  "key": "server.port",
                  "value": {
                    "intValue": "9005"
                  }
                }
              ],
              "events": [],
              "links": [],
              "status": {},
              "flags": 257
            },
            {
              "traceId": "225414209fb9b57b7821c607f9d9df4b",
              "spanId": "2b83e583b40c6a35",
              "name": "GET /api/greeting",
              "kind": 2,
              "startTimeUnixNano": "1746706378566000000",
              "endTimeUnixNano": "1746706378568363204",
              "attributes": [
                {
                  "key": "url.path",
                  "value": {
                    "stringValue": "/api/greeting"
                  }
                },
                {
                  "key": "server.address",
                  "value": {
                    "stringValue": "172.16.12.127"
                  }
                },
                {
                  "key": "client.address",
                  "value": {
                    "stringValue": "10.20.40.143"
                  }
                },
                {
                  "key": "network.peer.address",
                  "value": {
                    "stringValue": "10.20.40.143"
                  }
                },
                {
                  "key": "network.peer.port",
                  "value": {
                    "intValue": "46006"
                  }
                },
                {
                  "key": "http.route",
                  "value": {
                    "stringValue": "/api/greeting"
                  }
                },
                {
                  "key": "http.request.method",
                  "value": {
                    "stringValue": "GET"
                  }
                },
                {
                  "key": "url.scheme",
                  "value": {
                    "stringValue": "http"
                  }
                },
                {
                  "key": "thread.name",
                  "value": {
                    "stringValue": "http-nio-9005-exec-9"
                  }
                },
                {
                  "key": "http.response.status_code",
                  "value": {
                    "intValue": "200"
                  }
                },
                {
                  "key": "user_agent.original",
                  "value": {
                    "stringValue": "curl/8.5.0"
                  }
                },
                {
                  "key": "thread.id",
                  "value": {
                    "intValue": "31"
                  }
                },
                {
                  "key": "network.protocol.version",
                  "value": {
                    "stringValue": "1.1"
                  }
                },
                {
                  "key": "server.port",
                  "value": {
                    "intValue": "9005"
                  }
                }
              ],
              "events": [],
              "links": [],
              "status": {},
              "flags": 257
            },
            {
              "traceId": "cb85af06fdc4f683ab46db352c592f39",
              "spanId": "b28da1b34c81d6f3",
              "name": "GET /api/greeting",
              "kind": 2,
              "startTimeUnixNano": "1746706378772000000",
              "endTimeUnixNano": "1746706378774102618",
              "attributes": [
                {
                  "key": "url.path",
                  "value": {
                    "stringValue": "/api/greeting"
                  }
                },
                {
                  "key": "server.address",
                  "value": {
                    "stringValue": "172.16.12.127"
                  }
                },
                {
                  "key": "client.address",
                  "value": {
                    "stringValue": "10.20.40.143"
                  }
                },
                {
                  "key": "network.peer.address",
                  "value": {
                    "stringValue": "10.20.40.143"
                  }
                },
                {
                  "key": "network.peer.port",
                  "value": {
                    "intValue": "46010"
                  }
                },
                {
                  "key": "http.route",
                  "value": {
                    "stringValue": "/api/greeting"
                  }
                },
                {
                  "key": "http.request.method",
                  "value": {
                    "stringValue": "GET"
                  }
                },
                {
                  "key": "url.scheme",
                  "value": {
                    "stringValue": "http"
                  }
                },
                {
                  "key": "thread.name",
                  "value": {
                    "stringValue": "http-nio-9005-exec-1"
                  }
                },
                {
                  "key": "http.response.status_code",
                  "value": {
                    "intValue": "200"
                  }
                },
                {
                  "key": "user_agent.original",
                  "value": {
                    "stringValue": "curl/8.5.0"
                  }
                },
                {
                  "key": "thread.id",
                  "value": {
                    "intValue": "23"
                  }
                },
                {
                  "key": "network.protocol.version",
                  "value": {
                    "stringValue": "1.1"
                  }
                },
                {
                  "key": "server.port",
                  "value": {
                    "intValue": "9005"
                  }
                }
              ],
              "events": [],
              "links": [],
              "status": {},
              "flags": 257
            },
            {
              "traceId": "1d27a85bf89dbbd47039ec8533b15d6a",
              "spanId": "85435a274bb93a32",
              "name": "GET /api/greeting",
              "kind": 2,
              "startTimeUnixNano": "1746706378933000000",
              "endTimeUnixNano": "1746706378935305161",
              "attributes": [
                {
                  "key": "url.path",
                  "value": {
                    "stringValue": "/api/greeting"
                  }
                },
                {
                  "key": "server.address",
                  "value": {
                    "stringValue": "172.16.12.127"
                  }
                },
                {
                  "key": "client.address",
                  "value": {
                    "stringValue": "10.20.40.143"
                  }
                },
                {
                  "key": "network.peer.address",
                  "value": {
                    "stringValue": "10.20.40.143"
                  }
                },
                {
                  "key": "network.peer.port",
                  "value": {
                    "intValue": "46022"
                  }
                },
                {
                  "key": "http.route",
                  "value": {
                    "stringValue": "/api/greeting"
                  }
                },
                {
                  "key": "http.request.method",
                  "value": {
                    "stringValue": "GET"
                  }
                },
                {
                  "key": "url.scheme",
                  "value": {
                    "stringValue": "http"
                  }
                },
                {
                  "key": "thread.name",
                  "value": {
                    "stringValue": "http-nio-9005-exec-3"
                  }
                },
                {
                  "key": "http.response.status_code",
                  "value": {
                    "intValue": "200"
                  }
                },
                {
                  "key": "user_agent.original",
                  "value": {
                    "stringValue": "curl/8.5.0"
                  }
                },
                {
                  "key": "thread.id",
                  "value": {
                    "intValue": "25"
                  }
                },
                {
                  "key": "network.protocol.version",
                  "value": {
                    "stringValue": "1.1"
                  }
                },
                {
                  "key": "server.port",
                  "value": {
                    "intValue": "9005"
                  }
                }
              ],
              "events": [],
              "links": [],
              "status": {},
              "flags": 257
            }
          ]
        }
      ],
      "schemaUrl": "https://opentelemetry.io/schemas/1.24.0"
    }
  ]
}

[jkwatson]

You'll need to provide more information about how your application is constructed. Can you provide a simple example that replicates the issue? Otherwise, we'd just be guessing at things.