Skip to content

Commit 0f9eb5a

Browse files
lucaspimentelclaude
andcommitted
Add PowerShell script for querying Datadog traces
Add Get-DatadogTrace.ps1 helper script that simplifies retrieving and visualizing trace spans from the Datadog API. The script supports three output formats: - Table: Formatted table with key span information - Hierarchy: Tree view showing parent-child relationships - JSON: Raw output for further processing Also update QueryingDatadogAPIs.md documentation with: - PowerShell script usage and examples - Response structure details - Shell variable substitution troubleshooting note 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude <noreply@anthropic.com>
1 parent 8325ae9 commit 0f9eb5a

File tree

2 files changed

+295
-0
lines changed

2 files changed

+295
-0
lines changed

docs/development/QueryingDatadogAPIs.md

Lines changed: 106 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -59,6 +59,7 @@ curl -s -X POST https://api.datadoghq.com/api/v2/spans/events/search \
5959
- The request body must be wrapped in `{"data": {"attributes": {...}, "type": "search_request"}}` structure
6060
- API keys can be obtained from https://app.datadoghq.com/organization-settings/api-keys
6161
- Full API documentation: https://docs.datadoghq.com/api/latest/spans/
62+
- **Shell variable substitution issue**: If using environment variables like `${DD_API_KEY}` fails with "Unauthorized" errors, try using the key values directly in the curl command instead of shell variable substitution. Some shells may have issues with variable expansion in curl headers.
6263

6364
## Common Use Cases
6465

@@ -79,6 +80,49 @@ The `query` field supports the Datadog query syntax:
7980
- Combine with boolean operators: `service:my-service AND env:prod`
8081
- Exclude with `-`: `service:my-service -resource_name:*ping*`
8182

83+
## Response Structure
84+
85+
The Spans API returns data in the following structure:
86+
87+
```json
88+
{
89+
"data": [
90+
{
91+
"id": "...",
92+
"type": "spans",
93+
"attributes": {
94+
"operation_name": "azure_functions.invoke",
95+
"resource_name": "GET /api/endpoint",
96+
"span_id": "4208934728885019856",
97+
"parent_id": "0",
98+
"trace_id": "690507fc00000000b882bcd2bdac6b9e",
99+
"start_timestamp": 1761937404333164200,
100+
"end_timestamp": 1761937404533785600,
101+
"status": "ok",
102+
"service": "service-name",
103+
"env": "environment",
104+
"tags": ["tag1:value1", "tag2:value2"],
105+
"custom": {
106+
"duration": 200621200,
107+
"aas": {
108+
"function": {
109+
"process": "host",
110+
"name": "HttpTest"
111+
}
112+
}
113+
}
114+
}
115+
}
116+
]
117+
}
118+
```
119+
120+
Key fields:
121+
- `attributes.span_id` and `attributes.parent_id` - Direct span relationship fields (not in `custom`)
122+
- `attributes.operation_name` and `attributes.resource_name` - At root of `attributes`
123+
- `attributes.custom.*` - Custom tags and metadata (including nested objects like `aas.function.process`)
124+
- `attributes.tags[]` - Array of tag strings in `key:value` format
125+
82126
## Example: Debugging Span Parenting
83127

84128
When investigating span parenting issues (e.g., verifying that child spans are correctly linked to parent spans):
@@ -130,6 +174,68 @@ When investigating span parenting issues (e.g., verifying that child spans are c
130174

131175
4. Verify the parent-child relationships by checking that each span's `parent_id` matches another span's `span_id` in the trace.
132176

177+
## PowerShell Helper Script
178+
179+
The repository includes a PowerShell script that simplifies querying traces from the Datadog API.
180+
181+
### Get-DatadogTrace.ps1
182+
183+
**Location**: `tracer/tools/Get-DatadogTrace.ps1`
184+
185+
This script retrieves all spans for a given trace ID and displays them in various formats.
186+
187+
**Prerequisites:**
188+
- Set environment variables: `DD_API_KEY` and `DD_APPLICATION_KEY`
189+
- API keys can be obtained from https://app.datadoghq.com/organization-settings/api-keys
190+
191+
**Basic usage:**
192+
```powershell
193+
.\tracer\tools\Get-DatadogTrace.ps1 -TraceId "690507fc00000000b882bcd2bdac6b9e"
194+
```
195+
196+
**Parameters:**
197+
- `-TraceId` (required) - The 128-bit trace ID in hex format
198+
- `-TimeRange` (optional) - How far back to search (default: "2h"). Examples: "15m", "1h", "2h", "1d"
199+
- `-OutputFormat` (optional) - Output format: "table" (default), "json", or "hierarchy"
200+
201+
**Output formats:**
202+
203+
1. **Table** (default) - Formatted table with key span information:
204+
```powershell
205+
.\tracer\tools\Get-DatadogTrace.ps1 -TraceId "690507fc00000000b882bcd2bdac6b9e"
206+
```
207+
Shows: Operation, Resource, Span ID, Parent ID, Process, Duration
208+
209+
2. **Hierarchy** - Tree view showing parent-child relationships:
210+
```powershell
211+
.\tracer\tools\Get-DatadogTrace.ps1 -TraceId "690507fc00000000b882bcd2bdac6b9e" -OutputFormat hierarchy
212+
```
213+
Useful for visualizing span nesting and verifying proper parent-child relationships
214+
215+
3. **JSON** - Raw JSON output for further processing:
216+
```powershell
217+
.\tracer\tools\Get-DatadogTrace.ps1 -TraceId "690507fc00000000b882bcd2bdac6b9e" -OutputFormat json
218+
```
219+
220+
**Example workflow:**
221+
222+
1. Get a recent trace ID from your service:
223+
```bash
224+
curl -s -X POST https://api.datadoghq.com/api/v2/spans/events/search \
225+
-H "DD-API-KEY: $DD_API_KEY" \
226+
-H "DD-APPLICATION-KEY: $DD_APPLICATION_KEY" \
227+
-H "Content-Type: application/json" \
228+
-d '{"data":{"attributes":{"filter":{"query":"service:my-service","from":"now-5m","to":"now"},"sort":"-timestamp","page":{"limit":1}},"type":"search_request"}}' \
229+
| jq -r '.data[0].attributes.trace_id'
230+
```
231+
232+
2. Analyze the full trace with the PowerShell script:
233+
```powershell
234+
.\tracer\tools\Get-DatadogTrace.ps1 -TraceId "<trace-id-from-step-1>" -OutputFormat hierarchy
235+
```
236+
237+
**Note**: The script uses `Invoke-RestMethod` which properly handles authentication headers, avoiding the shell variable substitution issues that can occur with curl.
238+
133239
## Logs Search API
134240

135241
### Search for logs with specific criteria

tracer/tools/Get-DatadogTrace.ps1

Lines changed: 189 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,189 @@
1+
<#
2+
.SYNOPSIS
3+
Retrieves all spans for a given trace ID from the Datadog API.
4+
5+
.DESCRIPTION
6+
Queries the Datadog Spans API to retrieve all spans belonging to a specific trace ID.
7+
Requires DD_API_KEY and DD_APPLICATION_KEY environment variables to be set.
8+
9+
.PARAMETER TraceId
10+
The 128-bit trace ID to query (hex string, e.g., "690507fc00000000b882bcd2bdac6b9e")
11+
12+
.PARAMETER TimeRange
13+
How far back to search for the trace. Defaults to "2h" (2 hours).
14+
Examples: "15m", "1h", "2h", "1d"
15+
16+
.PARAMETER OutputFormat
17+
Output format: "table" (default), "json", or "hierarchy"
18+
19+
.EXAMPLE
20+
.\Get-DatadogTrace.ps1 -TraceId "690507fc00000000b882bcd2bdac6b9e"
21+
22+
.EXAMPLE
23+
.\Get-DatadogTrace.ps1 -TraceId "690507fc00000000b882bcd2bdac6b9e" -OutputFormat hierarchy
24+
25+
.EXAMPLE
26+
.\Get-DatadogTrace.ps1 -TraceId "690507fc00000000b882bcd2bdac6b9e" -OutputFormat json | ConvertFrom-Json | ConvertTo-Json -Depth 10
27+
#>
28+
29+
[CmdletBinding()]
30+
param(
31+
[Parameter(Mandatory = $true, Position = 0)]
32+
[string]$TraceId,
33+
34+
[Parameter(Mandatory = $false)]
35+
[string]$TimeRange = "2h",
36+
37+
[Parameter(Mandatory = $false)]
38+
[ValidateSet("table", "json", "hierarchy")]
39+
[string]$OutputFormat = "table"
40+
)
41+
42+
# Check for required environment variables
43+
$apiKey = $env:DD_API_KEY
44+
$appKey = $env:DD_APPLICATION_KEY
45+
46+
if ([string]::IsNullOrEmpty($apiKey)) {
47+
Write-Error "DD_API_KEY environment variable is not set. Please set it before running this script."
48+
exit 1
49+
}
50+
51+
if ([string]::IsNullOrEmpty($appKey)) {
52+
Write-Error "DD_APPLICATION_KEY environment variable is not set. Please set it before running this script."
53+
exit 1
54+
}
55+
56+
# Build the request
57+
$uri = "https://api.datadoghq.com/api/v2/spans/events/search"
58+
$headers = @{
59+
"DD-API-KEY" = $apiKey
60+
"DD-APPLICATION-KEY" = $appKey
61+
"Content-Type" = "application/json"
62+
}
63+
64+
$body = @{
65+
data = @{
66+
attributes = @{
67+
filter = @{
68+
query = "trace_id:$TraceId"
69+
from = "now-$TimeRange"
70+
to = "now"
71+
}
72+
sort = "start_timestamp"
73+
page = @{
74+
limit = 100
75+
}
76+
}
77+
type = "search_request"
78+
}
79+
} | ConvertTo-Json -Depth 10
80+
81+
Write-Verbose "Querying Datadog API for trace ID: $TraceId"
82+
Write-Verbose "Time range: now-$TimeRange to now"
83+
84+
try {
85+
$response = Invoke-RestMethod -Uri $uri -Method Post -Headers $headers -Body $body -ErrorAction Stop
86+
87+
if ($null -eq $response.data -or $response.data.Count -eq 0) {
88+
Write-Warning "No spans found for trace ID: $TraceId"
89+
Write-Warning "Make sure the trace ID is correct and the trace is within the time range (now-$TimeRange to now)"
90+
exit 0
91+
}
92+
93+
$spans = $response.data | ForEach-Object {
94+
$attrs = $_.attributes
95+
96+
# Convert duration (nanoseconds to milliseconds)
97+
$durationMs = if ($attrs.custom.duration) {
98+
[math]::Round([double]$attrs.custom.duration / 1000000.0, 2)
99+
} else {
100+
0
101+
}
102+
103+
[PSCustomObject]@{
104+
SpanId = $attrs.span_id
105+
ParentId = $attrs.parent_id
106+
OperationName = $attrs.operation_name
107+
ResourceName = $attrs.resource_name
108+
Status = $attrs.status
109+
Duration = $durationMs
110+
Process = $attrs.custom.aas.function.process
111+
Service = $attrs.service
112+
}
113+
}
114+
115+
switch ($OutputFormat) {
116+
"table" {
117+
Write-Host "`nFound $($spans.Count) spans in trace:" -ForegroundColor Green
118+
Write-Host "Trace ID: $TraceId`n" -ForegroundColor Cyan
119+
120+
$spans | Format-Table -Property @(
121+
@{Label = "Operation"; Expression = { $_.OperationName }; Width = 25 }
122+
@{Label = "Resource"; Expression = { $_.ResourceName }; Width = 30 }
123+
@{Label = "Span ID"; Expression = { $_.SpanId }; Width = 20 }
124+
@{Label = "Parent ID"; Expression = { $_.ParentId }; Width = 20 }
125+
@{Label = "Process"; Expression = { $_.Process }; Width = 8 }
126+
@{Label = "Duration (ms)"; Expression = { $_.Duration }; Width = 12 }
127+
) -AutoSize
128+
}
129+
"json" {
130+
$spans | ConvertTo-Json -Depth 10
131+
}
132+
"hierarchy" {
133+
Write-Host "`nFound $($spans.Count) spans in trace:" -ForegroundColor Green
134+
Write-Host "Trace ID: $TraceId`n" -ForegroundColor Cyan
135+
136+
# Build hierarchy
137+
$spanMap = @{}
138+
foreach ($span in $spans) {
139+
$spanMap[$span.SpanId] = $span
140+
}
141+
142+
function Show-SpanHierarchy {
143+
param(
144+
[PSCustomObject]$Span,
145+
[int]$Indent = 0,
146+
[hashtable]$SpanMap,
147+
[PSCustomObject[]]$AllSpans
148+
)
149+
150+
$prefix = " " * $Indent
151+
$arrow = if ($Indent -gt 0) { "└─ " } else { "" }
152+
153+
$processTag = if ($Span.Process) { " [$($Span.Process)]" } else { "" }
154+
Write-Host "$prefix$arrow$($Span.OperationName)$processTag" -ForegroundColor Cyan
155+
Write-Host "$prefix Resource: $($Span.ResourceName)" -ForegroundColor Gray
156+
Write-Host "$prefix Span ID: $($Span.SpanId), Parent: $($Span.ParentId)" -ForegroundColor DarkGray
157+
Write-Host "$prefix Duration: $($Span.Duration) ms`n" -ForegroundColor DarkGray
158+
159+
# Find children
160+
$children = $AllSpans | Where-Object { $_.ParentId -eq $Span.SpanId }
161+
foreach ($child in $children) {
162+
Show-SpanHierarchy -Span $child -Indent ($Indent + 1) -SpanMap $SpanMap -AllSpans $AllSpans
163+
}
164+
}
165+
166+
# Find root span(s) (parent_id = 0)
167+
$rootSpans = $spans | Where-Object { $_.ParentId -eq "0" }
168+
169+
if ($rootSpans.Count -eq 0) {
170+
Write-Warning "No root span found (parent_id = 0). Showing all spans:"
171+
$spans | ForEach-Object {
172+
Show-SpanHierarchy -Span $_ -Indent 0 -SpanMap $spanMap -AllSpans $spans
173+
}
174+
} else {
175+
foreach ($root in $rootSpans) {
176+
Show-SpanHierarchy -Span $root -Indent 0 -SpanMap $spanMap -AllSpans $spans
177+
}
178+
}
179+
}
180+
}
181+
182+
} catch {
183+
Write-Error "Failed to query Datadog API: $_"
184+
Write-Error $_.Exception.Message
185+
if ($_.ErrorDetails.Message) {
186+
Write-Error "API Response: $($_.ErrorDetails.Message)"
187+
}
188+
exit 1
189+
}

0 commit comments

Comments
 (0)