feat: add observability CLI for querying and visualizing agent traces #310
+5,135
−0
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Add comprehensive observability CLI (agentcore obs) for querying CloudWatch spans, traces, and runtime logs with rich terminal visualization. Commands Added: agentcore obs show # Show latest trace from config session agentcore obs show --last N # Show Nth most recent trace agentcore obs show --trace-id <id> # Show specific trace with full details agentcore obs show --session-id <id> # Show session summary table agentcore obs show --all # Show all traces in session agentcore obs show --errors # Show only failed traces agentcore obs show --simple # Minimal view without verbose metadata agentcore obs show -o trace.json # Export to JSON agentcore obs list # List all traces in a table Architecture: - ObservabilityClient: CloudWatch Logs Insights query execution - CloudWatchQueryBuilder: Query construction with camelCase fields (traceId, spanId) - Telemetry Models: Span/RuntimeLog/TraceData with hierarchy building - TraceVisualizer: Rich terminal rendering with Tree and Table displays Test Coverage: - 115 comprehensive unit tests (0.68s execution) - ObservabilityClient (18): AWS integration, batch queries, error handling - CloudWatchQueryBuilder (25): Query construction, field naming - Telemetry Models (27): Parsing, hierarchy building, serialization - TraceVisualizer (45): Rendering, formatting, deduplication TODO: Add Python/notebook interface for observability 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude <noreply@anthropic.com>
Add 29 parameterized unit tests for CLI observability commands to improve coverage from 7.69% to significantly higher. Tests Added: - Helper functions (15 tests): - _get_default_time_range: time range calculation with various days - _get_agent_config_from_file: config loading, missing fields, errors - _create_observability_client: CLI args, config file, error handling - _export_trace_data_to_json: successful export and error handling - show command (8 tests): - Trace ID and session ID flows - Error validation (both IDs, incompatible flags) - Config file session handling - Exception handling - list_traces command (6 tests): - Session ID and config file flows - Empty results handling - Error filtering - Exception handling Total Test Count: 144 tests (115 operations + 29 CLI) All tests passing in 0.61s 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude <noreply@anthropic.com>
Added comprehensive test coverage for observability modules: - CLI commands: Added 40 tests covering helper functions, show/list commands, and internal view functions - Telemetry models: Added 13 edge case tests for RuntimeLog and Span parsing - TraceVisualizer: Added 8 verbose formatting tests for messages/events/exceptions Coverage improvements: - trace_visualizer.py: 67% → 89% (+22%) - telemetry.py: 78% → 88% (+10%) - Overall: 84% → 90.12% Total tests: 1226 passed 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude <noreply@anthropic.com>
|
we should expand this to support all primitives and not only runtime |
|
agentcore obs --help doesn't show how to list all sessions and select a session |
| return None | ||
|
|
||
| event_name = attributes.get("event.name", "") | ||
| if not event_name.startswith("gen_ai."): |
There was a problem hiding this comment.
This filter appears to be specific to Bedrock LLM generated telemetry. Any reason we do not want to process other agent generated messages such as invocations, event loops, etc?
|
|
||
| # Extract role from event name | ||
| role = None | ||
| if "system.message" in event_name: |
There was a problem hiding this comment.
appears to be tailored to Bedrock.
|
are there plans to display span event body? currently only span chain is printed. Looks great. 
|
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
3 participants
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Summary
Add comprehensive observability CLI (
agentcore obs) for querying CloudWatch spans, traces, and runtime logs with rich terminal visualization.Commands Added
Architecture
Features
Test Coverage
ObservabilityClient(18 tests): AWS integration, batch queries, error handlingCloudWatchQueryBuilder(25 tests): Query construction, field namingTelemetry Models(27 tests): Parsing, hierarchy building, serializationTraceVisualizer(45 tests): Rendering, formatting, deduplicationTODO
Test Plan