Instrumentation Metadata System

[jaydeluca]

This is a meta issue, see full Project board here

Is your feature request related to a problem? Please describe.

The java instrumentation project currently includes 250 individual instrumentations. There are 242 javaagent instrumentations - 15 of which currently have readmes, and 59 library instrumentations - 35 of which currently have readmes. These readmes can have varying depth in terms of their contents. Going through one by one and creating and updating documentation for each instrumentation manually would require considerable effort and toil.

Describe the solution you'd like

Having a system to manage and generate meta data about each instrumentation can unlock automated documentation as well as unlock other tooling capabilities. This information will give users much better insight into what they can expect when using the instrumentation, as well as more information around changes between releases.

We will establish a standard set of metadata we want to track for each instrumentation, and design a system that leverages automated gathering of information that is then augmented by some human maintained metadata file per instrumentation to generate an instrumentation list yaml file that can then be used to feed other workflows.

Metadata that would be useful to have available:

• Classification of the instrumentation
  • library - instrumentation for particular libraries
  • internal - instrumentation used internally within the agent
  • custom - instrumentation associated with supporting or generating custom instrumentation
• Some description of what an instrumentation provides
  • Ability add descriptions via the metadata.yaml file
  • Descriptions added for all modules
• A breakdown of the library versions that are supported, broken down by javaagent vs library support
• Minimum Java version supported (if not 8+)
• Whether the instrumentation is enabled or disabled by default
• The key to use to disable/enable
• The configuration options and defaults
• Scope information
  • name
  • schemaUrl
  • attributes
• Semantic conventions
• Span attributes
• Metrics

There are various ways we can attempt to obtain some of this information, and for other pieces we will create a metadata.yaml file for each instrumentation, similar to the collector

Ideas

Things to explore:

• Regex parsing gradle files for dependency information
  • Initial implementation: Automatically Generate Instrumentation Documentation #13449
• Creating metadata.yaml files for each instrumentation to augment the other information obtained by automation
  • Initial Implementation: Add support for basic metadata.yaml file #13480
• Parsing/capturing test activity / output for span and metric information
  • Initial Impelementation: Collect Scope, Span data, and metrics from test runs #13505
  • Metric implementation: Intercept metrics in test runner #13891, Differentiate emitted metrics by configuration options #14018, Collect metrics from more http and database clients #14038
  • Span Implementation Intercept and collect spans in test runner #14065
• We could create a new abstraction around configuration options where we use code generation to manage them
• How to differentiate the telemetry emitted by different configuration options
  • Ended up using a when indicator, implemented in Differentiate emitted metrics by configuration options #14018
• Can this system generate and/or document the spring configuration metadata file

Other notes:

• For target versions:
  • can we include just the primary target somehow?
  • we could potentially find the upperbound via the latestDepTestLibrary declaration
    • Parse latestDepTestLibrary and use as library upper bound #13478
  • Should we also include skipped versions?
  • We will want a separate way to handle non-standard instrumentation (methods, annotations, internal etc)

Additional context

If we complete this issue, I believe it will also solve the following:

• Instrumentation documentation for users #594
• Documentation for library instrumentations #6947
• Document available configuration #342

---

See latest instrument list output here

[jaydeluca]

Capturing input from @jackshirazi around scope information:

Having all of the fields from the InstrumentationScopeInfo object available to be able to re-construct it is useful because it is used as the key in a map to the tracer, and if you want to let users dynamically adjust the scope, that's how they would access it.