[client][test] Add OTel key count metric

[lluwm]

Problem Statement

Add key_count metrics in OTel for all clients.

Solution

This change adds the following OTel metrics:

venice.thin_client.key_count
venice.fast_client.key_count
venice.davinci_client.key_count

The existing tehuti metric e.g. success_request_key_count metric will still be emitted along with the key_count metrics in OTel.

Code changes

• Added new code behind a config. If so list the config names and their default values in the PR description.
• Introduced new log lines.
  • Confirmed if logs need to be rate limited to avoid excessive logging.

Concurrency-Specific Checks

Both reviewer and PR author to verify

• Code has no race conditions or thread safety issues.
• Proper synchronization mechanisms (e.g., synchronized, RWLock) are used where needed.
• No blocking calls inside critical sections that could lead to deadlocks or performance degradation.
• Verified thread-safe collections are used (e.g., ConcurrentHashMap, CopyOnWriteArrayList).
• Validated proper exception handling in multi-threaded code to avoid silent thread termination.

How was this PR tested?

• New unit tests added.
• New integration tests added.
• Modified or extended existing tests.
• Verified backward compatibility (if applicable).

Does this PR introduce any user-facing or breaking changes?

• No. You can skip the rest of this section.
• Yes. Clearly explain the behavior change and its impact.

[m-nagarajan]

Thanks @lluwm. Leaving some nit comments until we decide on the major changes.

[m-nagarajan]

Thanks for iterating. Left some comments.

[m-nagarajan]

LGTM overall. Left One minor comment.

[m-nagarajan]

Thanks @lluwm