Version Conflict Error with Snappy.net or any third party lib in OpenTelemetry Instrumented .NET Application

[CodingwithKetan]

Hi everyone,

I’m working on a custom OpenTelemetry implementation for my .NET application, where I’ve integrated Snappy.net to compress data before writing traces to a file. However, I’m encountering a version conflict issue when I try to instrument my application with OpenTelemetry while also using Snappy.net.

Has anyone else faced a similar issue when using Snappy.net alongside OpenTelemetry in a .NET project? If so, how did you resolve the version conflict? I would appreciate any insights or best practices to tackle this issue effectively.

Thanks in advance!

[Kielek]

Please provide Minimal, Reproducible Example.
Without this, it will be hard to help.

[CodingwithKetan]

I have modified the OpenTelemetry .NET SDK to export traces using the OTLP exporter. Instead of sending the trace data to an endpoint, I am compressing the data using Snappy.net (v1.3.2) and then writing the compressed data to a file. This works fine with applications that use Snappy.net 1.3.2, but I encounter a version conflict when trying to auto-instrument applications that use a different version of Snappy.net.

So my questing is in SDK, how we are able to manage conflict of third party library?

[Kielek]

I am asking to prepare code which we can investigate, eg. separate repository. I/We can spend time to try to do on our own, but I assume that you have it ready.

[CodingwithKetan]

here is modified code of SDK

// Copyright The OpenTelemetry Authors
// SPDX-License-Identifier: Apache-2.0

using System.Collections;
using System.Diagnostics;
using System.Net;
using System.Text;
**using Newtonsoft.Json.Linq;**
using OpenTelemetry.Exporter.OpenTelemetryProtocol.Implementation;
using OpenTelemetry.Exporter.OpenTelemetryProtocol.Implementation.Transmission;
using OtlpCollector = OpenTelemetry.Proto.Collector.Trace.V1;
using OtlpResource = OpenTelemetry.Proto.Resource.V1;

namespace OpenTelemetry.Exporter;

/// <summary>
/// Exporter consuming <see cref="Activity"/> and exporting the data using
/// the OpenTelemetry protocol (OTLP).
/// </summary>
public class OtlpTraceExporter : BaseExporter<Activity>
{
    private readonly SdkLimitOptions sdkLimitOptions;
    private readonly OtlpExporterTransmissionHandler<OtlpCollector.ExportTraceServiceRequest> transmissionHandler;

    private OtlpResource.Resource processResource;

    private static readonly string PATH_SEPARATOR = Path.DirectorySeparatorChar.ToString();

    private static readonly string AGENT_INSTALL_DIR = GetAgentDirectory();

    private static string DEFAULT_SERVICE_NAME = "unknown_service";

    private static string TRACE_FILE_FORMAT = "trace-{0}-{1}.cache";

    private string serviceName;


    private Timer timer;

    private volatile bool isShutdown;

    public static readonly string DATA_DIR = AGENT_INSTALL_DIR + "cache" + PATH_SEPARATOR;

    /// <summary>
    /// Initializes a new instance of the <see cref="OtlpTraceExporter"/> class.
    /// </summary>
    /// <param name="options">Configuration options for the export.</param>
    public OtlpTraceExporter(OtlpExporterOptions options)
        : this(options, sdkLimitOptions: new(), experimentalOptions: new(), transmissionHandler: null)
    {
    }

    /// <summary>
    /// Initializes a new instance of the <see cref="OtlpTraceExporter"/> class.
    /// </summary>
    /// <param name="exporterOptions"><see cref="OtlpExporterOptions"/>.</param>
    /// <param name="sdkLimitOptions"><see cref="SdkLimitOptions"/>.</param>
    /// <param name="experimentalOptions"><see cref="ExperimentalOptions"/>.</param>
    /// <param name="transmissionHandler"><see cref="OtlpExporterTransmissionHandler{T}"/>.</param>
    internal OtlpTraceExporter(
        OtlpExporterOptions exporterOptions,
        SdkLimitOptions sdkLimitOptions,
        ExperimentalOptions experimentalOptions,
        OtlpExporterTransmissionHandler<OtlpCollector.ExportTraceServiceRequest> transmissionHandler = null)
    {
        Debug.Assert(exporterOptions != null, "exporterOptions was null");
        Debug.Assert(sdkLimitOptions != null, "sdkLimitOptions was null");

        this.sdkLimitOptions = sdkLimitOptions;

        this.transmissionHandler = transmissionHandler ?? exporterOptions.GetTraceExportTransmissionHandler(experimentalOptions);
    }

    internal OtlpResource.Resource ProcessResource => this.processResource ??= this.ParentProvider.GetResource().ToOtlpResource();

    /// <inheritdoc/>
    public override ExportResult Export(in Batch<Activity> activityBatch)
    {
        Console.WriteLine("printing the path separator : " + PATH_SEPARATOR);
        // Prevents the exporter's gRPC and HTTP operations from being instrumented.
        using var scope = SuppressInstrumentationScope.Begin();

        var request = new OtlpCollector.ExportTraceServiceRequest();

        request.AddBatch(this.sdkLimitOptions, this.ProcessResource, activityBatch);

            try
            {
                Console.WriteLine("Exporting...");
                byte[] jsonBytes = Encoding.UTF8.GetBytes(request.ToString());
                Console.WriteLine($"Size before Compression : {jsonBytes.Length}");

                byte[] compressData = **IronSnappy.Snappy.Encode(jsonBytes);**

                Console.WriteLine($"size after compression: {compressData.Length}");


                byte[] reverseData = IronSnappy.Snappy.Decode(compressData);
                string json = Encoding.UTF8.GetString(reverseData);
                Console.WriteLine("After reverse engineering");
                Console.WriteLine(json);
                long timestamp = DateTimeOffset.UtcNow.ToUnixTimeMilliseconds();
                string fileName = string.Format(TRACE_FILE_FORMAT, this.serviceName, timestamp);
                string filePath = Path.Combine(DATA_DIR, fileName);

                // Write the compressed trace data to file.
                File.WriteAllBytes(filePath, compressData);

                Console.WriteLine($"Trace written to file: {filePath}");


                if (!this.transmissionHandler.TrySubmitRequest(request))
                {
                    return ExportResult.Failure;
                }
            }
            catch (Exception ex)
            {
                OpenTelemetryProtocolExporterEventSource.Log.ExportMethodException(ex);
                return ExportResult.Failure;
            }
            finally
            {
                request.Return();
            }
        }

        return ExportResult.Success;
    }

    /// <inheritdoc />
    protected override bool OnShutdown(int timeoutMilliseconds)
    {
        return this.transmissionHandler.Shutdown(timeoutMilliseconds);
    }

}

here is sample application that I am trying to instrument using auto instrument

using System;
using IronSnappy;
using System.Text;

namespace SnappyCompressionDemo
{
    class Program
    {
        static void Main(string[] args)
        {
            // Sample text to be compressed
            string originalText = "This is a simple example of using Snappy compression in .NET.";
            Console.WriteLine("Original Text: ");
            Console.WriteLine(originalText);
            
            // Convert string to byte array
            byte[] originalBytes = Encoding.UTF8.GetBytes(originalText);
            
            // Compress the byte array
            byte[] compressedBytes = Snappy.Compress(originalBytes);
            Console.WriteLine("\nCompressed Data (Base64): ");
            Console.WriteLine(Convert.ToBase64String(compressedBytes));
            
            // Decompress the byte array
            byte[] decompressedBytes = Snappy.Uncompress(compressedBytes);
            string decompressedText = Encoding.UTF8.GetString(decompressedBytes);
            
            Console.WriteLine("\nDecompressed Text: ");
            Console.WriteLine(decompressedText);
        }
    }
}

Now consider that in our SDK I am using IronSnappy(1.3.1) and in my console application I am using IronSnappy(1.3.0)

so while dumping the traces I am getting error IronSnappy.dll is not found with version 1.3.0 at location of where Otel's zip is extracted even though there is IronSnappy.dll but it is version as 1.3.1

same applies to Newtonsoft.Json.Linq;

[Kielek]

Please add also csproj file. It is crucial to have all dependecies.

[CodingwithKetan]

here is csproj of exporter

  <PropertyGroup>
    <TargetFrameworks>$(TargetFrameworksForLibrariesExtended)</TargetFrameworks>
    <Description>OpenTelemetry protocol exporter for OpenTelemetry .NET</Description>
    <PackageTags>$(PackageTags);OTLP</PackageTags>
    <MinVerTagPrefix>core-</MinVerTagPrefix>

    <!-- this is temporary. will remove in future PR. -->
    <Nullable>disable</Nullable>
    <DefineConstants>BUILDING_INTERNAL_PERSISTENT_STORAGE;$(DefineConstants)</DefineConstants>
    <EnableConfigurationBindingGenerator>true</EnableConfigurationBindingGenerator>

    <!-- Note: SYSLIB1100;SYSLIB1101 - Configuration.Binder: Can't create
    instance and unsupported type are being emitted by config source generator
    processing OtlpExporterBuilderOptions. It would be preferable to suppress
    this at the call site but there is a bug. This could possibly be cleaned up
    in the future (hopefully .NET 9) see:
    https://github.com/open-telemetry/opentelemetry-dotnet/pull/5520#discussion_r1556221048
    and https://github.com/dotnet/runtime/issues/92509 -->
    <NoWarn>$(NoWarn);SYSLIB1100;SYSLIB1101</NoWarn>
  </PropertyGroup>

  <ItemGroup>
    <ProjectReference Include="$(RepoRoot)\src\OpenTelemetry\OpenTelemetry.csproj" />
  </ItemGroup>

  <ItemGroup>
    <PackageReference Include="Grpc.Net.Client" Condition="'$(TargetFramework)' != 'netstandard2.0' AND '$(TargetFramework)' != '$(NetFrameworkMinimumSupportedVersion)'" />
    <PackageReference Include="Grpc" Condition="'$(TargetFramework)' == 'netstandard2.0' OR '$(TargetFramework)' == '$(NetFrameworkMinimumSupportedVersion)'" />
    <PackageReference Include="Google.Protobuf" />
    <PackageReference Include="Grpc.Tools" PrivateAssets="All" />
    <PackageReference Include="IronSnappy" />
    <PackageReference Include="Newtonsoft.Json"/>
    <PackageReference Include="Microsoft.Extensions.Configuration.Binder" />
  </ItemGroup>

  <ItemGroup>
    <Reference Include="System.Net.Http" Condition="'$(TargetFramework)' == '$(NetFrameworkMinimumSupportedVersion)'" />
  </ItemGroup>

  <ItemGroup>
    <Compile Include="$(RepoRoot)\src\Shared\PeriodicExportingMetricReaderHelper.cs" Link="Includes\PeriodicExportingMetricReaderHelper.cs" />
    <Compile Include="$(RepoRoot)\src\Shared\PeerServiceResolver.cs" Link="Includes\PeerServiceResolver.cs" />
    <Compile Include="$(RepoRoot)\src\Shared\TagWriter\ArrayTagWriter.cs" Link="Includes\TagWriter\ArrayTagWriter.cs" />
    <Compile Include="$(RepoRoot)\src\Shared\TagWriter\TagWriter.cs" Link="Includes\TagWriter\TagWriter.cs" />
  </ItemGroup>

  <ItemGroup>
    <Protobuf Include="**/*.proto" Access="internal">
      <ProtoRoot>Implementation</ProtoRoot>
    </Protobuf>
  </ItemGroup>

</Project>

here is Directory.Packages.props

<Project>
  <PropertyGroup>
    <ManagePackageVersionsCentrally>true</ManagePackageVersionsCentrally>
    <OTelLatestStableVer>1.8.1</OTelLatestStableVer>
  </PropertyGroup>
  <!--
      This section covers packages that are directly referenced by the NuGet packages published from this repository.
      Any security vulnerability in these packages or their downstream dependencies will be considered as a security
      vulnerability in the NuGet packages that are published from this repository.
  -->
  <ItemGroup>
    <PackageVersion Include="Google.Protobuf" Version="[3.22.5,4.0)" />
    <PackageVersion Include="Grpc" Version="[2.44.0,3.0)" />
    <PackageVersion Include="Grpc.Net.Client" Version="[2.52.0,3.0)" />
    <PackageVersion Include="IronSnappy" Version="1.3.1" />
    <PackageReference Include="Newtonsoft.Json" Version="13.0.1" />
    <PackageVersion Include="Microsoft.AspNetCore.Http.Abstractions" Version="[2.1.1,6.0)" />
    <PackageVersion Include="Microsoft.AspNetCore.Http.Features" Version="[2.1.1,6.0)" />
    <!--
        Typically, for the Microsoft.Extensions.* packages relating to DI Abstractions, Hosting Abstractions, and Logging,
        the latest stable version should be used because:
        1) Each major version bump will have some new API capabilities (e.g.For Logging, .NET 6 introduced compile-time logging
          source generation, .NET 8 introduced automatic event id generation).
        2) Each minor version bump is normally security hotfixes or critical bug fixes.
        3) Since version 3.1.0, the .NET runtime team is holding a high bar for backward compatibility on
          these packages even during major version bumps, so compatibility is not a concern here.
    -->
    <PackageVersion Include="Microsoft.Extensions.Configuration" Version="8.0.0" />
    <PackageVersion Include="Microsoft.Extensions.Configuration.Binder" Version="8.0.1" />
    <PackageVersion Include="Microsoft.Extensions.DependencyInjection.Abstractions" Version="8.0.0" />
    <PackageVersion Include="Microsoft.Extensions.Diagnostics.Abstractions" Version="8.0.0" />
    <PackageVersion Include="Microsoft.Extensions.Hosting.Abstractions" Version="8.0.0" />
    <PackageVersion Include="Microsoft.Extensions.Logging" Version="8.0.0" />
    <PackageVersion Include="Microsoft.Extensions.Logging.Configuration" Version="8.0.0" />
    <PackageVersion Include="Microsoft.Extensions.Options" Version="8.0.0" />
    <PackageVersion Include="OpenTelemetry" Version="$(OTelLatestStableVer)" />
    <PackageVersion Include="OpenTelemetry.Api" Version="$(OTelLatestStableVer)" />
    <PackageVersion Include="OpenTelemetry.Api.ProviderBuilderExtensions" Version="$(OTelLatestStableVer)" />
    <PackageVersion Include="OpenTelemetry.Exporter.InMemory" Version="$(OTelLatestStableVer)" />
    <PackageVersion Include="OpenTelemetry.Extensions.Hosting" Version="$(OTelLatestStableVer)" />
    <PackageVersion Include="OpenTelemetry.Extensions.Propagators" Version="$(OTelLatestStableVer)" />
    <PackageVersion Include="OpenTracing" Version="[0.12.1,0.13)" />
    <!--
        Typically, the latest stable version of System.Diagnostics.DiagnosticSource should be used here because:
        1) Each major version bump will have some new OpenTelemetry API capabilities (e.g. .NET 6 introduced Meter
          API, .NET 7 added UpDownCounter, .NET 8 is adding Meter/Instrument level attributes support, .NET 9 might
          add Advice/Hint API) that the OpenTelemetry components rely on.
        2) Each minor version bump is normally security hotfixes or critical bug fixes.
        3) The .NET runtime team provides extra backward compatibility guarantee to System.Diagnostics.DiagnosticSource
          even during major version bumps, so compatibility is not a concern here.
    -->
    <PackageVersion Include="System.Diagnostics.DiagnosticSource" Version="8.0.0" />
    <!-- A conservative version of System.Text.Encodings.Web must be used here since there is no backward compatibility guarantee during major version bumps. -->
    <PackageVersion Include="System.Text.Encodings.Web" Version="4.7.2" />
    <!-- A conservative version of System.Text.Json must be used here since there is no backward compatibility guarantee during major version bumps. -->
    <PackageVersion Include="System.Text.Json" Version="4.7.2" />
    <!-- A conservative version of System.Threading.Tasks.Extensions must be used here since there is no backward compatibility guarantee during major version bumps. -->
    <PackageVersion Include="System.Threading.Tasks.Extensions" Version="4.5.4" />
  </ItemGroup>
  <!--
      This section covers packages that are **not** directly referenced by the NuGet packages published from this repository.
      For example, these packages are used in the tests, examples or referenced as "PrivateAssets", but not in the NuGet packages themselves.
  -->
  <!-- 'net8.0' is the default `TargetFramework`. Use `VersionOverride` in the project to override the package versions from a different `TargetFramework` -->
  <ItemGroup>
    <PackageVersion Include="BenchmarkDotNet" Version="[0.13.12,0.14)" />
    <PackageVersion Include="CommandLineParser" Version="[2.9.1,3.0)" />
    <PackageVersion Include="Grpc.AspNetCore" Version="[2.59.0,3.0)" />
    <PackageVersion Include="Grpc.AspNetCore.Server" Version="[2.59.0, 3.0)" />
    <PackageVersion Include="Grpc.Tools" Version="[2.59.0,3.0)" />
    <PackageVersion Include="Microsoft.CSharp" Version="[4.7.0]" />
    <PackageVersion Include="Microsoft.CodeAnalysis.PublicApiAnalyzers" Version="[3.11.0-beta1.23525.2]" />
    <PackageVersion Include="Microsoft.Extensions.DependencyInjection" Version="[8.0.0,)" />
    <PackageVersion Include="Microsoft.Extensions.Hosting" Version="[8.0.0,)" />
    <PackageVersion Include="Microsoft.Extensions.Http" Version="[8.0.0,)" />
    <PackageVersion Include="Microsoft.Extensions.Logging.Abstractions" Version="[8.0.0,)" />
    <PackageVersion Include="Microsoft.Extensions.Telemetry.Abstractions" Version="[8.6.0,)" />
    <PackageVersion Include="Microsoft.NETFramework.ReferenceAssemblies" Version="[1.0.3,2.0)" />
    <PackageVersion Include="Microsoft.NET.Test.Sdk" Version="[17.10.0,18.0.0)" />
    <PackageVersion Include="Microsoft.SourceLink.GitHub" Version="[1.1.1,2.0)" />
    <PackageVersion Include="MinVer" Version="[5.0.0,6.0)" />
    <PackageVersion Include="OpenTelemetry.Instrumentation.AspNetCore" Version="[1.8.1,2.0)" />
    <PackageVersion Include="OpenTelemetry.Instrumentation.GrpcNetClient" Version="[1.8.0-beta.1,2.0)" />
    <PackageVersion Include="OpenTelemetry.Instrumentation.Http" Version="[1.8.1,2.0)" />
    <PackageVersion Include="OpenTelemetry.Instrumentation.Runtime" Version="[1.8.1,2.0)" />
    <PackageVersion Include="RabbitMQ.Client" Version="[6.8.1,7.0)" />
    <PackageVersion Include="StyleCop.Analyzers" Version="[1.2.0-beta.556,2.0)" />
    <PackageVersion Include="Swashbuckle.AspNetCore" Version="[6.6.2,)" />
    <PackageVersion Include="xunit" Version="[2.8.1,3.0)" />
    <PackageVersion Include="xunit.runner.visualstudio" Version="[2.8.1,3.0)" />
  </ItemGroup>
  <ItemGroup Condition="'$(TargetFramework)' == 'net6.0'">
    <PackageVersion Include="Microsoft.AspNetCore.TestHost" Version="6.0.31" />
  </ItemGroup>
  <ItemGroup Condition="'$(TargetFramework)' == 'net7.0'">
    <PackageVersion Include="Microsoft.AspNetCore.TestHost" Version="7.0.20" />
  </ItemGroup>
  <ItemGroup Condition="'$(TargetFramework)' == 'net8.0'">
    <PackageVersion Include="Microsoft.AspNetCore.TestHost" Version="8.0.6" />
  </ItemGroup>
</Project>

here is csproj file for console application

<Project Sdk="Microsoft.NET.Sdk">

  <PropertyGroup>
    <TargetFramework>net6.0</TargetFramework>
    <RootNamespace>SnappyCompressionDemo</RootNamespace>
  </PropertyGroup>

  <ItemGroup>
    <PackageReference Include="Iron.Snappy" Version="1.3.0" />
  </ItemGroup>

</Project>

[Kielek]

If the goal is to use AutoInstrumentation please go through troubleshooting docs especially consider referencing nuget package.

Other parts related to SnappyCompressionDemo.

1. It is not compiling - Iron.Snappy is not available in the nuget org.
2. It is compiled against .NET6. This version is not supported by any new OTel version/.NET SDK team. Probably it is also vulnerable as there were couple of .NET8 releases containing security fixes. Consider updating your framework.

[CodingwithKetan]

It is not compiling - Iron.Snappy is not available in the nuget org.
My bad here is updated

<Project Sdk="Microsoft.NET.Sdk">

  <PropertyGroup>
    <TargetFramework>net6.0</TargetFramework>
    <RootNamespace>SnappyCompressionDemo</RootNamespace>
  </PropertyGroup>

  <ItemGroup>
    <PackageReference Include="IronSnappy" Version="1.3.0" />
  </ItemGroup>

</Project>

[CodingwithKetan]

It is compiled against .NET6. This version is not supported by any new OTel version/.NET SDK team. Probably it is also vulnerable as there were couple of .NET8 releases containing security fixes. Consider updating your framework.

I am using version 1.9.0, as for the safer side If I am using net8.0 or later then also I am getting same error.

[Kielek]

My recommendation will be the same, go with OTel .NET Auto nuget package.

[CodingwithKetan]

@Kielek thank you very much for help!!

If possible could you please let me as we are using Google.Protobuf at SDK and let's assume that application that we are going to instrument it also using Google.Protobuf but with different version, at that same issue should be occured, Right?

if yes and how we are able to resolve it?, because somehow I am not able to use nuget package.

[Kielek]

Bump to the newer version of this package. When you reference AutoInstrumentation by nuget, it should be done automatically.
If only protobuf package is a problem, please check the latest release. This dependency was removed.

[CodingwithKetan]

Once again Thank you @Kielek!
I think I need to find solution for same.