after autoinstrumention FileNotFoundException: Could not load file or assembly 'Microsoft.Extensions.DependencyInjection.Abstractions

[soulmate1314]

The program based on dotnet 6.0 can be started normally without access. The following exception is reported after access

localhost:/home/Blog.Core/.PublishFiles# . /root/.otel-dotnet-auto/instrument.sh
localhost:/home/Blog.Core/.PublishFiles# OTEL_SERVICE_NAME=servicepatrol-configservice OTEL_TRACES_EXPORTER=otlp OTEL_METRICS_EXPORTER=otlp OTEL_LOGS_EXPORTER=otlp OTEL_EXPORTER_OTLP_ENDPOINT=http://localhost:4317 OTEL_EXPORTER_OTLP_PROTOCOL=grpc OTEL_RESOURCE_ATTRIBUTES=service.name=servicepatrol-configservice ./Blog.Core.Api
Unhandled exception. System.IO.FileNotFoundException: Could not load file or assembly 'Microsoft.Extensions.DependencyInjection.Abstractions, Version=7.0.0.0, Culture=neutral, PublicKeyToken=adb9793829ddae60'. The system cannot find the file specified.

File name: 'Microsoft.Extensions.DependencyInjection.Abstractions, Version=7.0.0.0, Culture=neutral, PublicKeyToken=adb9793829ddae60'
at Microsoft.AspNetCore.Builder.WebApplicationBuilder..ctor(WebApplicationOptions options, Action`1 configureDefaults)
at Microsoft.AspNetCore.Builder.WebApplication.CreateBuilder(String[] args)
at Program.

$(String[] args) in /home/Blog.Core/Blog.Core.Api/Program.cs:line 26
Aborted

[Kielek]

@soulmate1314,
could you please provide us at least

• AutoInstrumetnation version?
• verify if you application is not referencing (directly or indirectly) any ohter/older versions of Microsoft.Extensions.DependencyInjection.Abstractions.

[soulmate1314]

AutoInstrumetnation version: https://github.com/open-telemetry/opentelemetry-dotnet-instrumentation/releases/tag/v0.6.0-beta.2
Operating system ：centos7

No version of ‘Microsoft.Extensions.DependencyInjection.Abstractions’ is referenced in the project

Then I use a simple web application developed based on dotnet 6.0 and asp. net to access the instrument on the same centos7 machine, and there is no exception

[Kielek]

@soulmate1314, could you please share with us what was the issue?

[soulmate1314]

@soulmate1314, could you please share with us what was the issue?
The problem is that in the centos7 and dotnet6 environments, I Instrumentation the Blog.Core.Api application
, and then reported the following errors

localhost:/home/Blog.Core/.PublishFiles# . $home/.otel-dotnet-auto/instrument.sh
localhost:/home/Blog.Core/.PublishFiles# OTEL_SERVICE_NAME=servicepatrol-configservice OTEL_TRACES_EXPORTER=otlp OTEL_METRICS_EXPORTER=otlp OTEL_LOGS_EXPORTER=otlp OTEL_EXPORTER_OTLP_ENDPOINT=http://localhost:4317 OTEL_EXPORTER_OTLP_PROTOCOL=grpc OTEL_RESOURCE_ATTRIBUTES=service.name=servicepatrol-configservice ./Blog.Core.Api
Unhandled exception. System.IO.FileNotFoundException: Could not load file or assembly 'Microsoft.Extensions.DependencyInjection.Abstractions, Version=7.0.0.0, Culture=neutral, PublicKeyToken=adb9793829ddae60'. The system cannot find the file specified.

File name: 'Microsoft.Extensions.DependencyInjection.Abstractions, Version=7.0.0.0, Culture=neutral, PublicKeyToken=adb9793829ddae60'
at Microsoft.AspNetCore.Builder.WebApplicationBuilder..ctor(WebApplicationOptions options, Action`1 configureDefaults)
at Microsoft.AspNetCore.Builder.WebApplication.CreateBuilder(String[] args)
at Program.

$(String[] args) in /home/Blog.Core/Blog.Core.Api/Program.cs:line 26
Aborted

[soulmate1314]

@soulmate1314, could you please share with us what was the issue?

Hello, could you please explain this bug

[Kielek]

@soulmate1314, you have decided to close the issue. I assume that you have solved it somehow. Can you share the steps?

If you need any help, could you please be more descriptive? Pleas create Minimal, Reproducible Example.

[pellared]

hey @soulmate1314 👋

Then I use a simple web application developed based on dotnet 6.0 and asp. net to access the instrument on the same centos7 machine, and there is no exception

I do not follow.

Can you please share with us a minimal application that you are trying to instrument? Is dotnet new web enough to reproduce the issue? Is the CentOS 7 inside a container?

[soulmate1314]

@pellared @Kielek

AutoInstrumetnation version: [v0.6.0-beta.2]
operating system : centos7
dotnet version: 6.0
application ：https://github.com/anjoy8/Blog.Core

Sorry, it may have been accidentally closed.

1. First I used the automatic access script on the centos7 machine

. $home/.otel-dotnet-auto/instrument.sh

2. I started the application through the following startup parameters

OTEL_SERVICE_NAME=servicepatrol-configservice OTEL_TRACES_EXPORTER=otlp OTEL_METRICS_EXPORTER=otlp OTEL_LOGS_EXPORTER=otlp OTEL_EXPORTER_OTLP_ENDPOINT=http://localhost:4317/ OTEL_EXPORTER_OTLP_PROTOCOL=grpc OTEL_RESOURCE_ATTRIBUTES=service.name=servicepatrol-configservice ./Blog.Core.Api

3.Then I encountered these errors

`Unhandled exception. System.IO.FileNotFoundException: Could not load file or assembly 'Microsoft.Extensions.DependencyInjection.Abstractions, Version=7.0.0.0, Culture=neutral, PublicKeyToken=adb9793829ddae60'. The system cannot find the file specified.

File name: 'Microsoft.Extensions.DependencyInjection.Abstractions, Version=7.0.0.0, Culture=neutral, PublicKeyToken=adb9793829ddae60'
at Microsoft.AspNetCore.Builder.WebApplicationBuilder..ctor(WebApplicationOptions options, Action`1 configureDefaults)
at Microsoft.AspNetCore.Builder.WebApplication.CreateBuilder(String[] args)
at Program.

$(String[] args) in /home/Blog.Core/Blog.Core.Api/Program.cs:line 26
Aborted`

[pellared]

@soulmate1314 the information you have provided is not enough the reproduce the bug.

PTAL #2269 (comment)

[pjanotti]

@soulmate1314 if getting a minimal repro is hard, could you please capture the corehost logs? This may give us a better clue of what is happening in your app. Please, take a look here for instructions on how to capture such logs.

[soulmate1314]

@pjanotti @pellared @Kielek
hello，The following contains the source code and steps for reproduction the exception

environment

AutoInstrumetnation version: [v0.6.0-beta.2]
operating system : centos7
dotnet version: 6.0

Bug recurrence process

#go to the home directory
cd /home
#clone test code
git clone https://github.com/anjoy8/Blog.Core
#update build script permission
chmod u=rwx Blog.Core.Publish.Linux.sh
#build the application
./Blog.Core.Publish.Linux.sh
#go to the build directory
cd .PublishFiles/
#excute AutoInstrumetnation script
. $home/.otel-dotnet-auto/instrument.sh
#start the application with otlp parameters
OTEL_SERVICE_NAME=servicepatrol-configservice OTEL_TRACES_EXPORTER=otlp OTEL_METRICS_EXPORTER=otlp OTEL_LOGS_EXPORTER=otlp OTEL_EXPORTER_OTLP_ENDPOINT=http://localhost:4317/ OTEL_EXPORTER_OTLP_PROTOCOL=grpc OTEL_RESOURCE_ATTRIBUTES=service.name=servicepatrol-configservice COREHOST_TRACE=1 COREHOST_TRACEFILE=/root/corehost_verbose_tracing.log ./Blog.Core.Api

The execution commands and parameters are shown in the figure below

The following is the corehost logs

https://github.com/soulmate1314/issue-logs/blob/main/corehost_verbose_tracing.log

[Kielek]

I was able to reproduce issue on docker image.

Dockerfile

FROM centos:centos7.9.2009

RUN rpm -Uvh https://packages.microsoft.com/config/centos/7/packages-microsoft-prod.rpm

RUN yum update -y \
    && yum -y install git \
    # unzip needed for our otel script 
    && yum -y install unzip \
    && yum -y install dotnet-sdk-6.0

# Download the bash script
RUN curl -sSfL https://raw.githubusercontent.com/open-telemetry/opentelemetry-dotnet-instrumentation/v0.6.0/otel-dotnet-auto-install.sh -O

# Install core files
RUN sh ./otel-dotnet-auto-install.sh

RUN git clone https://github.com/anjoy8/Blog.Core \
&& cd Blog.Core \
&& chmod u=rwx Blog.Core.Publish.Linux.sh \
&& ./Blog.Core.Publish.Linux.sh

then I have executed following methods in interactive commands:

# Log4Net.cofnig needed to execute /home/Blog.Core/Blog.Core.Api/bin/Debug/.PublishFiles/Blog.Core.Api without instrumentation
 touch /home/Blog.Core/Blog.Core.Api/bin/Debug/.PublishFiles/Log4Net.config 
. $HOME/.otel-dotnet-auto/instrument.sh
OTEL_SERVICE_NAME=servicepatrol-configservice OTEL_TRACES_EXPORTER=otlp OTEL_METRICS_EXPORTER=otlp OTEL_LOGS_EXPORTER=otlp OTEL_EXPORTER_OTLP_ENDPOINT=http://localhost:4317/ OTEL_EXPORTER_OTLP_PROTOCOL=grpc OTEL_RESOURCE_ATTRIBUTES=service.name=servicepatrol-configservice COREHOST_TRACE=1 COREHOST_TRACEFILE=/root/corehost_verbose_tracing.log /home/Blog.Core/Blog.Core.Api/bin/Debug/.PublishFiles/Blog.Core.Api

It leads to

Unhandled exception. System.IO.FileNotFoundException: Could not load file or assembly 'Microsoft.Extensions.DependencyInjection.Abstractions, Version=7.0.0.0, Culture=neutral, PublicKeyToken=adb9793829ddae60'. The system cannot find the file specified.

File name: 'Microsoft.Extensions.DependencyInjection.Abstractions, Version=7.0.0.0, Culture=neutral, PublicKeyToken=adb9793829ddae60'
   at Microsoft.AspNetCore.Builder.WebApplicationBuilder..ctor(WebApplicationOptions options, Action`1 configureDefaults)
   at Microsoft.AspNetCore.Builder.WebApplication.CreateBuilder(String[] args)
   at Program.<Main>$(String[] args) in /Blog.Core/Blog.Core.Api/Program.cs:line 26

The reason:
/home/Blog.Core/Blog.Core.Api/bin/Debug/.PublishFiles/ contains Microsoft.Extensions.DependencyInjection.Abstractions.dll v 6.0.0.

From my first comment:

verify if you application is not referencing (directly or indirectly) any ohter/older versions of Microsoft.Extensions.DependencyInjection.Abstractions.

Your answer:

No version of ‘Microsoft.Extensions.DependencyInjection.Abstractions’ is referenced in the project

For sure. You have some indirect reference to this library.

What you can do is to add direct reference to Microsoft.Extensions.DependencyInjection.Abstractions v 7.0.0 on Blog.Core.Api.csproj.

[soulmate1314]

@Kielek Thank you for your patient help. I did solve that exception according to your method, but I encountered a new problem, which is also an exception that the package could not find. What is the root cause of this problem? Is there a version conflict between the project package and the openteletry package?

[Kielek]

Exactly, all of them should be solved in the same way.
We are working on nuget package which should make the process easier. See #2286

[pjanotti]

(sorry, I wrote this comment yesteday, after seeing @soulmate1314 response at #2269 (comment) - now it will be a bit out of order - EDIT: I was able to try with Piotr's repro and my suggestion below doesn't have any effect)

Thanks @soulmate1314, at first glance, it is looking like the issue is due to your app allowing only minor version roll-forward and a recent change intended to reduce the shared store size. To verify this possibility could you please allow a major version roll-forward on your app to see if that resolves the issue? You don't need to rebuild your app for that, you can use an environment variable to control the app behavior, PTAL at https://learn.microsoft.com/en-us/dotnet/core/versions/selection#control-roll-forward-behavior

Anyway, I hope to take a deeper look tomorrow, it seems that we may have to revert change #2223: I don't like the idea of requiring apps to change their roll-forward policy.

[pjanotti]

As @Kielek noticed there is an indirect reference to Microsoft.Extensions.DependencyInjection.Abstractions:

     "Autofac.Extensions.DependencyInjection/8.0.0": {
        "type": "package",
        "dependencies": {
          "Autofac": "6.4.0",
          "Microsoft.Extensions.DependencyInjection.Abstractions": "6.0.0"
        },

This causes the assembly to be added to the bin folder of the application. The AdditionalDeps bumps it up to version 7.0.0. However, since the assembly is in the folder of the application, version 6.0.0 gets added to the list of trusted assemblies.

Processing TPA for deps entry [Microsoft.Extensions.DependencyInjection.Abstractions, 7.0.0, lib/net7.0/Microsoft.Extensions.DependencyInjection.Abstractions.dll]
  Considering entry [Microsoft.Extensions.DependencyInjection.Abstractions/7.0.0/lib/net7.0/Microsoft.Extensions.DependencyInjection.Abstractions.dll], probe dir [], probe fx level:0, entry fx level:0
    Local path query exists /home/Blog.Core/Blog.Core.Api/bin/Debug/.PublishFiles/Microsoft.Extensions.DependencyInjection.Abstractions.dll
    Probed deps dir and matched '/home/Blog.Core/Blog.Core.Api/bin/Debug/.PublishFiles/Microsoft.Extensions.DependencyInjection.Abstractions.dll'
Adding tpa entry: /home/Blog.Core/Blog.Core.Api/bin/Debug/.PublishFiles/Microsoft.Extensions.DependencyInjection.Abstractions.dll, AssemblyVersion: 7.0.0.0, FileVersion: 7.0.22.51805

Later when the runtime later checks the version the error is raised. As mentioned this will be easy to fix via a NuGet package with our dependencies. That said I still want to look if it is possible to reasonably work around the problem.

[pjanotti]

@soulmate1314 a temporary workaround will be to add the following NuGet packages to your project:

    <PackageReference Include="Microsoft.Extensions.Logging" Version="7.0.0" />
    <PackageReference Include="Microsoft.Extensions.Logging.Configuration" Version="7.0.0" />

This works because the AdditionalDeps are bringing version 7.0.0 of the "Microsoft.Extensions.*" assemblies in order to support Logging. These two packages should be all that you need to cover it.

[pellared]

@soulmate1314 Please close the issue if we helped you and answered all your questions 😉

[soulmate1314]

@pjanotti Thank you for your patient help. I will try it according to your method.