Document diagnostics data collection

[maartenba]

Document diagnostics data collection. Ideally in a separate page (with a v7.3 badge in the sidebar), and link it from deployment/troubleshooting pages.

Please target release/is7.3 in the pull request.

Make sure to cover how to capture this data in various logging frameworks. Check with @bhazen for examples.

Diagnostic data has been added to IdentityServer to help support with troubleshooting. Read more about this new feature in our docs. TODO: link to docs

• Diagnostic Summary Setup by @bhazen in #2021
• Auth Scheme Diagnostic Entry by @bhazen in #2022
• Registered Implementations Diagnostic Entry by @bhazen in #2023
• IdentityServerOptions Diagnostic Entry by @bhazen #2024
• Data Protection Diagnostic Entry by @bhazen in #2025
• Token Issue Count Diagnostic Entry by @bhazen in #2029
• License Usage Summary Diagnostic Entry by @bhazen in #2033
• Basic Server Info Diagnostic Entry by @bhazen in #2034
• Endpoint Usage Diagnostic Entry by @bhazen in #2036
• Chunk diagnostics and update diagnostic options by @josephdecock in #2042
• Client Info Diagnostic Entry by @bhazen in #2043
• Resource Info Diagnostic Entry by @bhazen in #2047
• Registered Implementation Diagnostic Entry Tweaks by @bhazen in #2048
• Assembly Info Diagnostic Entry Tweaks in @bhazen in #2051
• Remove assemblyCount from diagnostics by @bhazen in #2054
• Remove pop from the names of token counts in diagnostics by @josephdecock in #2055
• Diagnostic Summary Log Tweaks by @bhazen in #2057
• Update Diagnostic Summary Log Source Name by @bhazen in #2059
• Log diagnostic summary on shutdown by @bhazen in #2061

[bhazen]

Potential config using Serilog configuration in code:

builder.Host.UseSerilog((context, services, configuration) =>
        {
            configuration
                .ReadFrom.Configuration(context.Configuration)
                .ReadFrom.Services(services)
                .Enrich.FromLogContext()
                .MinimumLevel.Debug()
                .MinimumLevel.Override("Microsoft", LogEventLevel.Warning)
                .MinimumLevel.Override("Microsoft.AspNetCore.Authentication", LogEventLevel.Information)
                .MinimumLevel.Override("Microsoft.Hosting.Lifetime", LogEventLevel.Information)
                .MinimumLevel.Override("System", LogEventLevel.Warning)
                .WriteTo.Logger(fileLogger =>
                {
                    fileLogger
                        .WriteTo.File("./diagnostics/diagnostic.log",
                            rollingInterval: RollingInterval.Day,
                            fileSizeLimitBytes: 1024 * 1024 * 10, // 10 MB
                            rollOnFileSizeLimit: true,
                            outputTemplate:
                            "[{Timestamp:HH:mm:ss} {Level} {EventId}] {SourceContext}{NewLine}{Message:lj}{NewLine}{Exception}{NewLine}")
                        .Filter.ByIncludingOnly(Matching.FromSource("Duende.IdentityServer.Diagnostics.Summary"));
                })
                .WriteTo.Logger(consoleLogger =>
                {
                    consoleLogger
                        .WriteTo.Console(
                            outputTemplate:
                            "[{Timestamp:HH:mm:ss} {Level} {EventId}] {SourceContext}{NewLine}{Message:lj}{NewLine}{Exception}{NewLine}")
                        .Filter.ByExcluding(Matching.FromSource("Duende.IdentityServer.Diagnostics.Summary"));
                });

[bhazen]

Potential config using Serilog with configuration which also requires the Serilog.Expressions package:

{
  "Serilog": {
    "Using": ["Serilog.Sinks.Console", "Serilog.Sinks.File"],
    "Enrich": [ "FromLogContext" ],
    "MinimumLevel": {
      "Default": "Debug",
      "Override": {
        "Microsoft": "Warning",
        "Microsoft.Hosting.Lifetime": "Information",
        "Microsoft.AspNetCore.Authentication": "Debug",
        "System": "Warning"
      }
    },
    "WriteTo": [
      {
        "Name": "Logger",
        "Args": {
          "configureLogger": {
            "WriteTo": [
              {
                "Name": "File",
                "Args": {
                  "path": "diagnostics/identity-server-diagnostics.log",
                  "outputTemplate": "[{Timestamp:HH:mm:ss} {Level} {EventId}] {SourceContext}{NewLine}{Message:lj}{NewLine}{Exception}{NewLine}",
                  "rollingInterval": "Day",
                  "fileSizeLimitBytes": 10000000,
                  "rollOnFileSizeLimit": true
                }
              }
            ],
            "Filter": [
              {
                "Name": "ByIncludingOnly",
                "Args": {
                  "expression": "StartsWith(SourceContext, 'Duende.IdentityServer.Diagnostics.Summary')"
                }
              }
            ]
          }
        }
      },
      {
        "Name": "Logger",
        "Args": {
          "configureLogger": {
            "WriteTo": [
              {
                "Name": "Console",
                "Args": {
                  "outputTemplate": "[{Timestamp:HH:mm:ss} {Level} {EventId}] {SourceContext}{NewLine}{Message:lj}{NewLine}{Exception}{NewLine}"
                }
              }
            ],
            "Filter": [
              {
                "Name": "ByExcluding",
                "Args": {
                  "expression": "SourceContext = 'Duende.IdentityServer.Diagnostics.Summary'"
                }
              }
            ]
          }
        }
      }
    ]
  }
}

[bhazen]

Potential config for log4net:

<?xml version="1.0" encoding="utf-8"?>
<log4net>
  <appender name="ConsoleAppender" type="log4net.Appender.ConsoleAppender">
    <layout type="log4net.Layout.PatternLayout">
      <conversionPattern value="%date [%thread] %-5level %logger - %message%newline" />
    </layout>
    <filter type="log4net.Filter.LoggerMatchFilter">
      <loggerToMatch value="Duende.IdentityServer.Diagnostics.Summary" />
      <acceptOnMatch value="false" />
    </filter>
  </appender>
  <appender name="FileAppender" type="log4net.Appender.FileAppender">
    <file value="./diagnostics/Host.Log4net.log" />
    <appendToFile value="true" />
    <layout type="log4net.Layout.PatternLayout">
      <conversionPattern value="%date [%thread] %-5level %logger - %message%newline" />
    </layout>
    <lockingModel type="log4net.Appender.FileAppender+MinimalLock" />
    <filter type="log4net.Filter.LoggerMatchFilter">
      <loggerToMatch value="Duende.IdentityServer.Diagnostics.Summary" />
    </filter>
    <filter type="log4net.Filter.DenyAllFilter" />
  </appender>
  <root>
    <level value="INFO"/>
    <appender-ref ref="ConsoleAppender" />
  </root>
  <logger name="Duende.IdentityServer.Diagnostics.Summary">
    <level value="INFO" />
    <appender-ref ref="FileAppender" />
  </logger>
</log4net>

[bhazen]

Potential config for NLog:

"NLog": {
    "ThrowConfigExceptions": true,
    "Targets": {
      "file": {
        "type": "File",
        "fileName": "${basedir}/diagnostics/${shortdate}.log",
        "layout": "${longdate} ${level:uppercase=true} ${logger} ${message} ${exception:format=ToString}"
      },
      "console": {
        "type": "ColoredConsole",
        "layout": "${longdate} ${level:uppercase=true} ${logger} ${message} ${exception:format=ToString}"
      }
    },
    "Rules": [
      {
        "logger": "Duende.IdentityServer.Diagnostics.Summary",
        "writeTo": "file",
        "final": true,
        "maxLevel": "Info"
      },
      {
        "logger": "*",
        "minLevel": "Info",
        "writeTo": "console"
      }
    ]
  }