Add support for trusting dev certs on linux (dotnet#56582)

* Add support for trusting dev certs on linux

There's no consistent way to do this that works for all clients on all Linux distros, but this approach gives us pretty good coverage.  In particular, we aim to support .NET (esp HttpClient), Chromium, and Firefox on Ubuntu- and Fedora-based distros.

Certificate trust is applied per-user, which is simpler and preferable for security reasons, but comes with the notable downside that the process can't be completed within the tool - the user has to update an environment variable, probably in their user profile.  In particular, OpenSSL consumes the `SSL_CERT_DIR` environment variable to determine where it should look for trusted certificates.

We break establishing trust into two categories: OpenSSL, which backs .NET, and NSS databases (henceforth, nssdb), which backs browsers.

To establish trust in OpenSSL, we put the certificate in `~/.dotnet/corefx/cryptography/trusted`, run a simplified version of OpenSSL's `c_rehash` tool on the directory, and ask the user to update `SSL_CERT_DIR`.

To establish trust in nssdb, we search the home directory for Firefox profiles and `~/.pki/nssdb`.  For each one found, we add an entry to the nssdb therein.

Each of these locations (the trusted certificate folder and the list of nssdbs) can be overridden with an environment variable.

This large number of steps introduces a problem that doesn't exist on Windows or macOS - the dev cert can end up trusted by some clients but not by others.  This change introduces a `TrustLevel` concept so that we can produce clearer output when this happens.

The only non-bundled tools required to update certificate trust are `openssl` (the CLI) and `certutil`.  `sudo` is not required, since all changes are within the user's home directory.

* Also trust certificates in the Current User/Root store

A belt-and-suspenders approach for dotnet trust (i.e. in addition to OpenSSL trust) that has the notable advantage of not requiring any environment variables.

* Clarify the mac-specific comments in GetDevelopmentCertificateFromStore

(cherry picked from commit 27ae082)

Author: amcasey

src/Servers/Kestrel/Core/src/Internal/LoggerExtensions.cs

@@ -40,4 +40,7 @@ internal static partial class LoggerExtensions
 
     [LoggerMessage(8, LogLevel.Warning, "The ASP.NET Core developer certificate is not trusted. For information about trusting the ASP.NET Core developer certificate, see https://aka.ms/aspnet/https-trust-dev-cert.", EventName = "DeveloperCertificateNotTrusted")]
     public static partial void DeveloperCertificateNotTrusted(this ILogger<KestrelServer> logger);
+
+    [LoggerMessage(9, LogLevel.Warning, "The ASP.NET Core developer certificate is only trusted by some clients. For information about trusting the ASP.NET Core developer certificate, see https://aka.ms/aspnet/https-trust-dev-cert", EventName = "DeveloperCertificatePartiallyTrusted")]
+    public static partial void DeveloperCertificatePartiallyTrusted(this ILogger<KestrelServer> logger);
 }

src/Servers/Kestrel/Core/src/KestrelServerOptions.cs

@@ -385,23 +385,34 @@ internal void Serialize(Utf8JsonWriter writer)
                 return null;
             }
 
-            var status = CertificateManager.Instance.CheckCertificateState(cert, interactive: false);
+            var status = CertificateManager.Instance.CheckCertificateState(cert);
             if (!status.Success)
             {
-                // Display a warning indicating to the user that a prompt might appear and provide instructions on what to do in that
-                // case. The underlying implementation of this check is specific to Mac OS and is handled within CheckCertificateState.
-                // Kestrel must NEVER cause a UI prompt on a production system. We only attempt this here because Mac OS is not supported
-                // in production.
+                // Failure is only possible on MacOS and indicates that, if there is a dev cert, it must be from
+                // a dotnet version prior to 7.0 - newer versions store it in such a way that this check succeeds.
+                // (Success does not mean that the dev cert has been trusted).
+                // In practice, success.FailureMessage will always be MacOSCertificateManager.InvalidCertificateState.
+                // Basically, we're just going to encourage the user to generate and trust the dev cert.  We support
+                // these older certificates not by accepting them as-is, but by modernizing them when dev-certs is run.
+                // If we detect an issue here, we can avoid a UI prompt below.
                 Debug.Assert(status.FailureMessage != null, "Status with a failure result must have a message.");
                 logger.DeveloperCertificateFirstRun(status.FailureMessage);
 
                 // Prevent binding to HTTPS if the certificate is not valid (avoid the prompt)
                 return null;
             }
 
-            if (!CertificateManager.Instance.IsTrusted(cert))
+            // On MacOS, this may cause a UI prompt, since it requires accessing the keychain.  Kestrel must NEVER
+            // cause a UI prompt on a production system. We only attempt this here because MacOS is not supported
+            // in production.
+            switch (CertificateManager.Instance.GetTrustLevel(cert))
             {
-                logger.DeveloperCertificateNotTrusted();
+                case CertificateManager.TrustLevel.Partial:
+                    logger.DeveloperCertificatePartiallyTrusted();
+                    break;
+                case CertificateManager.TrustLevel.None:
+                    logger.DeveloperCertificateNotTrusted();
+                    break;
             }
 
             return cert;

src/Shared/CertificateGeneration/CertificateManager.cs

@@ -11,6 +11,7 @@ internal enum EnsureCertificateResult
     ErrorSavingTheCertificateIntoTheCurrentUserPersonalStore,
     ErrorExportingTheCertificate,
     FailedToTrustTheCertificate,
+    PartiallyFailedToTrustTheCertificate,
     UserCancelledTrustStep,
     FailedToMakeKeyAccessible,
     ExistingHttpsCertificateTrusted,

src/Shared/CertificateGeneration/EnsureCertificateResult.cs

@@ -69,12 +69,7 @@ internal sealed class MacOSCertificateManager : CertificateManager
         "To fix this issue, run 'dotnet dev-certs https --clean' and 'dotnet dev-certs https' " +
         "to remove all existing ASP.NET Core development certificates " +
         "and create a new untrusted developer certificate. " +
-        "On macOS or Windows, use 'dotnet dev-certs https --trust' to trust the new certificate.";
-
-    public const string KeyNotAccessibleWithoutUserInteraction =
-        "The application is trying to access the ASP.NET Core developer certificate key. " +
-        "A prompt might appear to ask for permission to access the key. " +
-        "When that happens, select 'Always Allow' to grant 'dotnet' access to the certificate key in the future.";
+        "Use 'dotnet dev-certs https --trust' to trust the new certificate.";
 
     public MacOSCertificateManager()
     {
@@ -85,12 +80,14 @@ internal MacOSCertificateManager(string subject, int version)
     {
     }
 
-    protected override void TrustCertificateCore(X509Certificate2 publicCertificate)
+    protected override TrustLevel TrustCertificateCore(X509Certificate2 publicCertificate)
     {
-        if (IsTrusted(publicCertificate))
+        var oldTrustLevel = GetTrustLevel(publicCertificate);
+        if (oldTrustLevel != TrustLevel.None)
         {
+            Debug.Assert(oldTrustLevel == TrustLevel.Full); // Mac trust is all or nothing
             Log.MacOSCertificateAlreadyTrusted();
-            return;
+            return oldTrustLevel;
         }
 
         var tmpFile = Path.GetTempFileName();
@@ -111,6 +108,7 @@ protected override void TrustCertificateCore(X509Certificate2 publicCertificate)
                 }
             }
             Log.MacOSTrustCommandEnd();
+            return TrustLevel.Full;
         }
         finally
         {
@@ -125,7 +123,7 @@ protected override void TrustCertificateCore(X509Certificate2 publicCertificate)
         }
     }
 
-    internal override CheckCertificateStateResult CheckCertificateState(X509Certificate2 candidate, bool interactive)
+    internal override CheckCertificateStateResult CheckCertificateState(X509Certificate2 candidate)
     {
         return File.Exists(GetCertificateFilePath(candidate)) ?
             new CheckCertificateStateResult(true, null) :
@@ -149,7 +147,7 @@ internal override void CorrectCertificateState(X509Certificate2 candidate)
     }
 
     // Use verify-cert to verify the certificate for the SSL and X.509 Basic Policy.
-    public override bool IsTrusted(X509Certificate2 certificate)
+    public override TrustLevel GetTrustLevel(X509Certificate2 certificate)
     {
         var tmpFile = Path.GetTempFileName();
         try
@@ -166,7 +164,7 @@ public override bool IsTrusted(X509Certificate2 certificate)
                 RedirectStandardError = true,
             });
             checkTrustProcess!.WaitForExit();
-            return checkTrustProcess.ExitCode == 0;
+            return checkTrustProcess.ExitCode == 0 ? TrustLevel.Full : TrustLevel.None;
         }
         finally
         {