|
| 1 | +# Troubleshooting Azure Key Vault SDK Issues |
| 2 | +The Azure Key Vault SDKs for Java use a common HTTP pipeline and authentication to create, update, and delete secrets, keys, and certificates in Key Vault and Managed HSM. This troubleshooting guide contains steps for diagnosing issues common to these SDKs. |
| 3 | + |
| 4 | +For package-specific troubleshooting guides, see any of the following: |
| 5 | + |
| 6 | +* [Troubleshooting guide for the Azure Key Vault Administration SDK][kv_admin_troubleshooting] |
| 7 | +* [Troubleshooting guide for the Azure Key Vault Certificates SDK][kv_certs_troubleshooting] |
| 8 | +* [Troubleshooting guide for the Azure Key Vault Keys SDK][kv_keys_troubleshooting] |
| 9 | +* [Troubleshooting guide for the Azure Key Vault Secrets SDK][kv_secrets_troubleshooting] |
| 10 | + |
| 11 | +## Table of Contents |
| 12 | +* [Troubleshooting Authentication Issues](#troubleshooting-authentication-issues) |
| 13 | + * [HTTP 401 Errors](#http-401-errors) |
| 14 | + * [Frequent HTTP 401 Errors in Logs](#frequent-http-401-errors-in-logs) |
| 15 | + * [AKV10032: Invalid issuer](#akv10032--invalid-issuer) |
| 16 | + * [HTTP 403 Errors](#http-403-errors) |
| 17 | + * [Operation Not Permitted](#operation-not-permitted) |
| 18 | + * [Access Denied to First Party Service](#access-denied-to-first-party-service) |
| 19 | + * [Other authentication issues](#other-authentication-issues) |
| 20 | + * [Tenant authentication issues](#tenant-authentication-issues) |
| 21 | + * [Incorrect challenge resource](#incorrect-challenge-resource) |
| 22 | +* [Other Service Errors](#other-service-errors) |
| 23 | + * [HTTP 429: Too Many Requests](#http-429--too-many-requests) |
| 24 | +* [Support](#support) |
| 25 | + |
| 26 | +## Troubleshooting Authentication Issues |
| 27 | +### HTTP 401 Errors |
| 28 | +HTTP 401 errors may indicate problems authenticating, but silent 401 errors are also an expected part of the Azure Key Vault authentication flow. |
| 29 | + |
| 30 | +#### Frequent HTTP 401 Errors in Logs |
| 31 | +Most often, this is expected. Azure Key Vault issues a challenge for initial requests that force authentication. You may see these errors most often during application startup, but may also see these periodically during the application's lifetime when authentication tokens are near expiration. |
| 32 | + |
| 33 | +If you are not seeing subsequent exceptions from the Key Vault SDKs, authentication challenges are likely the cause. If you continuously see 401 errors without successful operations, there may be an issue with the authentication library that's being used. We recommend using the Azure SDK's [azure-identity] library for authentication. |
| 34 | + |
| 35 | +#### AKV10032: Invalid issuer |
| 36 | +You may see an error similar to: |
| 37 | + |
| 38 | +```text |
| 39 | +com.azure.v2.core.exception.HttpResponseException: Status code 401, "{"error":{"code":"Unauthorized","message":"AKV10032: Invalid issuer. Expected one of https://sts.windows.net/{tenant 1}/, found https://sts.windows.net/{tenant 2}/."}}" |
| 40 | +``` |
| 41 | + |
| 42 | +This is most often caused by being logged into a different tenant than the Key Vault authenticates. See our [DefaultAzureCredential] documentation to see the order credentials are read. You may be logged into a different tenant for one credential that gets read before another credential. For example, you might be logged into Visual Studio under the wrong tenant even though you're logged into the Azure CLI under the correct tenant. |
| 43 | + |
| 44 | +Automatic tenant discovery support has been added when referencing package `azure-identity` version 1.4.0 or newer, and any of the following Key Vault SDK package versions or newer: |
| 45 | + |
| 46 | +| Package | Minimum Version | |
| 47 | +|------------------------------------------|-----------------| |
| 48 | +| `azure-security-keyvault-administration` | 5.0.0-beta.1 | |
| 49 | +| `azure-security-keyvault-certificates` | 5.0.0-beta.1 | |
| 50 | +| `azure-security-keyvault-keys` | 5.0.0-beta.1 | |
| 51 | +| `azure-security-keyvault-secrets` | 5.0.0-beta.1 | |
| 52 | + |
| 53 | +Upgrading to the package versions should resolve any "Invalid Issuer" errors as long as the application or user is a member of the resource's tenant. |
| 54 | + |
| 55 | +### HTTP 403 Errors |
| 56 | +HTTP 403 errors indicate the user is not authorized to perform a specific operation in Key Vault or Managed HSM. |
| 57 | + |
| 58 | +#### Operation Not Permitted |
| 59 | +You may see an error similar to: |
| 60 | + |
| 61 | +```text |
| 62 | +com.azure.v2.core.exception.HttpResponseException: Status code 403, {"error":{"code":"Forbidden","message":"Operation decrypt is not permitted on this key.","innererror":{"code":"KeyOperationForbidden"}}} |
| 63 | +``` |
| 64 | + |
| 65 | +The operation and inner `code` may vary, but the rest of the text will indicate which operation is not permitted. This error indicates that the authenticated application or user does not have permissions to perform that operation, though the cause may vary. |
| 66 | + |
| 67 | +1. Check that the application or user has the appropriate permissions: |
| 68 | + * [Access policies][access_policies] (Key Vault) |
| 69 | + * [Role-Based Access Control (RBAC)][rbac] (Key Vault and Managed HSM) |
| 70 | +2. If the appropriate permissions are assigned to your application or user, make sure you are authenticating as that user. |
| 71 | + If using the [DefaultAzureCredential], a different credential might've been used than one you expected. |
| 72 | + [Enable logging][identity_logging] and you will see which credential the [DefaultAzureCredential] used as shown below, and why previously-attempted credentials were rejected. |
| 73 | + |
| 74 | + ```text |
| 75 | + [ERROR] c.azure.identity.EnvironmentCredential : Azure Identity => ERROR in EnvironmentCredential: Missing required environment variable AZURE_CLIENT_ID |
| 76 | + [ERROR] c.azure.identity.EnvironmentCredential : EnvironmentCredential authentication unavailable. Environment variables are not fully configured. |
| 77 | + [INFO] c.azure.identity.DefaultAzureCredential : Azure Identity => Attempted credential EnvironmentCredential is unavailable. |
| 78 | + [ERROR] c.a.i.implementation.IdentityClient : ManagedIdentityCredential authentication unavailable. Connection to IMDS endpoint cannot be established, connect timed out. |
| 79 | + [ERROR] c.a.identity.ManagedIdentityCredential : Azure Identity => ERROR in getToken() call for scopes [https://management.core.windows.net//.default]: ManagedIdentityCredential authentication unavailable. Connection to IMDS endpoint cannot be established, connect timed out. |
| 80 | + [INFO] c.azure.identity.DefaultAzureCredential : Azure Identity => Attempted credential ManagedIdentityCredential is unavailable. |
| 81 | + [ERROR] c.a.identity.SharedTokenCacheCredential : Azure Identity => ERROR in getToken() call for scopes [https://management.core.windows.net//.default]: SharedTokenCacheCredential authentication unavailable. No accounts were found in the cache. |
| 82 | + [INFO] c.azure.identity.DefaultAzureCredential : Azure Identity => Attempted credential SharedTokenCacheCredential is unavailable. |
| 83 | + [ERROR] com.azure.v2.identity.IntelliJCredential : Azure Identity => ERROR in getToken() call for scopes [https://management.core.windows.net//.default]: Unrecognized field "tenantId" (class com.azure.v2.identity.implementation.IntelliJAuthMethodDetails), not marked as ignorable (4 known properties: "authMethod", "azureEnv", "accountEmail", "credFilePath"]) |
| 84 | + ``` |
| 85 | + |
| 86 | +#### Access Denied to First Party Service |
| 87 | +You may see an error similar to: |
| 88 | + |
| 89 | +```text |
| 90 | +com.azure.v2.core.exception.HttpResponseException: Status code 403, {"error":{"code":"Forbidden","message":"Access denied to first party service. ...","innererror":{"code":"AccessDenied"}}} |
| 91 | +``` |
| 92 | + |
| 93 | +The error `message` may also contain the tenant ID (`tid`) and application ID (`appid`). This error may occur because: |
| 94 | + |
| 95 | +1. You have the **Allow trust services** option enabled and are trying to access the Key Vault from a service not on |
| 96 | + [this list](https://docs.microsoft.com/azure/key-vault/general/overview-vnet-service-endpoints#trusted-services) of |
| 97 | + trusted services. |
| 98 | +2. You are authenticated against a Microsoft Account (MSA) in Visual Studio or another credential provider. See |
| 99 | + [above](#operation-not-permitted) for troubleshooting steps. |
| 100 | + |
| 101 | +### Other authentication issues |
| 102 | + |
| 103 | +See our Azure Identity [troubleshooting guide][identity_troubleshooting] for general guidance on authentication errors. |
| 104 | + |
| 105 | +#### Multi-tenant authentication issues |
| 106 | + |
| 107 | +If a `ClientAuthenticationException` is thrown with a message similar to: |
| 108 | + |
| 109 | +> The current credential is not configured to acquire tokens for tenant |
| 110 | +
|
| 111 | +See our [troubleshooting guide for multi-tenant authentication issues][identity_multitenant]. Read our [release notes](https://aka.ms/azsdk/blog/multi-tenant-guidance) for more information about this change. |
| 112 | + |
| 113 | +#### Incorrect challenge resource |
| 114 | + |
| 115 | +If an exception is thrown with a message similar to: |
| 116 | + |
| 117 | +> The challenge resource '<my-key-vault>.vault.azure.net' does not match the requested domain. If you wish to disable this check for your client, pass 'true' to the SecretClientBuilder.disableChallengeResourceVerification() method when building it. See https://aka.ms/azsdk/blog/vault-uri for more information. |
| 118 | +
|
| 119 | +Check that the resource is expected - that you're not receiving an authentication challenge from an unknown host which may indicate an incorrect request URI. If the resource is correct but you're using a mock service or non-transparent proxy, use the `disableChallengeResourceVerification()` when creating your clients with a client builder, for example: |
| 120 | + |
| 121 | +```java |
| 122 | +import com.azure.v2.core.TokenCredential; |
| 123 | +import com.azure.v2.identity.DefaultAzureCredentialBuilder; |
| 124 | +import com.azure.v2.security.keyvault.secrets.SecretClient; |
| 125 | +import com.azure.v2.security.keyvault.secrets.SecretClientBuilder; |
| 126 | + |
| 127 | +public class MyClass { |
| 128 | + public static void main(String[] args) { |
| 129 | + TokenCredential credential = new DefaultAzureCredentialBuilder().build(); |
| 130 | + SecretClient client = new SecretClientBuilder() |
| 131 | + .vaultUrl("https://my-key-vault.vault.azure.net/") |
| 132 | + .credential(credential) |
| 133 | + .disableChallengeResourceVerification(); |
| 134 | + } |
| 135 | +} |
| 136 | +``` |
| 137 | + |
| 138 | +Read our [release notes][release_notes_resource] for more information about this change. |
| 139 | + |
| 140 | +## Other Service Errors |
| 141 | +To troubleshoot additional HTTP service errors not described below, see [Azure Key Vault REST API Error Codes][kv_error_codes]. |
| 142 | + |
| 143 | +### HTTP 429: Too Many Requests |
| 144 | +If you get an exception or see logs that describe HTTP 429, you may be making too many requests to Key Vault too quickly. |
| 145 | + |
| 146 | +Possible solutions include: |
| 147 | + |
| 148 | +1. Use a singleton for any `CertificateClient`, `KeyClient`, `CryptographyClient`, or `SecretClient` and their async counterparts in your application for a single Key Vault. |
| 149 | +2. Use a single instance of [DefaultAzureCredential] or other credential you use to authenticate your clients for each Key Vault or Managed HSM endpoint you need to access. |
| 150 | +3. You could cache a certificate, key, or secret in memory for a time to reduce calls to retrieve them. |
| 151 | +4. Use [Azure App Configuration][azure_appconfiguration] for storing non-secrets and references to Key Vault secrets. Storing all app configuration in Key Vault will increase the likelihood of requests being throttled as more application instances are started. |
| 152 | +5. If you are performing encryption or decryption operations, consider using wrap and unwrap operations for a symmetric key which this may also improve application throughput. |
| 153 | + |
| 154 | +See our [Azure Key Vault throttling guide][throttling_guide] for more information. |
| 155 | + |
| 156 | +## Support |
| 157 | + |
| 158 | +For additional support, please search our [existing issues](https://github.com/Azure/azure-sdk-for-java/issues) or [open a new issue](https://github.com/Azure/azure-sdk-for-java/issues/new/choose). You may also find existing answers on community sites like [Stack Overflow](https://stackoverflow.com/questions/tagged/azure-keyvault+java). |
| 159 | + |
| 160 | +[access_policies]: https://docs.microsoft.com/azure/key-vault/general/assign-access-policy |
| 161 | +[azure_appconfiguration]: https://github.com/Azure/azure-sdk-for-java/blob/main/sdk/appconfiguration/azure-data-appconfiguration/README.md |
| 162 | +[azure-identity]: https://github.com/Azure/azure-sdk-for-java/blob/main/sdk/identity/azure-identity/README.md |
| 163 | +[DefaultAzureCredential]: https://github.com/Azure/azure-sdk-for-java/blob/main/sdk/identity/azure-identity/README.md#defaultazurecredential |
| 164 | +[kv_admin_troubleshooting]: https://github.com/Azure/azure-sdk-for-java/tree/main/sdk/keyvault-v2/azure-security-keyvault-administration/TROUBLESHOOTING.md |
| 165 | +[kv_certs_troubleshooting]: https://github.com/Azure/azure-sdk-for-java/tree/main/sdk/keyvault-v2/azure-security-keyvault-certificates/TROUBLESHOOTING.md |
| 166 | +[kv_error_codes]: https://docs.microsoft.com/azure/key-vault/general/rest-error-codes |
| 167 | +[kv_keys_troubleshooting]: https://github.com/Azure/azure-sdk-for-java/tree/main/sdk/keyvault-v2/azure-security-keyvault-keys/TROUBLESHOOTING.md |
| 168 | +[kv_secrets_troubleshooting]: https://github.com/Azure/azure-sdk-for-java/tree/main/sdk/keyvault-v2/azure-security-keyvault-secrets/TROUBLESHOOTING.md |
| 169 | +[identity_logging]: https://docs.microsoft.com/azure/developer/java/sdk/logging-overview |
| 170 | +[identity_troubleshooting]: https://github.com/Azure/azure-sdk-for-python/blob/main/sdk/identity/azure-identity/TROUBLESHOOTING.md |
| 171 | +[identity_multitenant]: https://github.com/Azure/azure-sdk-for-java/blob/main/sdk/identity/azure-identity/TROUBLESHOOTING.md#troubleshoot-multi-tenant-authentication-issues |
| 172 | +[rbac]: https://docs.microsoft.com/azure/key-vault/general/rbac-guide |
| 173 | +[release_notes_resource]: https://aka.ms/azsdk/blog/vault-uri |
| 174 | +[release_notes_tenant]: https://aka.ms/azsdk/blog/multi-tenant-guidance |
| 175 | +[throttling_guide]: https://docs.microsoft.com/azure/key-vault/general/overview-throttling |
0 commit comments