fix(*) multiple cleanup and refactor around dns-01

README.md

@@ -139,14 +139,33 @@ with the fallback certificate.
 
 Note that `domain_whitelist` or `domain_whitelist_callback` must be set to include your domain
 that you wish to server autossl, to prevent potential abuse using fake SNI in SSL handshake.
-`domain_whitelist` defines a table that includes all domains should be included, and
-`domain_whitelist_callback` defines a function that accepts domain as parameter and return
-boolean to indicate if it should be included.
+
+`domain_whitelist` defines a table that includes all domains should be included and the CN to be
+used to create cert for. Only a single `*` is allowed as a wildcard.
+
 ```lua
-domain_whitelist = { "domain1.com", "domain2.com", "domain3.com" },
+domain_whitelist = { "domain1.com", "domain2.com", "domain3.com", "*.domain4.com" },
 ```
 
-To match a pattern in your domain name, for example all subdomains under `example.com`, use:
+## Wildcard certificates
+
+To enable this library to create wildcard certificate, the following requirements must be met:
+
+- The wildcard domain appear exactly as `*.somedomain.com` in `domain_whitelist`.
+- `dns-01` challenge is enabled and a dns provider that has `domains` matching the domain is configured.
+
+Otherwise a non-wildcard certificate will be created as fallback.
+
+By default, the wildcard domain `*.example.com` will appear in Common Name. When `wildcard_domain_in_san` is set to `true` however, a cert with Common Name `example.com` and Subject Alternate Name `*.example.com` will be created. Note both `*.example.com` and `example.com` should appear in `dns_provider_accounts`.
+
+## Advanced Usage
+
+### Use a function to include domains
+
+`domain_whitelist_callback` defines a function that accepts domain as parameter and return
+boolean to indicate if it should be included.
+
+To match a pattern in your domain name, for example **all** subdomains under `example.com`, use:
 
 ```lua
 domain_whitelist_callback = function(domain, is_new_cert_needed)
@@ -177,6 +196,8 @@ end}),
 `domain_whitelist_callback` function is provided with a second argument,
 which indicates whether the certificate is about to be served on incoming HTTP request (false) or new certificate is about to be requested (true). This allows to use cached values on hot path (serving requests) while fetching fresh data from storage for new certificates. One may also implement different logic, e.g. do extra checks before requesting new cert.
 
+### Define failure cooloff period
+
 In case of certificate request failure one may want to prevent ACME client to request another certificate immediatelly. By default, the cooloff period it is set to 300 seconds (5 minutes). It may be customized with `failure_cooloff` or with `failure_cooloff_callback` function, e.g. to implement exponential backoff.
 
 ```lua
@@ -345,8 +366,11 @@ than `shm`. If you must use `shm`, you will need to apply
 
 DNS-01 challenge is supported on lua-resty-acme > 0.13.0. Currently, following DNS providers are supported:
 
-- Cloudflare
-- Dynv6
+- `cloudflare`: Cloudflare
+- `dynv6`: Dynv6
+- `dnspod-intl`: Dnspod International (only Dnspod token is supported and use `id,token` in secret field)
+
+To read to how to extend a new DNS provider to work with `dns-01` challenge, see [DNS provider](#dns-providers).
 
 An example config to use `dns-01` challenge would be:
 
@@ -365,24 +389,27 @@ require("resty.acme.autossl").init({
   account_email = "youemail@youdomain.com",
   domain_whitelist = { "example.com", "subdomain.anotherdomain.com" },
 
-  domain_dns_provider_mapping = {
-    ["example.com"] = "cloudflare_account_1",
-    ["*.anotherdomain.com"] = "dynv6_account_1",
-  },
-
-  dns_provider_keys = {
-    cloudflare_account_1 = {
+  dns_provider_accounts = {
+    {
+      name = "cloudflare_prod",
       provider = "cloudflare",
-      content = "apikey of cloudflare",
+      secret = "apikey of cloudflare",
+      domains = { "example.com" },
     },
-    dynv6_account_1 = {
+    {
+      name = "dynv6_staging",
       provider = "dynv6",
-      content = "apikey of dynv6",
+      secret = "apikey of dynv6",
+      domains = { "*.anotherdomain.com" },
     },
   },
+  -- uncomment following to create anotherdomain.com in CN and *.anotherdomain.com in SAN
+  -- wildcard_domain_in_san = true,
 })
 ```
 
+By default, this library tries up to 5 minutes for DNS propagation. If the default TTL for dns provider is longer than that, user may want to tune up `challenge_start_delay` manually to wait longer.
+
 ## resty.acme.autossl
 
 A config table can be passed to `resty.acme.autossl.init()`, the default values are:
@@ -434,14 +461,16 @@ default_config = {
   blocking = false,
   -- if true, the certificate for domain not in whitelist will be deleted from storage
   enabled_delete_not_whitelisted_domain = false,
-  -- the maps of domain name to the dns_provider name defined in the dns_provider_keys table; domain name supports wildcard
-  domain_dns_provider_mapping = {},
   -- the dict of dns providers, each provider should have following struct:
   -- {
+  --   name = "prod_account",
   --   provider = "provider_name", -- "cloudflare" or "dynv6"
-  --   content = "the api key or token",
+  --   secret  = "the api key or token",
+  --   domains = { "example.com", "*.example.com" }, -- the list of domains that can be used with this provider
   -- }
-  dns_provider_keys = {},
+  dns_provider_accounts = {},
+  -- if enabled, wildcard domains like *.example.com will be created as SAN and CN will be example.com
+  wildcard_domain_in_san = false,
 }
 ```
 
@@ -523,14 +552,8 @@ default_config = {
   preferred_chain = nil,
   -- callback function that allows to wait before signaling ACME server to validate
   challenge_start_callback = nil,
-  -- the maps of domain name to the dns_provider name defined in the dns_provider_keys table
-  domain_dns_provider_mapping = {},
   -- the dict of dns providers, each provider should have following struct:
-  -- {
-  --   provider = "provider_name", -- "cloudflare" or "dynv6"
-  --   content = "the api key or token",
-  -- }
-  dns_provider_keys = {},
+  dns_provider_accounts = {},
 }
 ```
 
@@ -763,6 +786,31 @@ Etcd storage requires [lua-resty-etcd](https://github.com/api7/lua-resty-etcd) l
 It can be manually installed with `opm install api7/lua-resty-etcd` or `luarocks install lua-resty-etcd`.
 
 
+## DNS providers
+
+TO create a custom DNS provider, follow these steps:
+
+- Create a file like `route53.lua` under `lib/resty/acme/dns_provider`
+- Implement following function signature
+
+```lua
+function _M.new(token)
+  -- ... 
+  return self
+end
+
+function _M:post_txt_record(fqdn, content)
+  return ok, err
+end
+
+function _M:delete_txt_record(fqdn)
+  return ok, err
+end
+```
+
+Where `token` is the apikey, `fqdn` is the DNS record name to set record, and `content` is the value of the record.
+
+
 TODO
 ====
 - autossl: ocsp staping

lib/resty/acme/autossl.lua

@@ -60,20 +60,22 @@ local default_config = {
   blocking = false,
   -- if true, the certificate for domain not in whitelist will be deleted from storage
   enabled_delete_not_whitelisted_domain = false,
-  -- the maps of domain name to the dns_provider name defined in the dns_provider_keys table; domain name supports wildcard
-  domain_dns_provider_mapping = {},
   -- the dict of dns providers, each provider should have following struct:
   -- {
+  --   name = "prod_account",
   --   provider = "provider_name", -- "cloudflare" or "dynv6"
-  --   content = "the api key or token",
+  --   secret  = "the api key or token",
+  --   domains = { "example.com", "*.example.com" }, -- the list of domains that can be used with this provider
   -- }
-  dns_provider_keys = {},
+  dns_provider_accounts = {},
+  -- if enabled, wildcard domains like *.example.com will be created as SAN and CN will be example.com
+  wildcard_domain_in_san = false,
 }
 
 local domain_pkeys = {}
 
 local domain_key_types, domain_key_types_count
-local domain_whitelist, domain_whitelist_callback, wildcard_domain_matcher
+local domain_whitelist, domain_whitelist_callback, domain_wildcard_matcher
 local failure_cooloff_callback
 
 --[[
@@ -212,7 +214,16 @@ local function update_cert_handler(data)
     ngx.update_time()
     log(ngx_INFO, ngx.now() - t,  "s spent in creating new ", typ, " private key")
   end
-  local cert, err = AUTOSSL.client:order_certificate(pkey, domain)
+
+  -- create example.com automatically if we are creating *.example.com
+  local alt_domain
+  if domain:sub(1, 1) == "*" and AUTOSSL.config.wildcard_domain_in_san then
+    alt_domain = domain
+    domain = domain:sub(3)
+    log(ngx_INFO, "creating wildcard domain certificate using SAN")
+  end
+
+  local cert, err = AUTOSSL.client:order_certificate(pkey, domain, alt_domain)
   if err then
     log(ngx_ERR, "error updating cert for ", domain, " err: ", err)
     return err
@@ -378,7 +389,7 @@ function AUTOSSL.check_renew(premature)
   end
 end
 
-local function build_wildcard_domain_matcher(domains)
+local function build_domain_wildcard_matcher(domains)
   local domains_wildcard = {}
   local domains_wildcard_count = 0
 
@@ -452,11 +463,19 @@ function AUTOSSL.init(autossl_config, acme_config)
     acme_config.api_uri = "https://acme-staging-v02.api.letsencrypt.org/directory"
   end
   acme_config.account_email = autossl_config.account_email
-  acme_config.enabled_challenge_handlers = autossl_config.enabled_challenge_handlers
-  acme_config.domain_dns_provider_mapping = autossl_config.domain_dns_provider_mapping
-  acme_config.dns_provider_keys = autossl_config.dns_provider_keys
+  acme_config.dns_provider_accounts = autossl_config.dns_provider_accounts
+
+  local ech = autossl_config.enabled_challenge_handlers
+  for _, ch in ipairs(ech) do
+    if ch == "dns-01" then
+      AUTOSSL.dns_01_enabled = true
+      break
+    end
+  end
+  acme_config.enabled_challenge_handlers = ech
 
   acme_config.challenge_start_callback = function()
+    ngx.log(ngx.INFO, "wait for ", autossl_config.challenge_start_delay, " seconds to continue")
     ngx.sleep(autossl_config.challenge_start_delay)
     return true
   end
@@ -471,7 +490,7 @@ function AUTOSSL.init(autossl_config, acme_config)
       domain_whitelist[w] = true
     end
     -- build regex for wildcard domain
-    wildcard_domain_matcher = build_wildcard_domain_matcher(domain_whitelist)
+    domain_wildcard_matcher = build_domain_wildcard_matcher(domain_whitelist)
   end
   domain_whitelist_callback = autossl_config.domain_whitelist_callback
   if domain_whitelist_callback and type(domain_whitelist_callback) ~= "function" then
@@ -557,14 +576,14 @@ end
 function AUTOSSL.is_domain_whitelisted(domain, is_new_cert_needed)
   if domain_whitelist_callback then
     -- domain_whitelist_callback always first
-    return domain_whitelist_callback(domain, is_new_cert_needed)
+    return domain_whitelist_callback(domain, is_new_cert_needed) and domain or false
   end
-  if domain_whitelist[domain] then
+  if domain_whitelist and domain_whitelist[domain] then
     -- exact match
     return domain
-  else
+  elseif domain_wildcard_matcher then
     -- wildcard match
-    local result = wildcard_domain_matcher[domain]
+    local result = domain_wildcard_matcher[domain]
     if result then
       return "*" .. result[1]
     end
@@ -580,24 +599,25 @@ function AUTOSSL.ssl_certificate()
     return
   end
 
-  domain = string.lower(domain)
+  local orig_domain = string.lower(domain)
+  domain = AUTOSSL.is_domain_whitelisted(orig_domain, false)
 
-  local result = AUTOSSL.is_domain_whitelisted(domain, false)
-  if type(result) == "string" and result == domain then
-    -- exact match
-    log(ngx_DEBUG, "exact match in domain_whitelist: ", result)
-  elseif type(result) == "string" and string.sub(result, 1, 1) == "*" then
-    -- wildcard match
-    log(ngx_DEBUG, "wildcard match in domain_whitelist: ", result)
-    domain = result
-  elseif result then
-    -- domain_whitelist_callback
-    log(ngx_DEBUG, "match by domain_whitelist_callback: ", domain)
-  else
-    log(ngx_INFO, "domain ", domain, " not in whitelist, skipping")
+  if not domain then
+    log(ngx_INFO, "domain ", orig_domain, " not in whitelist, skipping")
     return
   end
 
+  if domain ~= orig_domain then
+    if AUTOSSL.dns_01_enabled then
+      log(ngx_INFO, "domain ", orig_domain, " is matched to ", domain, " in certificate")
+    else
+      log(ngx_INFO, orig_domain, " matched to wildcard ", domain, " but dns-01 is not enabled, ",
+                    "non-wildcard cert will be created")
+      domain = orig_domain
+    end
+
+  end
+
   local chains_set_count = 0
   local chains_set = {}