Vault preferres HCL

[linuxmail]

Hello,

Hashicorp wants to use HCL instead of JSON for configuration. There is also a comment about this problem also here and I've added to a comment here too.

I had to switch from the Debian package to the Archive method, which is not a problem but it took a bit time, to understand, why it wasn't working. After the switch and getting most parts up and running, I tried to add service registration to Consul, but the JSON wasn't parsed correctly.
It seems, it wasn't fixed correctly, as I still get:

2022-11-04T11:35:23.879+0100 [WARN]  unknown or unsupported field consul found in configuration at /etc/vault/config.json
2022-11-04T11:35:23.879+0100 [INFO]  proxy environment: http_proxy="" https_proxy="" no_proxy=""
2022-11-04T11:35:23.887+0100 [INFO]  core: Initializing version history cache for core
2022-11-04T11:35:23.889+0100 [INFO]  core: raft retry join initiated

with:

    extra_config      => {
      cluster_addr => "https://${::fqdn}:8201",
      api_addr     => "https://${::fqdn}:8200",
      service_registration =>  {
         consul            => { 
          address           => "${::fqdn}:8501",
          token             => $consul_agent_token,
          scheme            => https,
          tls_cert_file     => $tls_cert_file,
          tls_min_version   => 'tls12',
          tls_cipher_suites => 'TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384,TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256,TLS_RSA_WITH_AES_256_GCM_SHA384,TLS_RSA_WITH_AES_128_GCM_SHA256',
          service_tags      => "vault",
        }
      }
    },

It works, but the message "unknown or unsupported field consul found" is still there. And also the question is .. how to handle the systemd file, when install_method package / repository is used.

cu denny

[avitacco]

I created my own vault module to manage our vault instances at my job, which outputs HCL. I will get a PR started to add HCL output to this module. Some of my code is pretty opinionated, so hopefully, @bastelfreak doesn't mind too much and we can merge the two modules into one.

Edit: Did some end-of-the-day work and this is my very rough non-working first pass at merging my work into this module. https://github.com/avitacco/puppet-vault

[alaunay]

It's even worse, I just tried to install, spent 2 hours not understanding why my parameters from the module weren't used -- there is a /etc/vault.d/vault.hcl which is used and the config.json is completely ignored.

If you try to delete the vault.hcl file, vault refuses to run:
Jan 8 14:54:58 vault systemd[1]: vault.service - "HashiCorp Vault - A tool for managing secrets" was skipped because of an unmet condition check (ConditionFileNotEmpty=/etc/vault.d/vault.hcl).

It's /lib/systemd/system/vault.service has a condition to forbid it to run if the hcl file is not present...
ConditionFileNotEmpty=/etc/vault.d/vault.hcl

As is, the puppet-vault module is totally unusable.

[bastelfreak]

The problem with HCL is that, as far as I know, there's no native way in Ruby nor Puppet to generate it. For JSON, we can just use a Puppet Hash and throw it into to_json_pretty(). That makes it quite simple to manage it.

As is, the puppet-vault module is totally unusable.

To be fair that's not true. it depends which package you're using. If your source add the condition I suggest to use a systemd::dropin_file that clears ConditionFileNotEmpty or adjusts the file ending.

If someone has an idea for easy HCL generation I'm happy to talk about it.

[alaunay]

Agreed, my comment was a bit harsh, sorry. Probably a result of the lost time.

The source is the debian repo for hashicorp, configured by this module via the hashi_stack module, so nothing fancy here, just the last official source :/

root@vault:~# apt-cache policy vault
vault:
Installed: 1.15.4-1
Candidate: 1.15.4-1
Version table:
*** 1.15.4-1 500
500 https://apt.releases.hashicorp.com bookworm/main amd64 Packages
100 /var/lib/dpkg/status

In order to use the config.json...

root@vault:/etc/vault.d# ls -l
total 8
-rwxr-x--- 1 vault vault 276 Jan 8 13:20 config.json
-rw-r--r-- 1 vault vault 0 Dec 5 02:49 vault.env
-rw-r--r-- 1 vault vault 1053 Jan 8 14:54 vault.hcl.orig

Commenting out the condition gives:

Jan 8 15:39:40 vault vault[12899]: error loading configuration from /etc/vault.d/vault.hcl: stat /etc/vault.d/vault.hcl: no such file or directory

... aaaand I found out why.

I use:
vault::install_method: 'repo'

Which gives in manifests/config.pp:
'repo': { $real_manage_service_file = false }

So...
vault::manage_service_file: true

Jan 8 15:59:58 vault (vault)[13955]: vault.service: Failed to locate executable /usr/local/bin/vault: No such file or directory

So:
vault::bin_dir: '/usr/bin'

Did the final trick.

For the HCL, besides a ERB template (like the ntp module), I have no idea.

[edrude]

Agreed, my comment was a bit harsh, sorry. Probably a result of the lost time.

The source is the debian repo for hashicorp, configured by this module via the hashi_stack module, so nothing fancy here, just the last official source :/

root@vault:~# apt-cache policy vault vault: Installed: 1.15.4-1 Candidate: 1.15.4-1 Version table: *** 1.15.4-1 500 500 https://apt.releases.hashicorp.com bookworm/main amd64 Packages 100 /var/lib/dpkg/status

In order to use the config.json...

root@vault:/etc/vault.d# ls -l total 8 -rwxr-x--- 1 vault vault 276 Jan 8 13:20 config.json -rw-r--r-- 1 vault vault 0 Dec 5 02:49 vault.env -rw-r--r-- 1 vault vault 1053 Jan 8 14:54 vault.hcl.orig

Commenting out the condition gives:

Jan 8 15:39:40 vault vault[12899]: error loading configuration from /etc/vault.d/vault.hcl: stat /etc/vault.d/vault.hcl: no such file or directory

... aaaand I found out why.

I use: vault::install_method: 'repo'

Which gives in manifests/config.pp: 'repo': { $real_manage_service_file = false }

So... vault::manage_service_file: true

Jan 8 15:59:58 vault (vault)[13955]: vault.service: Failed to locate executable /usr/local/bin/vault: No such file or directory

So: vault::bin_dir: '/usr/bin'

Did the final trick.

For the HCL, besides a ERB template (like the ntp module), I have no idea.

This worked for me on RHEL 9