Ascii_only? error when attempting tls connection via Openssl

[Hos73]

I am having an issue when attempting to forward a buffer from a client fluentD to a server via TLS and making use of TLS certificates. The 2 clients have no issues connecting to each other but when attemting to send this buffer message I get a "#0 got unrecoverable error in primary and no secondary error_class=NoMethodError error="undefined method "ascii_only?" for nil:NilClass" error, in particular this comes from /usr/lib/ruby/3.0.0/openssl/ssl.rb:309:in "verify_hostname". For additional background I have successfully had these 2 machines communicating in a similar method without TLS, I am attempting to encrypt this traffic and the introduction of the TLS transport method from the forward documentation is where these issues were introduced. Secondly using the most barebones of that TLS system has worked to communicate with this given certificate. Any help understanding my issue would be greatly appreciated!

Attempted message to send over buffer.
2023-09-05 13:55:19.000000000 -0400 uuidStandin.test.testStandin: {"log_level":"critical","message":"I am logging a critical message"}

Client conf:

<source>
	@type forward
	port 24224
	bind 0.0.0.0
</source>

<match *.*.*>

       @type copy
       <store>
       @type stdout
       </store>

       <store>
       @type forward
       
       transport tls
       heartbeat_type udp
       tls_cert_path /etc/fluent/cert/fluentd.crt

       <server>
       host XX.XX.XX.XXX
       port 24224
       </server>

       </store>
</match>

Server Conf:

<source>
        @type forward
        @id forward_input
        port 24224
        bind 0.0.0.0

        <transport tls>
                   cert_path /etc/fluent/cert/fluentd.crt
                   private_key_path /etc/fluent/cert/fluentd.key
                   private_key_passphrase password
        </transport>
</source>

<match *.*.*>
       @type file
       path /fixture/logging/${tag[0]}/${tag[1]}/${tag[2]}/%Y.%m.%d-%H.%M

       <format>
       utc true
       </format>

       <buffer tag,time>
       timekey_use_utc true
       timekey 1m
       flush_mode immediate
       flush_at_shutdown true
       </buffer>
</match>

Full Error trace:

2023-09-05 14:11:00 -0400 [warn]: #0 got unrecoverable error in primary and no secondary error_class=NoMethodError error="undefined method `ascii_only?' for nil:NilClass"
  2023-09-05 14:11:00 -0400 [warn]: #0 /usr/lib/ruby/3.0.0/openssl/ssl.rb:309:in `verify_hostname'
  2023-09-05 14:11:00 -0400 [warn]: #0 /usr/lib/ruby/3.0.0/openssl/ssl.rb:298:in `block in verify_certificate_identity'
  2023-09-05 14:11:00 -0400 [warn]: #0 /usr/lib/ruby/3.0.0/openssl/ssl.rb:296:in `each'
  2023-09-05 14:11:00 -0400 [warn]: #0 /usr/lib/ruby/3.0.0/openssl/ssl.rb:296:in `verify_certificate_identity'
  2023-09-05 14:11:00 -0400 [warn]: #0 /usr/lib/ruby/3.0.0/openssl/ssl.rb:404:in `post_connection_check'
  2023-09-05 14:11:00 -0400 [warn]: #0 /var/lib/gems/3.0.0/gems/fluentd-1.16.2/lib/fluent/plugin_helper/socket.rb:183:in `socket_create_tls'
  2023-09-05 14:11:00 -0400 [warn]: #0 /var/lib/gems/3.0.0/gems/fluentd-1.16.2/lib/fluent/plugin/out_forward.rb:382:in `create_transfer_socket'
  2023-09-05 14:11:00 -0400 [warn]: #0 /var/lib/gems/3.0.0/gems/fluentd-1.16.2/lib/fluent/plugin/out_forward/connection_manager.rb:46:in `call'
  2023-09-05 14:11:00 -0400 [warn]: #0 /var/lib/gems/3.0.0/gems/fluentd-1.16.2/lib/fluent/plugin/out_forward/connection_manager.rb:46:in `connect'
  2023-09-05 14:11:00 -0400 [warn]: #0 /var/lib/gems/3.0.0/gems/fluentd-1.16.2/lib/fluent/plugin/out_forward.rb:807:in `connect'
  2023-09-05 14:11:00 -0400 [warn]: #0 /var/lib/gems/3.0.0/gems/fluentd-1.16.2/lib/fluent/plugin/out_forward.rb:676:in `send_data'
  2023-09-05 14:11:00 -0400 [warn]: #0 /var/lib/gems/3.0.0/gems/fluentd-1.16.2/lib/fluent/plugin/out_forward.rb:365:in `block in write'
  2023-09-05 14:11:00 -0400 [warn]: #0 /var/lib/gems/3.0.0/gems/fluentd-1.16.2/lib/fluent/plugin/out_forward/load_balancer.rb:46:in `block in select_healthy_node'
  2023-09-05 14:11:00 -0400 [warn]: #0 /var/lib/gems/3.0.0/gems/fluentd-1.16.2/lib/fluent/plugin/out_forward/load_balancer.rb:37:in `times'
  2023-09-05 14:11:00 -0400 [warn]: #0 /var/lib/gems/3.0.0/gems/fluentd-1.16.2/lib/fluent/plugin/out_forward/load_balancer.rb:37:in `select_healthy_node'
  2023-09-05 14:11:00 -0400 [warn]: #0 /var/lib/gems/3.0.0/gems/fluentd-1.16.2/lib/fluent/plugin_helper/service_discovery/manager.rb:108:in `select_service'
  2023-09-05 14:11:00 -0400 [warn]: #0 /var/lib/gems/3.0.0/gems/fluentd-1.16.2/lib/fluent/plugin_helper/service_discovery.rb:82:in `service_discovery_select_service'
  2023-09-05 14:11:00 -0400 [warn]: #0 /var/lib/gems/3.0.0/gems/fluentd-1.16.2/lib/fluent/plugin/out_forward.rb:365:in `write'
  2023-09-05 14:11:00 -0400 [warn]: #0 /var/lib/gems/3.0.0/gems/fluentd-1.16.2/lib/fluent/plugin/output.rb:1225:in `try_flush'
  2023-09-05 14:11:00 -0400 [warn]: #0 /var/lib/gems/3.0.0/gems/fluentd-1.16.2/lib/fluent/plugin/out_forward.rb:353:in `try_flush'
  2023-09-05 14:11:00 -0400 [warn]: #0 /var/lib/gems/3.0.0/gems/fluentd-1.16.2/lib/fluent/plugin/output.rb:1538:in `flush_thread_run'
  2023-09-05 14:11:00 -0400 [warn]: #0 /var/lib/gems/3.0.0/gems/fluentd-1.16.2/lib/fluent/plugin/output.rb:510:in `block (2 levels) in start'
  2023-09-05 14:11:00 -0400 [warn]: #0 /var/lib/gems/3.0.0/gems/fluentd-1.16.2/lib/fluent/plugin_helper/thread.rb:78:in `block in thread_create'
2023-09-05 14:11:00 -0400 [warn]: #0 bad chunk is moved to /tmp/fluent/backup/worker0/object_80c/604a088fe157154062c9743abc88cd2e.log

[kenhys]

How do you generate certificates?

I'm not sure what ruby version is used,
it seems that if CN is not set, it raises above error.

https://github.com/ruby/ruby/blob/master/ext/openssl/lib/openssl/ssl.rb#L298-L304

you can check CN via;

openssl x509 -noout -subject -in /etc/fluent/cert/fluentd.crt

[Hos73]

I generated the certificate from the command that was provided in the tutorial, the CN that I set was the IP I am using for the host forwarding, i.e host XX.XX.XX.XXX.
openssl req -new -x509 -sha256 -days 1095 -newkey rsa:2048 -keyout fluentd.key -out fluentd.crt

Server version:

starting fluentd-1.16.2 pid=2990 ruby="2.7.4"

Client Version:
starting fluentd-1.16.2 pid=10230 ruby="3.0.2"
Are these mixed versions an Issue I had not noticed this previously, client is where the error is occurring

[kenhys]

I've overlooked the actual logic.

It seems that not san.ascii_only? but hostname.ascii_only was called.
In this case hostname is always nil so the solution is something like in client side:

<match *.*.*>

       ...(snip)

       transport tls
       heartbeat_type udp
       tls_cert_path /etc/fluent/cert/fluentd.crt
       tls_verify_hostname false

       <server>
       host XX.XX.XX.XXX
       port 24224
       </server>

       </store>
</match>

tls_verify_hostname false will work.

[Hos73]

Thank you so much! I sincerely appreciate it! This worked perfectly