Permission denied even though all permissions are set properly

[nikolachakalov]

Hello! I am at a loss at how to troubleshoot or fix this issue.

I am getting a permission denied problem in the systemd logs and have done everything according to the documentation. At least this is what I believe.

It was working just fine and somewhere in these last days it broke. No idea where to look for traces anymore.

Here is my setup:

OS: Ubuntu 24.04.2 LTS x86_64 
Host: NC40U 1.0 
Kernel: 6.8.0-60-generic 

.rwxrwxrwx 1.3M vector 2 Jun 10:25  vector_ufw.log

[Unit]
Description=Vector
Documentation=https://vector.dev
After=network-online.target
Requires=network-online.target

[Service]
User=vector
Group=vector
ExecStartPre=/usr/bin/vector validate
ExecStart=/usr/bin/vector
ExecReload=/usr/bin/vector validate --no-environment
ExecReload=/bin/kill -HUP $MAINPID
Restart=always
AmbientCapabilities=CAP_NET_BIND_SERVICE
EnvironmentFile=-/etc/default/vector
# Since systemd 229, should be in [Unit] but in order to support systemd <229,
# it is also supported to have it here.
StartLimitInterval=10
StartLimitBurst=5
[Install]
WantedBy=multi-user.target

vector.service - Vector
     Loaded: loaded (/usr/lib/systemd/system/vector.service; enabled; preset: enabled)
    Drop-In: /etc/systemd/system/vector.service.d
             └─override.conf
     Active: active (running) since Fri 2025-06-13 12:15:52 CEST; 8min ago
       Docs: https://vector.dev
    Process: 12583 ExecStartPre=/usr/bin/vector validate (code=exited, status=0/SUCCESS)
   Main PID: 12597 (vector)
      Tasks: 15 (limit: 38188)
     Memory: 12.1M (peak: 12.6M)
        CPU: 398ms
     CGroup: /system.slice/vector.service
             └─12597 /usr/bin/vector

Jun 13 12:15:52 ubuntuserver vector[12597]: 2025-06-13T10:15:52.652108Z ERROR sink{component_kind="sink" component_id=ufw_sink component_type=file}: vector::internal_events::file: Unable to open the file. path=b"/home/nikola/multimedia/cfg/fail2ban/test/vector_ufw.log" error=Permission denied (os error 13) error_code=failed_opening_file error_type="io_failed" stage="sending" internal_log_rate_limit=true
Jun 13 12:15:52 ubuntuserver vector[12597]: 2025-06-13T10:15:52.652140Z ERROR sink{component_kind="sink" component_id=ufw_sink component_type=file}: vector_common::internal_event::component_events_dropped: Events dropped intentional=false count=1 reason="Unable to open the file." internal_log_rate_limit=true
Jun 13 12:15:52 ubuntuserver vector[12597]: 2025-06-13T10:15:52.652197Z ERROR sink{component_kind="sink" component_id=ufw_sink component_type=file}: vector::internal_events::file: Internal log [Unable to open the file.] is being suppressed to avoid flooding.
Jun 13 12:15:52 ubuntuserver vector[12597]: 2025-06-13T10:15:52.652210Z ERROR sink{component_kind="sink" component_id=ufw_sink component_type=file}: vector_common::internal_event::component_events_dropped: Internal log [Events dropped] is being suppressed to avoid flooding.
Jun 13 12:17:02 ubuntuserver vector[12597]: 2025-06-13T10:17:02.351417Z ERROR sink{component_kind="sink" component_id=ufw_sink component_type=file}: vector::internal_events::file: Internal log [Unable to open the file.] has been suppressed 1 times.
Jun 13 12:17:02 ubuntuserver vector[12597]: 2025-06-13T10:17:02.351465Z ERROR sink{component_kind="sink" component_id=ufw_sink component_type=file}: vector::internal_events::file: Unable to open the file. path=b"/home/nikola/multimedia/cfg/fail2ban/test/vector_ufw.log" error=Permission denied (os error 13) error_code=failed_opening_file error_type="io_failed" stage="sending" internal_log_rate_limit=true
Jun 13 12:17:02 ubuntuserver vector[12597]: 2025-06-13T10:17:02.351514Z ERROR sink{component_kind="sink" component_id=ufw_sink component_type=file}: vector_common::internal_event::component_events_dropped: Internal log [Events dropped] has been suppressed 1 times.
Jun 13 12:17:02 ubuntuserver vector[12597]: 2025-06-13T10:17:02.351523Z ERROR sink{component_kind="sink" component_id=ufw_sink component_type=file}: vector_common::internal_event::component_events_dropped: Events dropped intentional=false count=1 reason="Unable to open the file." internal_log_rate_limit=true
Jun 13 12:21:57 ubuntuserver vector[12597]: 2025-06-13T10:21:57.507639Z ERROR sink{component_kind="sink" component_id=ufw_sink component_type=file}: vector::internal_events::file: Unable to open the file. path=b"/home/nikola/multimedia/cfg/fail2ban/test/vector_ufw.log" error=Permission denied (os error 13) error_code=failed_opening_file error_type="io_failed" stage="sending" internal_log_rate_limit=true
Jun 13 12:21:57 ubuntuserver vector[12597]: 2025-06-13T10:21:57.507758Z ERROR sink{component_kind="sink" component_id=ufw_sink component_type=file}: vector_common::internal_event::component_events_dropped: Events dropped intentional=false count=1 reason="Unable to open the file." internal_log_rate_limit=true

$ groups vector
vector : vector adm systemd-journal nikola

[nikolachakalov]

I figured it out. The file was nested deep into a folder structure on my server.

I had to make sure that each folder along the path had read/permissions recursively. This is probably something new which I missed in one of the last updates of Vector? No idea ... everything was working fine with the previous permissions settings until the 5th of June.