Skip to content

Commit 9a41597

Browse files
committed
Added a kqueue baswd directory monitoring
Documentation of syslog-ng/syslog-ng#5022 Signed-off-by: Hofi [hofione@gmail.com](mailto:hofione@gmail.com) Signed-off-by: Hofi <hofione@gmail.com>
1 parent 69ffce4 commit 9a41597

File tree

2 files changed

+15
-12
lines changed

2 files changed

+15
-12
lines changed

doc/_admin-guide/060_Sources/020_File/001_File_following.md

Lines changed: 9 additions & 6 deletions
Original file line numberDiff line numberDiff line change
@@ -6,19 +6,22 @@ description: >-
66
The {{ site.product.short_name }} application can use various methods to detect changes
77
in the followed [[file()|adm-src-file]] and wildcard-file() sources. Under the hood, two different change
88
watchers operate: one for detecting changes in the followed directories and one for
9-
monitoring changes in the watched files.
9+
monitoring changes in the followed files.
1010
---
1111

12-
The detection method for directory changes can be controlled via monitor-method(). If the method is `poll`, {{ site.product.short_name }} will set up an (ivykis) timer that periodically checks and compares the content of the given directory at the frequency specified by monitor-freq(). This periodic polling can be resource-intensive (mainly CPU), so selecting the proper value for monitor-freq() is important. This is not an issue on Linux, where you can use `inotify` for monitor-method(), which automatically notifies {{ site.product.short_name }} about changes with no significant resource usage impact.
12+
## Directory monitoring
1313

14-
**NOTE:** For macOS and FreeBSD, a `kqueue` based implementation will be available in the near future, offering similar performance to `inotify` on Linux. Stay tuned!
15-
{: .notice--info}
14+
The detection method for directory changes can be controlled via monitor-method(). If the method is `poll`, {{ site.product.short_name }} will set up an (ivykis) timer that periodically checks and compares the content of the given directory at the frequency specified by monitor-freq(). This periodic polling can be resource-intensive (mainly CPU), so selecting the proper value for monitor-freq() is important!\
15+
This is not an issue on Linux, where you can use `inotify` for monitor-method(), or on BSD-based systems, where `kqueue` is available. These methods can automatically notify {{ site.product.short_name }} about changes with no significant resource usage impact.\
16+
Setting the `auto` method will select the best available method on the given OS.
17+
18+
## Following file changes
1619

1720
Detecting file content changes involves more factors that can affect resource usage and overall performance. {{ site.product.short_name }} uses two main methods to monitor file content changes.
1821

1922
The first method is automatically selected if the follow-freq() option has a value greater than 0. It works like the directory monitoring `poll` monitor-method() and uses an (ivykis) timer with the frequency of follow-freq(). It tries to detect changes in the file content (as well as state, file deletion, and moves) each time the timer fires. Similar to directory change monitoring, this process can be resource-intensive, so you should find the proper setting to balance performance, resource usage, and fault tolerance (such as avoiding log message loss).
2023

21-
The second method is activated if the follow-freq() option is set to 0. It uses ivykis poll methods, sometimes resembling the poll method for directory change watching described above (with its performance penalties), but often working similarly to the `inotify` version (with seamless performance).
24+
The second method is activated if the follow-freq() option is set to 0. It uses ivykis poll methods, sometimes resembling the poll method for directory change watching described above (with its performance penalties), but often working similarly to the `inotify` or `kqueue` version (with seamless performance).
2225

2326
The following table shows which method is selected in different cases.
2427

@@ -154,7 +157,7 @@ A bit more detail about the notation in the platform columns and what they reall
154157
`n.a.` - Means that the feature is not supported on the given platform by default, which has a significant impact on how the final ivykis poll method is selected. Ivykis tries to set up (at initialization time) the method to be used in the order enumerated in the table above. If an option is `n.a.` (determined at build time), then the next option will be used automatically. The first available option will be used, and if it does not work on the given platform (see `does not work` below), then {{ site.product.short_name }} will stop that file source with an error. This can be controlled using the `IV_EXCLUDE_POLL_METHOD` environment variable. Methods enumerated in it will be excluded from the ivykis initialization flow, and the next available (and not excluded) one will be used. The strings that can be used in `IV_EXCLUDE_POLL_METHOD` are `port-timer port dev_poll epoll-timerfd epoll kqueue ppoll poll` in the same order as in the table.\
155158
e.g., on Linux you should use `IV_EXCLUDE_POLL_METHOD="epoll-timerfd epoll"` to force the usage of the `ppoll` method, as `port-timer port dev_poll` are not available, and `epoll-timerfd epoll` are not working currently. However, note that currently `ppoll` and `poll` are the only working options on Linux, and they are far from optimal, unlike on BSD-based systems like macOS, where the default `kqueue` is a perfect option to use.
156159

157-
**NOTE:** We are planning to add an `inotify` or similar-based solution to ivykis that could perform similarly to `kqueue`. Stay tuned!
160+
**NOTE:** We are planning to add an `inotify`, `io_uring` or similar-based solution to ivykis that could perform similarly to `kqueue`. Stay tuned!
158161
{: .notice--info}
159162

160163
`works` - Means it is tested and works seamlessly (based on our tests).

doc/_admin-guide/060_Sources/030_Wildcard-file/000_Wildcard-file_options.md

Lines changed: 6 additions & 6 deletions
Original file line numberDiff line numberDiff line change
@@ -117,14 +117,14 @@ Floating-point numbers (for example, **1.5**) can be used as well. Please note,
117117

118118
## monitor-method()
119119

120-
|Accepted values:| auto \| inotify \| poll|
120+
|Accepted values:| auto \| inotify \| kqueue \| poll |
121121
|Default: | auto |
122122

123-
*Description:* If the platform supports inotify, {{ site.product.short_name }} uses it
124-
automatically to detect creation, move, or deletion of the source files. If inotify is not
125-
available, {{ site.product.short_name }} polls the above mentioned file changes as set in
126-
the monitor-freq() option. To force {{ site.product.short_name }} poll the file changes even if
127-
inotify is available, set this option to **poll**.
123+
*Description:* If the platform supports `inotify`, {{ site.product.short_name }} uses it
124+
automatically to detect the creation, move, or deletion of source files. If the platform
125+
supports `kqueue`, that will be used. If neither is available, {{ site.product.short_name }}
126+
polls the aforementioned file changes as set in the monitor-freq() option. To force {{ site.product.short_name }}
127+
to poll the file changes even if `inotify` or `kqueue` is available, set this option to **poll**.
128128

129129
{% include doc/admin-guide/warnings/file-source-follow-warning.md %}
130130

0 commit comments

Comments
 (0)