Skip to content

axosyslog-4.12.0

Latest
Latest
Compare
Choose a tag to compare
@MrAnno MrAnno released this 19 Jun 16:42
· 76 commits to main since this release
axosyslog-4.12.0
6a59897
This commit was created on GitHub.com and signed with GitHub’s verified signature.
GPG key ID: B5690EEEBB952194
Verified
Learn about vigilant mode.

4.12.0

AxoSyslog is binary-compatible with syslog-ng [1] and serves as a drop-in replacement.

We provide cloud-ready container images and Helm charts.

Packages are available in our APT and RPM repositories (Ubuntu, Debian, AlmaLinux, Fedora).

Check out the AxoSyslog documentation for all the details.

License

The licensing of AxoSyslog has been simplified and upgraded from a combination of LGPL-2.1-or-later and GPL-2.0-or-later to GPL-3.0-or-later.

As before, Contributory Licensing Agreements (CLAs) are NOT required to contribute to AxoSyslog: contributors retain their own copyright, making AxoSyslog a combination of code from hundreds of individuals and companies.
This, and the use of GPL v3 ensures that AxoSyslog or AxoSyslog derived code cannot become proprietary software.

While this has basically no impact on users of AxoSyslog, it reflects a step towards a more open and more community-friendly project. Read more here

Features

  • clickhouse() destination: Added server-side-schema() option.

    Sending data to ClickHouse in Protobuf format has some limitations.
    The Protobuf formatted data does not carry type information,
    so there are two ways of determining the type mapping in ClickHouse:

    1. Using autogenerated schema: https://clickhouse.com/docs/interfaces/formats/Protobuf#using-autogenerated-protobuf-schema

      This method looks at the table that the data is getting inserted
      into and generates the default protobuf mapping for it.

      The problem with this solution is that some more complex column
      types can receive data from different input types, for example
      DateTime64 can recieve from uint64, string, float, but with the
      autogenerated schema you can utilize only one of them.

    2. Using format schema: https://clickhouse.com/docs/interfaces/formats#formatschema

      This method solves the issue of the autogenerated schema, but
      needs a more complex setup. First you need to place a .proto
      file along your server in a specific directory, then you can
      reference that schema during insert, which is done by the
      server-side-schema() option.

    Example: server-side-schema("my_proto_file_on_server:my_message_schema_name")
    (#666)

  • log-flow-control(yes/no) global option

    This option allows enabling flow control for all log paths. When set to yes,
    flow control is globally enabled, but it can still be selectively disabled
    within individual log paths using the no-flow-control flag.

    WARNING: Enabling global flow control can cause the system() source to block.
    As a result, if messages accumulate at the destination, applications that log
    through the system may become completely stalled, potentially halting their
    operation. We don't recommend enabling flow control in log paths that
    include the system() source.

    For example,

    options {
  log-flow-control(yes);
};

log {
  source { system(); };
  destination { network("server" port(5555)); };
  flags(no-flow-control);
};

log { ... };

    (#606)

FilterX features

  • Failure information tracking

    The following functions have been added to allow tracking failures in FilterX code:

    • failure_info_enable(), optional parameter: collect_falsy=true/false, defaults to false:
      Enable failure information collection from this point downwards through all branches of the pipeline.

    • failure_info_clear():
      Clear accumulated failure information

    • failure_info_meta({}):
      Attach metadata to the given section of FilterX code. The metadata remains in effect until the next call
      or until the end of the enclosing FilterX block, whichever comes first.

    • failure_info():
      Return failure information as a FilterX dictionary. This should ideally be called as late as possible, for example, in the last log path of your configuration or within a fallback path.

    Example output:

    [
  {
    "meta": {
      "step": "Setting common fields"
    },
    "location": "/etc/syslog-ng/syslog-ng.conf:33:7",
    "line": "nonexisting.key = 13;",
    "error": "No such variable: nonexisting"
  }
]

    (#629)

  • - * / %: Added new filterx arithmetic operators.

    Example usage:

    if (3 - 3 == 0)
if (3.0 * 3 == 9.0)
if (3.0 / 3.0 == 1.0)
if (4 % 3 == 1)

    Note: A general + operator already exists. % only accepts integer values.
    (#625)

  • in: Added new filterx operator for membership check.

    Example usage:

    s_arr = ['foo', 'bar', 'asd'];
if ("foo" in s_arr)

    or

    s_arr = ['foo', 'bar', 'asd'];
if ("bar" not in s_arr)

    (#617)

  • get_timestamp(): query the timestamps of log messages
    (#586)

  • strcasecmp(): case insensitive string comparison
    (#580)

  • update_metric(): Labels containing null and "" (empry string) values are now skipped.
    (#671)

Bugfixes

  • Fixed some time parsing and time formatting issues.
    (#626)

  • network(), syslog() destinations: handle async TLS messages (KeyUpdate, etc.)
    (#609)

  • rate-limit(): fix precision issue that could occur at a very low message rate
    (#599)

  • rewrite: fix not creating empty values for non-matching regexp optional match groups
    (#669)

  • regexp-parser(): fix double-free with invalid configuration
    (#670)

  • metrics: fix syslogng_last_config_file_modification_timestamp_seconds
    (#612)

  • pubsub-grpc() destination: Fixed message size counting.
    (#652)

  • collectd(): fix not reading server responses
    (#609)

  • network()/syslog() TLS destinations: fix a possible infinite looping issue
    (#615)

FilterX bugfixes

  • metrics_labels: Fixed a crash that occurred when trying to get a label from an empty object.
    (#601)

  • regexp_*(): fix not creating empty values for non-matching optional match groups
    (#669)

  • cache_json_file(): fix updating json content on file changes
    (#612)

Other changes

  • loggen: statistics output has slightly changed

    The new --perf option can be used to measure the log throughput of AxoSyslog.
    (#598)

  • stats(): freq() now defaults to 0

    Internal statistic messages were produced every 10 minutes by default.
    Metrics are available through syslog-ng-ctl, we believe modern monitoring and
    observability render this periodic message obsolete.
    (#600)

[1] syslog-ng is a trademark of One Identity.

Discord

For a bit more interactive discussion, join our Discord server:

Axoflow Discord Server

Credits

AxoSyslog is developed as a community project, and as such it relies
on volunteers, to do the work necessary to produce AxoSyslog.

Reporting bugs, testing changes, writing code or simply providing
feedback is an important contribution, so please if you are a user
of AxoSyslog, contribute.

We would like to thank the following people for their contribution:

Andras Mitzki, Attila Szakacs, Balazs Scheidler, Bálint Horváth,
Christian Heusel, Eli Schwartz, Franco Fichtner, Hofi,
Kovacs, Gergo Ferenc, László Várady, Mate Ory, Tamás Kosztyu, shifter

Assets 3
Loading