Functions, count conditions & more

Lots of new features:

• Long awaited function support! You can now use a slew of functions anywhere that an expression is accepted. concat, contains, parseDate, and substring are useful for strings. Most math functions you can think of are there as well. (Huge thanks to @tstack)
• Bug fix from v.014.0, division will now promote both values to floats (previous code used integer division in some cases). round can be used to get back the old behavior. (@rcoh)
• Sort now accepts arbitrary expressions, eg. sort by length(message) . This is part of a larger effort to accept expressions anywhere that currently accepts a column name. One of the few remaining holdouts is the fields operator. (@rcoh)
• count now supports an optional argument to conditionally count certain records: (@rcoh)

* | json | count(status == 200) as ok, count as all | ok / all as success_ratio

The build is running now & 0.15.0 has been pushed to crates.io -- binaries should be hot off the presses shortly.

Expressions & Performance Improvements

• The expression is language is now much more powerful thanks to @tstack. Everywhere you can use expressions, you can now also use operators: (+, -, *, /). You can also create variables from existing variables using as
• Assorted performance improvements, the biggest one (almost 2x!) came from switching to Jemalloc. (@rcoh)
• parse is now slightly more flexible when...being parsed. Specifically, from can now come anywhere (@rcoh)
• parse will consume any whitespace character when whitespace exists in the pattern. Technically a breaking change, but I don't expect this will break anything IRL. (@rcoh)

Controllable output formats & alias operators

This release adds 2 major new features:

• Support for aliases -- predefined pipelines for specific formats like apache (@DarrenTsung) Add your own!
• Support for structured output like JSON. The render pipeline has been refactored to enable adding more easily. These formats can be used with --output <mode>. Currently supported: json, logfmt, format=<rust fmt string> (@rcoh)

In addition:

• Whitespace is now automatically trimmed from values. This is technically a breaking change. (@rcoh)
• Add a basic benchmark suite to guide performance optimization. (@rcoh)

Users of v0.9.0 and later can upgrade with agrind --self-update

Array support & split operator

This release adds 2 major new features:

• Array support: json will now create an array when parsing a JSON array. Arrays can be accessed with [n], eg. sum(x.y[4]). (@rcoh )
• split operator: The forcing function for array support was the split operator, enabling converting csv & similar formats into arrays for quick and easy analysis. See README for examples (@DarrenTsung )

This release also switches to use the crate of logfmt, allowing publishing releases to cargo, once again.

Users of v0.9.0 and later can upgrade with agrind --self-update

Logfmt, Min/Max, Custom Output Formats

This release adds a number of features contributed from the community:

• logfmt parser: * | logfmt | ... see README for examples (@MikaelSmith)
• min(col) & max(col) aggregate operators (@slaunay)
• --format '{col a}: {col b}' flag for custom formatting -- see README (@MikaelSmith)
• Support for nested records -- nested JSON records can be parsed and accessed with . notation (eg. sum(response.latency_ms)). Previously, nested JSON resulted in a string which need to be reparsed with json from nested. (@rcoh)

Users of v0.9.0 and later can upgrade with agrind --self-update

Support for Complex Filters

The release adds support for complex filters in the query:

• AND
• OR
• NOR

The keywords must always be all CAPS. Currently, expressions must be fully parenthesized. There was a bit of effort for helpful errors, but if you get a confusing error, file an issue and I'll try to improve it! Users of 0.9.0 and above can upgrade with agrind --self-update.

Limit operator + helpful errors

Big release, thanks to @tstack and @akdor1154

• Add the limit operator (@tstack ) see README for more info
• Add nodrop option to the parse operator (@akdor1154 )
• 'abc' is now accepted anywhere we accepted "abc" before. This change is required to make angle-grinder easier to use on Windows. A preliminary windows build should be available fairly shortly. (@rcoh )
• Error messages are now useful! For a few cases, you'll now get a great error message, complete with color, highlighting and location (@tstack)
• Long values in aggregates no longer break the renderer, instead they should be elided with "..." (@rcoh )
• agrind now supports self-updates via the --self-update flag. It will update to the most recent Github release and is useful for Linux users where upgrading was annoying.

Add wildcard search + new operator

• Wildcard/keword search: Thanks to @tstack for contributing big improvements to the filter part of the query. You can now do queries like host-* to match lines like host-44, host-abc, etc.

host-* | parse "host-*" as hostname | count by hostname 

• total operator: As requested by @tallpsmith, angle grinder now has a total(column) operator which will compute a running total for aggregates. Examples in the docs.

* | sum(num_queries) by host, url | total(_sum)

Small bugfixes

v0.7.6

Bump version

Rendering Pipeline Improvements

The rendering pipeline is now multi threaded -- updates to the UI no longer block the main reader thread. This update also fixes a performance bug where output being written to a file was significantly slower than output being written to the terminal.