Can I avoid double json_parse() while filtering and remapping my events?

[remort]

I have to parse events coming from a file which are stored there as JSON-strings.
First I need to filter and event by some criterion, say value in a field. I do parse_json!(.message) in filter block.
Then I need to transform a message. I do parse_json!(.message) again in remap block.

Is there a way to pass an already parsed structure from filter to remap to work with a structure in remap instead of parsing the source string again? I suppose this is CPU consuming after all.

[jszwedko]

I'd recommend doing the parsing in a remap transform first, and then, doing the filter. Filter conditions are read-only and so cannot modify the event.

[remort]

Thanks for the quick answer. Yes I tried it first: remap then filter. But I need to parse and modify certain fields or transform another ones, there is some non-trivial conditional logic in remap. Before doing this, I'd like to filter unrelated events first. If I use ! in function calls during remap - their failures pollute Vector logs. Is there a way to skip/drop an event if function failed in remap silently?

[jszwedko]

Could you add a separate remap transform that just does the JSON parsing, passes through a filter transform, and then the subsequent remap transform to do the additional parsing?

Another thing you can do is utilize the abort keyword in VRL to drop the event without an error like:

., err = parse_json(.message)
if err != null {
  abort
}

# here . will be the parsed JSON

Hopefully that helps!

[remort]

The subsequent remap going after a filter need to to JSON parsing again, doesn't it ?
Abort snippet looks like if it could be my case. I'll try both of them. Thank you!

[jszwedko]

If you do the json parsing in the first remap transform, it'll already be parsed when it hits the filter transform and the second remap transform.

[remort]

Solution with abort in remap works perfectly and looks more concise to me than chaining remap with filters. I think my case is resolved. Thank you for your time.