Skip to content

Commit 81bc3da

Browse files
tothmanotothmano
authored
Add bag functions (#297)
1 parent 5b2b71e commit 81bc3da

File tree

5 files changed

+479
-1
lines changed

5 files changed

+479
-1
lines changed

‎apl/scalar-functions/array-functions.mdx

Lines changed: 4 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -23,6 +23,9 @@ The table summarizes the array functions available in APL.
2323
| [array_slice](/apl/scalar-functions/array-functions/array-slice) | Extracts a slice of a dynamic array. |
2424
| [array_split](/apl/scalar-functions/array-functions/array-split) | Splits an array to multiple arrays according to the split indices and packs the generated array in a dynamic array. |
2525
| [array_sum](/apl/scalar-functions/array-functions/array-sum) | Calculates the sum of elements in a dynamic array. |
26+
| [bag_has_key](/apl/scalar-functions/array-functions/bag-has-key) | Checks whether a dynamic property bag contains a specific key. |
27+
| [bag_keys](/apl/scalar-functions/array-functions/bag-keys) | Returns all keys in a dynamic property bag. |
28+
| [bag_pack](/apl/scalar-functions/array-functions/bag-pack) | Converts a list of key-value pairs to a dynamic property bag. |
2629
| [isarray](/apl/scalar-functions/array-functions/isarray) | Checks whether a value is an array. |
2730
| [pack_array](/apl/scalar-functions/array-functions/pack-array) | Packs all input values into a dynamic array. |
2831
| [strcat_array](/apl/scalar-functions/array-functions/strcat-array) | Takes an array and returns a single concatenated string with the array’s elements separated by the specified delimiter. |
@@ -31,4 +34,4 @@ The table summarizes the array functions available in APL.
3134

3235
Most array functions accept a dynamic array as their parameter. Dynamic arrays allow you to add or remove elements. You can change a dynamic array with an array function.
3336

34-
A dynamic array expands as you add more elements. This means that you don’t need to determine the size in advance.
37+
A dynamic array expands as you add more elements. This means that you don’t need to determine the size in advance.

‎apl/scalar-functions/array-functions/bag-has-key.mdx

Lines changed: 156 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,156 @@
1+
---
2+
title: bag_has_key
3+
description: 'This page explains how to use the bag_has_key function in APL.'
4+
---
5+
6+
Use the `bag_has_key` function in APL to check whether a dynamic property bag contains a specific key. This is helpful when your data includes semi-structured or nested fields encoded as dynamic objects, such as JSON-formatted logs or telemetry metadata.
7+
8+
You often encounter property bags in observability data where log entries, spans, or alerts carry key–value metadata. Use `bag_has_key` to filter, conditionally process, or join such records based on the existence of specific keys, without needing to extract the values themselves.
9+
10+
## For users of other query languages
11+
12+
If you come from other query languages, this section explains how to adjust your existing queries to achieve the same results in APL.
13+
14+
<AccordionGroup>
15+
<Accordion title="Splunk SPL users">
16+
17+
In Splunk SPL, you often check whether a key exists in a JSON object using `spath` and conditional logic. APL simplifies this with `bag_has_key`, which returns a boolean directly and avoids explicit parsing.
18+
19+
<CodeGroup>
20+
```sql Splunk example
21+
| eval hasKey=if(isnull(spath(data, "keyName")), false, true)
22+
````
23+
24+
```kusto APL equivalent
25+
['sample-http-logs']
26+
| where bag_has_key(dynamic_field, 'keyName')
27+
```
28+
29+
</CodeGroup>
30+
31+
</Accordion>
32+
<Accordion title="ANSI SQL users">
33+
34+
ANSI SQL doesn’t include native support for property bags or dynamic fields. You typically use JSON functions to access keys in JSON-formatted strings. In APL, dynamic fields are first-class, and `bag_has_key` provides direct support for key existence checks.
35+
36+
<CodeGroup>
37+
```sql SQL example
38+
SELECT *
39+
FROM logs
40+
WHERE JSON_EXTRACT(json_column, '$.keyName') IS NOT NULL
41+
```
42+
43+
```kusto APL equivalent
44+
['sample-http-logs']
45+
| where bag_has_key(dynamic_field, 'keyName')
46+
```
47+
48+
</CodeGroup>
49+
50+
</Accordion>
51+
</AccordionGroup>
52+
53+
## Usage
54+
55+
### Syntax
56+
57+
```kusto
58+
bag_has_key(bag: dynamic, key: string)
59+
```
60+
61+
### Parameters
62+
63+
| Name | Type | Description |
64+
| ----- | --------- | ---------------------------------------------------------------- |
65+
| `bag` | `dynamic` | A dynamic value representing a property bag (e.g., JSON object). |
66+
| `key` | `string` | The key to check for within the property bag. |
67+
68+
### Returns
69+
70+
Returns a `bool` value:
71+
72+
- `true` if the specified key exists in the property bag
73+
- `false` otherwise
74+
75+
## Use case examples
76+
77+
<Tabs>
78+
<Tab title="Log analysis">
79+
80+
Use `bag_has_key` to filter log entries that include a specific metadata key embedded in a dynamic object.
81+
82+
**Query**
83+
84+
```kusto
85+
['sample-http-logs']
86+
| extend metadata = bag_pack('source', 'cdn', 'env', 'prod')
87+
| where bag_has_key(metadata, 'env')
88+
| project _time, id, method, uri, status, metadata
89+
```
90+
91+
[Run in Playground](https://play.axiom.co/axiom-play-qf1k/query?initForm=%7B%22apl%22%3A%22%5B%27sample-http-logs%27%5D%20%7C%20extend%20metadata%20%3D%20bag_pack%28%27source%27%2C%20%27cdn%27%2C%20%27env%27%2C%20%27prod%27%29%20%7C%20where%20bag_has_key%28metadata%2C%20%27env%27%29%20%7C%20project%20_time%2C%20id%2C%20method%2C%20uri%2C%20status%2C%20metadata%22%7D)
92+
93+
**Output**
94+
95+
| _time | id | method | uri | status | metadata |
96+
| ----------------- | ---- | ------ | -------------- | ------ | ----------------------------- |
97+
| 2025-05-27T12:30Z | u123 | GET | /login | 200 | \{'source':'cdn','env':'prod'\} |
98+
| 2025-05-27T12:31Z | u124 | POST | /cart/checkout | 500 | \{'source':'cdn','env':'prod'\} |
99+
100+
The query filters logs where the synthetic `metadata` bag includes the key `'env'`.
101+
102+
</Tab>
103+
<Tab title="OpenTelemetry traces">
104+
105+
Use `bag_has_key` to filter spans that include specific dynamic span attributes.
106+
107+
**Query**
108+
109+
```kusto
110+
['otel-demo-traces']
111+
| extend attributes = bag_pack('user', 'alice', 'feature_flag', 'beta')
112+
| where bag_has_key(attributes, 'feature_flag')
113+
| project _time, trace_id, span_id, ['service.name'], kind, attributes
114+
```
115+
116+
[Run in Playground](https://play.axiom.co/axiom-play-qf1k/query?initForm=%7B%22apl%22%3A%22%5B%27otel-demo-traces%27%5D%20%7C%20extend%20attributes%20%3D%20bag_pack%28%27user%27%2C%20%27alice%27%2C%20%27feature_flag%27%2C%20%27beta%27%29%20%7C%20where%20bag_has_key%28attributes%2C%20%27feature_flag%27%29%20%7C%20project%20_time%2C%20trace_id%2C%20span_id%2C%20%5B%27service.name%27%5D%2C%20kind%2C%20attributes%22%7D)
117+
118+
**Output**
119+
120+
| _time | trace_id | span_id | ['service.name'] | kind | attributes |
121+
| ----------------- | --------- | -------- | ----------------- | ------ | --------------------------------------- |
122+
| 2025-05-27T10:02Z | abc123 | span567 | frontend | client | \{'user':'alice','feature_flag':'beta'\} |
123+
124+
The query selects spans with dynamic `attributes` bags containing the `'feature_flag'` key.
125+
126+
</Tab>
127+
<Tab title="Security logs">
128+
129+
Use `bag_has_key` to identify HTTP logs where the request metadata contains sensitive audit-related keys.
130+
131+
**Query**
132+
133+
```kusto
134+
['sample-http-logs']
135+
| extend audit_info = bag_pack('action', 'delete', 'reason', 'admin_override')
136+
| where bag_has_key(audit_info, 'reason')
137+
| project _time, id, uri, status, audit_info
138+
```
139+
140+
[Run in Playground](https://play.axiom.co/axiom-play-qf1k/query?initForm=%7B%22apl%22%3A%22%5B%27sample-http-logs%27%5D%20%7C%20extend%20audit_info%20%3D%20bag_pack%28%27action%27%2C%20%27delete%27%2C%20%27reason%27%2C%20%27admin_override%27%29%20%7C%20where%20bag_has_key%28audit_info%2C%20%27reason%27%29%20%7C%20project%20_time%2C%20id%2C%20uri%2C%20status%2C%20audit_info%22%7D)
141+
142+
**Output**
143+
144+
| _time | id | uri | status | audit_info |
145+
| ----------------- | ---- | ------------- | ------ | ---------------------------------------------- |
146+
| 2025-05-27T13:45Z | u999 | /admin/delete | 403 | \{'action':'delete','reason':'admin_override'\} |
147+
148+
The query returns only logs where the `audit_info` bag includes the `'reason'` key, indicating administrative override events.
149+
150+
</Tab>
151+
</Tabs>
152+
153+
## List of related functions
154+
155+
- [bag_keys](/apl/scalar-functions/array-functions/bag-keys): Returns all keys in a dynamic property bag. Use it when you need to enumerate available keys.
156+
- [bag_pack](/apl/scalar-functions/array-functions/bag-pack): Converts a list of key-value pairs to a dynamic property bag. Use when you need to build a bag.

‎apl/scalar-functions/array-functions/bag-keys.mdx

Lines changed: 161 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,161 @@
1+
---
2+
title: bag_keys
3+
description: 'This page explains how to use the bag_keys function in APL.'
4+
---
5+
6+
Use the `bag_keys` function in APL to extract the keys of a dynamic (bag) object as an array of strings. This is useful when you want to inspect or manipulate the structure of a dynamic field—such as JSON-like nested objects—without needing to know its exact schema in advance.
7+
8+
Use `bag_keys` when you’re working with semi-structured data and want to:
9+
10+
- Discover what properties are present in a dynamic object.
11+
- Iterate over the keys programmatically using other array functions.
12+
- Perform validation or debugging tasks to ensure all expected keys exist.
13+
14+
This function is especially helpful in log analytics, observability pipelines, and security auditing, where dynamic properties are often collected from various services or devices.
15+
16+
## For users of other query languages
17+
18+
If you come from other query languages, this section explains how to adjust your existing queries to achieve the same results in APL.
19+
20+
<AccordionGroup>
21+
<Accordion title="Splunk SPL users">
22+
23+
In Splunk SPL, you typically interact with JSON-like fields using the `spath` command or use `keys(_raw)` to retrieve field names. In APL, `bag_keys` serves a similar purpose by returning an array of keys from a dynamic object.
24+
25+
<CodeGroup>
26+
```sql Splunk example
27+
| eval key_list=keys(data_field)
28+
````
29+
30+
```kusto APL equivalent
31+
datatable(data: dynamic)
32+
[
33+
dynamic({ "ip": "127.0.0.1", "status": "200", "method": "GET" })
34+
]
35+
| extend keys = bag_keys(data)
36+
```
37+
38+
</CodeGroup>
39+
40+
</Accordion>
41+
<Accordion title="ANSI SQL users">
42+
43+
ANSI SQL doesn’t have native support for dynamic objects or JSON key introspection in the same way. However, some SQL dialects (like PostgreSQL or BigQuery) provide JSON-specific functions for extracting keys. `bag_keys` is the APL equivalent for dynamically introspecting JSON objects.
44+
45+
<CodeGroup>
46+
```sql SQL example
47+
SELECT JSON_OBJECT_KEYS(data) FROM logs;
48+
```
49+
50+
```kusto APL equivalent
51+
datatable(data: dynamic)
52+
[
53+
dynamic({ "ip": "127.0.0.1", "status": "200", "method": "GET" })
54+
]
55+
| extend keys = bag_keys(data)
56+
```
57+
58+
</CodeGroup>
59+
60+
</Accordion>
61+
</AccordionGroup>
62+
63+
## Usage
64+
65+
### Syntax
66+
67+
```kusto
68+
bag_keys(bag)
69+
```
70+
71+
### Parameters
72+
73+
| Name | Type | Description |
74+
| ----- | --------- | -------------------------------------------------- |
75+
| `bag` | `dynamic` | The dynamic object whose keys you want to extract. |
76+
77+
### Returns
78+
79+
An array of type `string[]` containing the names of the keys in the dynamic object. If the input is not a dynamic object, the function returns `null`.
80+
81+
## Use case examples
82+
83+
<Tabs>
84+
<Tab title="Log analysis">
85+
86+
Use `bag_keys` to audit dynamic metadata fields in HTTP logs where each record contains a nested object representing additional request attributes.
87+
88+
**Query**
89+
90+
```kusto
91+
['sample-http-logs']
92+
| extend metadata = dynamic({ 'os': 'Windows', 'browser': 'Firefox', 'device': 'Desktop' })
93+
| extend key_list = bag_keys(metadata)
94+
| project _time, uri, metadata, key_list
95+
```
96+
97+
[Run in Playground](https://play.axiom.co/axiom-play-qf1k/query?initForm=%7B%22apl%22%3A%22%5B'sample-http-logs'%5D%20%7C%20extend%20metadata%20%3D%20dynamic(%7B%20'os'%3A%20'Windows'%2C%20'browser'%3A%20'Firefox'%2C%20'device'%3A%20'Desktop'%20%7D)%20%7C%20extend%20key_list%20%3D%20bag_keys(metadata)%20%7C%20project%20_time%2C%20uri%2C%20metadata%2C%20key_list%22%7D)
98+
99+
**Output**
100+
101+
| _time | uri | metadata | key_list |
102+
| ------------------- | ------ | ------------------------------------------------ | ---------------------------- |
103+
| 2025-05-26 12:01:23 | /login | \{os: Windows, browser: Firefox, device: Desktop\} | [‘os’, ‘browser’, ‘device’] |
104+
105+
This query inspects a simulated metadata object and returns the list of its keys, helping you debug inconsistencies or missing fields.
106+
107+
</Tab>
108+
<Tab title="OpenTelemetry traces">
109+
110+
Use `bag_keys` to examine custom span attributes encoded as dynamic fields within OpenTelemetry trace events.
111+
112+
**Query**
113+
114+
```kusto
115+
['otel-demo-traces']
116+
| extend attributes = dynamic({ 'user_id': 'abc123', 'feature_flag': 'enabled' })
117+
| extend attribute_keys = bag_keys(attributes)
118+
| project _time, ['service.name'], kind, attributes, attribute_keys
119+
```
120+
121+
[Run in Playground](https://play.axiom.co/axiom-play-qf1k/query?initForm=%7B%22apl%22%3A%22%5B'otel-demo-traces'%5D%20%7C%20extend%20attributes%20%3D%20dynamic(%7B%20'user_id'%3A%20'abc123'%2C%20'feature_flag'%3A%20'enabled'%20%7D)%20%7C%20extend%20attribute_keys%20%3D%20bag_keys(attributes)%20%7C%20project%20_time%2C%20%5B'service.name'%5D%2C%20kind%2C%20attributes%2C%20attribute_keys%22%7D)
122+
123+
**Output**
124+
125+
| _time | ['service.name'] | kind | attributes | attribute_keys |
126+
| ------------------- | ----------------- | ------ | ------------------------------------------ | ------------------------------ |
127+
| 2025-05-26 13:14:01 | frontend | client | \{user_id: abc123, feature_flag: enabled\} | [‘user_id’, ‘feature_flag’] |
128+
129+
This query inspects the custom span-level attributes and extracts their keys to verify attribute coverage or completeness.
130+
131+
</Tab>
132+
<Tab title="Security logs">
133+
134+
Use `bag_keys` to list all security-related fields captured dynamically during request monitoring for auditing or compliance.
135+
136+
**Query**
137+
138+
```kusto
139+
['sample-http-logs']
140+
| extend security_context = dynamic({ 'auth_status': 'success', 'role': 'admin', 'ip': '192.168.1.5' })
141+
| extend fields = bag_keys(security_context)
142+
| project _time, status, ['geo.country'], security_context, fields
143+
```
144+
145+
[Run in Playground](https://play.axiom.co/axiom-play-qf1k/query?initForm=%7B%22apl%22%3A%22%5B'sample-http-logs'%5D%20%7C%20extend%20security_context%20%3D%20dynamic(%7B%20'auth_status'%3A%20'success'%2C%20'role'%3A%20'admin'%2C%20'ip'%3A%20'192.168.1.5'%20%7D)%20%7C%20extend%20fields%20%3D%20bag_keys(security_context)%20%7C%20project%20_time%2C%20status%2C%20%5B'geo.country'%5D%2C%20security_context%2C%20fields%22%7D)
146+
147+
**Output**
148+
149+
| _time | status | ['geo.country'] | security_context | fields |
150+
| ------------------- | ------ | ---------------- | ----------------------------------------------------- | ------------------------------- |
151+
| 2025-05-26 15:32:10 | 200 | US | \{auth_status: success, role: admin, ip: 192.168.1.5\} | [‘auth_status’, ‘role’, ‘ip’] |
152+
153+
This helps you audit security metadata in requests and ensure key fields are present across records.
154+
155+
</Tab>
156+
</Tabs>
157+
158+
## List of related functions
159+
160+
- [bag_pack](/apl/scalar-functions/array-functions/bag-pack): Converts a list of key-value pairs to a dynamic property bag. Use when you need to build a bag.
161+
- [bag_has_key](/apl/scalar-functions/array-functions/bag-has-key): Checks whether a dynamic property bag contains a specific key.

0 commit comments

Comments
 (0)