Skip to content

DOCS-752 - Copilot unstructured logs beta #5196

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 22 commits into from
Apr 25, 2025
Merged
Show file tree
Hide file tree
Changes from 14 commits
Commits
Show all changes
22 commits
Select commit Hold shift + click to select a range
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
50 changes: 50 additions & 0 deletions docs/search/copilot-unstructured-logs-beta.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,50 @@
---
id: copilot-unstructured-logs-beta
title: Sumo Logic Copilot - Unstructured Logs Support (Beta)
description: Streamline your log analysis with Sumo Logic Copilot, our AI-based assistant that simplifies log analysis by letting you ask questions in plain English, even for logs without a well-defined structure.
---

import Iframe from 'react-iframe';
import useBaseUrl from '@docusaurus/useBaseUrl';

<head>
<meta name="robots" content="noindex" />
</head>

<p><a href="/docs/beta"><span className="beta">Beta</span></a></p>

This feature is in Beta. For more information, contact your Sumo Logic account executive.

Unstructured Logs Support for [Sumo Logic Copilot](/docs/search/copilot), our AI assistant, enables it to understand and provide insights from raw, text-based logs, even if they don't follow a structured format like JSON. This means you can ask questions in plain English and get meaningful results from nearly any log data, without requiring Field Extraction Rules (FERs).

## What's new

Currently, [Copilot works best on structured (JSON) logs](/docs/search/copilot/#compatible-log-formats). With this beta update, Copilot automatically applies parsing logic to unstructured logs, even if no FERs are configured. This allows Copilot to interpret logs from many popular data sources out-of-the-box and enables support for a broader range of log types.

* **Broader coverage**. Copilot now parses and generates insights from unstructured log formats, even without FERs, making it useful for environments that include custom or inconsistent log types.
* **Improved usability**. Ask questions in natural language. Copilot interprets your intent and suggests relevant searches, even for raw, non-JSON logs.
* **Performance and reliability**. Response times and suggestion accuracy are consistent with Copilot’s structured log experience.
* **Security and compliance**. The same strict data handling and privacy standards apply. Unstructured Logs Support builds on Copilot’s secure foundation.

### Powered by Intelliparse mode

Copilot now uses [Intelliparse Mode](/docs/search/get-started-with-search/build-search/intelliparse) to extract fields from unstructured logs. This new parsing engine automatically applies parsing logic based on logs already used in your dashboards, allowing Copilot to work with raw logs that don’t follow a consistent format.

You’ll get more meaningful results from logs that are already powering visualizations and queries without needing to manually define parsing logic or create Field Extraction Rules (FERs).

Copilot uses a hidden `intelliparse` operator behind the scenes that's injected automatically into relevant queries to extract fields, making unstructured logs easier to work with.

### Common use cases

* **General log exploration**. Ask questions about unstructured logs that are already used in your dashboards, even if they lack predefined fields.
* **Error triage**. Investigate frequently visualized log data to surface patterns and recurring issues in unstructured formats.
* **Security insights**. Detect anomalies or signs of failed logins by querying raw logs already powering security dashboards.
* **Smarter prioritization**. Copilot focuses on unstructured logs that are visualized in dashboards, helping you get meaningful insights from high-value data sources.

## FAQ

**Will Copilot interpret all my logs?**<br/>
Copilot prioritizes unstructured logs that are already used in dashboards. This improves the relevance of insights and helps focus on high-value logs.

**How is this different from structured log support?**<br/>
Structured logs have predefined fields, allowing Copilot to map queries directly. For unstructured logs, Copilot uses AI and parsing techniques to infer structure on the fly.
Original file line number Diff line number Diff line change
@@ -1,5 +1,5 @@
---
id: intelliparse
id: intelliparse-beta
title: Intelliparse Mode (Beta)
description: Intelliparse mode extends automatic parsing to unstructured logs, allowing you to search and filter logs even when they don’t follow a consistent format like JSON.
---
Expand All @@ -14,16 +14,14 @@ import useBaseUrl from '@docusaurus/useBaseUrl';

This feature is currently available to select customers. Contact your Sumo Logic account representative to request access.

We've introduced a new parsing mode in the Log Search UI: Intelliparse mode. It extends automatic parsing to unstructured logs, allowing you to search and filter logs even when they don’t follow a consistent format like JSON.

<!-- link to Copilot unstructured logs doc -->
We've introduced a new Log Search parsing mode called Intelliparse. It extends automatic parsing to unstructured logs, allowing you to search and filter logs even when they don’t follow a consistent format like JSON.

## Available parsing modes

You can now choose from three parsing options in the log search UI:

* **Intelliparse (new)**. Combines JSON parsing with automatic parsing of unstructured logs using pre-discovered parsers.
* [**Auto Parse**](/docs/search/get-started-with-search/build-search/dynamic-parsing). JSON blocks within logs are automatically parsed.
* **Auto Parse**. JSON blocks within logs are automatically parsed ([learn more](/docs/search/get-started-with-search/build-search/dynamic-parsing)).
* **Manual**. No automatic parsing applied.

<img src={useBaseUrl('img/search/get-started-search/build-search/log-search-parsing-modes.png')} alt="log-search-parsing-modes.png" style={{border: '1px solid gray'}} width="700"/>
Expand Down Expand Up @@ -76,7 +74,6 @@ Copilot uses Intelliparse mode in the background to:

This integration allows Copilot to work with raw, unstructured log data; no setup required on your part.

<!-- When Copilot - Unstructured Logs (Beta) doc has been published, crosslink from there...
Want to learn more about Intelliparse mode? See how it works in Log Search
https://sumologic.atlassian.net/browse/DOCS-752
--->
:::tip
Want to learn more about Intelliparse mode? [See how it works in Log Search](/docs/search/copilot-unstructured-logs).
:::
Loading