Skip to content

Initial Draft of Confluent Cloud Document #5495

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
wants to merge 2 commits into
base: main
Choose a base branch
from
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion docs/integrations/product-list/product-list-a-l.md
Original file line number Diff line number Diff line change
Expand Up @@ -162,7 +162,7 @@ For descriptions of the different types of integrations Sumo Logic offers, see [
| <img src={useBaseUrl('img/platform-services/automation-service/app-central/logos/cofense.png')} alt="Thumbnail icon" width="50"/> | [Cofense](https://cofense.com/) | Automation integration: [Cofense](/docs/platform-services/automation-service/app-central/integrations/cofense/) |
| <img src={useBaseUrl('img/integrations/misc/commscope-logo.svg')} alt="Thumbnail icon" width="100"/> | [CommScope](https://www.commscope.com/) | Cloud SIEM integration: [CommScope](https://github.com/SumoLogic/cloud-siem-content-catalog/blob/master/vendors/e69bff14-6237-4997-a447-4fc02feaeeaf.md) |
| <img src={useBaseUrl('img/integrations/misc/configcat-logo.png')} alt="Thumbnail icon" width="50"/> | [ConfigCat](https://configcat.com/) | Webhook: [ConfigCat](/docs/integrations/webhooks/configcat/) |
| <img src={useBaseUrl('img/send-data/confluent-cloud-metrics.png')} alt="Thumbnail icon" width="100"/> | [Confluent](https://www.confluent.io/) | Collector: [Confluent Cloud Metrics Source](/docs/send-data/hosted-collectors/cloud-to-cloud-integration-framework/confluent-cloud-metrics-source) |
| <img src={useBaseUrl('img/send-data/confluent-cloud-metrics.png')} alt="Thumbnail icon" width="100"/> | [Confluent](https://www.confluent.io/) | App: [Confluent Cloud](/docs/integrations/saas-cloud/confluent-cloud)<br/>Collector: [Confluent Cloud Metrics Source](/docs/send-data/hosted-collectors/cloud-to-cloud-integration-framework/confluent-cloud-metrics-source) |
| <img src={useBaseUrl('img/platform-services/automation-service/app-central/logos/connectwise-manage.png')} alt="Thumbnail icon" width="100"/> | [ConnectWise PSA](https://info.connectwise.com/professional-services-automation/manage/demo/sem/demo) | Automation integration: [ConnectWise Manage](/docs/platform-services/automation-service/app-central/integrations/connectwise-manage/) |
| <img src={useBaseUrl('img/integrations/misc/contrast-security-logo.png')} alt="Thumbnail icon" width="100"/> | [Contrast Security](https://www.contrastsecurity.com/) | Cloud SIEM integration: [Contrast Security](https://github.com/SumoLogic/cloud-siem-content-catalog/blob/master/vendors/024b4302-9a57-4c60-9d0d-693b2d18225b.md) |
| <img src={useBaseUrl('img/platform-services/automation-service/app-central/logos/coralogix-query-logs.png')} alt="Thumbnail icon" width="75"/> | [Coralogix](https://coralogix.com/) | Automation integrations: <br/>- [Coralogix - Query Logs](/docs/platform-services/automation-service/app-central/integrations/coralogix-query-logs/) <br/>- [Coralogix - Send Logs](/docs/platform-services/automation-service/app-central/integrations/coralogix-send-logs/) |
Expand Down
107 changes: 107 additions & 0 deletions docs/integrations/saas-cloud/confluent-cloud.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,107 @@
---
id: confluent-cloud
title: Confluent Cloud
sidebar_label: Confluent Cloud
description: The Sumo Logic App for Confluent Cloud provides deep visibility into your Confluent Cloud environment. The app’s dashboards include preconfigured searches and filters to help you monitor Kafka cluster performance, topic-level throughput, consumer lag, producer latency, connector health, Flink compute pool utilization, KSQL activity, and schema registry operations—enabling proactive troubleshooting and streamlined data streaming operations.
---

import useBaseUrl from '@docusaurus/useBaseUrl';

<img src={useBaseUrl('img/integrations/saas-cloud/confluent-cloud.png')} alt="Thumbnail icon" width="100"/>

Confluent Cloud is a fully managed, cloud-native data streaming platform based on Apache Kafka. It enables real-time data processing, integration, and movement across applications and systems without managing infrastructure. With built-in support for connectors, ksqlDB, and stream governance, it simplifies building scalable, event-driven architectures.

The Sumo Logic App for Confluent Cloud provides deep visibility into your Confluent Cloud environment. The app’s dashboards include preconfigured searches and filters to help you monitor Kafka cluster performance, topic-level throughput, consumer lag, producer latency, connector health, Flink compute pool utilization, KSQL activity, and schema registry operations—enabling proactive troubleshooting and streamlined data streaming operations.

## Metrics type

The Confluent Cloud app uses metrics:
* [Confluent Cloud runtime metrics](https://api.telemetry.confluent.cloud/docs/descriptors/datasets/cloud).

### Collect Metrics

Sumo Logic supports collecting metrics by C2C source:

* Configure a [Confluent Cloud Metrics Source](/docs/send-data/hosted-collectors/cloud-to-cloud-integration-framework/confluent-cloud-metrics-source/)

## Installing the Confluent Cloud

Now that you have set up a collection for **Confluent Cloud**, install the Sumo Logic app to use the pre-configured [dashboards](#viewing-the-confluent-cloud-dashboards) that provide visibility into your environment for real-time analysis of overall usage.

import AppInstall from '../../reuse/apps/app-install-v2.md';

<AppInstall/>

## Viewing the Confluent Cloud dashboards

### Confluent Cloud - Compute Pool

Confluent Cloud - Compute Pool dashboard provides details on the status and utilization of compute resources in your Confluent Cloud environment. Use this dashboard to:

* Monitor the average Flink statement status to identify potential issues with stream processing jobs.
* Track CPU utilization against the set limit to ensure optimal resource allocation and prevent performance bottlenecks.
* Analyze trends in CPU minutes consumed over time to optimize cost and capacity planning for your Confluent Cloud deployment.
* Compare current CPU usage across different compute pools to balance workloads and improve overall system efficiency.

<img src={useBaseUrl('https://sumologic-app-data-v2.s3.amazonaws.com/dashboards/ConfluentCloud/Confluent-Cloud-Compute-Pool.png')} alt="Confluent Cloud - Compute Pool" style={{border: '1px solid gray'}} width="800" />

### Confluent Cloud - Connectors

Confluent Cloud - Connectors dashboard provides details on the status, performance, and health of Confluent Cloud connectors.

Use this dashboard to:
* Monitor the status of connectors and their tasks to quickly identify any that are failed, paused, or provisioning.
* Track the volume of sent and received records across different connectors to ensure data flow is as expected.
* Analyze the average sent and received bytes to detect any unusual spikes or drops in data transfer.
* Identify potential bottlenecks by correlating connector status with sent/received records and bytes.
* Monitor dead letter queue records to catch any messages that failed to be processed correctly.

<img src={useBaseUrl('https://sumologic-app-data-v2.s3.amazonaws.com/dashboards/ConfluentCloud/Confluent-Cloud-Connectors.png')} alt="Confluent Cloud - Connectors" style={{border: '1px solid gray'}} width="800" />

### Confluent Cloud - Kafka Cluster

Confluent Cloud - Kafka Cluster dashboard provides details on key performance metrics and operational statistics for your Kafka cluster in Confluent Cloud.

Use this dashboard to:
* Monitor producer latency and cluster load to ensure optimal performance of your Kafka cluster.
* Track active connections, request bytes, and response bytes to identify potential bottlenecks or unusual traffic patterns.
* Analyze request count trends over time to understand usage patterns and plan for capacity needs.

<img src={useBaseUrl('https://sumologic-app-data-v2.s3.amazonaws.com/dashboards/ConfluentCloud/Confluent-Cloud-Kafka-Cluster.png')} alt="Confluent Cloud - Kafka Cluster" style={{border: '1px solid gray'}} width="800" />

### Confluent Cloud - Kafka Cluster Topic

Confluent Cloud - Kafka Cluster Topic dashboard provides details on topic-level metrics for Kafka clusters in Confluent Cloud, including consumer lag, data throughput, and partition information.

Use this dashboard to:
* Monitor consumer lag offsets to identify potential bottlenecks in data processing and ensure timely consumption of messages.
* Analyze the relationship between received and sent bytes to detect any data loss or transmission issues within the Kafka cluster.
* Track the number of received and sent records over time to understand topic usage patterns and optimize resource allocation.
* Observe retained bytes and partition count to manage storage utilization and ensure proper topic scaling.

<img src={useBaseUrl('https://sumologic-app-data-v2.s3.amazonaws.com/dashboards/ConfluentCloud/Confluent-Cloud-Kafka-Cluster-Topic.png')} alt="Confluent Cloud - Kafka Cluster Topic" style={{border: '1px solid gray'}} width="800" />

### Confluent Cloud - KSQL

Confluent Cloud - KSQL dashboard provides details on query performance, data processing, and resource utilization for KSQL operations in Confluent Cloud.

Use this dashboard to:
* Monitor query saturation and identify potential bottlenecks in KSQL operations.
* Track processing errors and query restarts to ensure smooth data flow and query execution.
* Analyze storage utilization and streaming unit consumption to optimize resource allocation.
* Correlate offset lag with processing errors to troubleshoot data processing delays.
* Examine produced and consumed bytes to understand data throughput and potential network issues.

<img src={useBaseUrl('https://sumologic-app-data-v2.s3.amazonaws.com/dashboards/ConfluentCloud/Confluent-Cloud-KSQL.png')} alt="Confluent Cloud - KSQL" style={{border: '1px solid gray'}} width="800" />

### Confluent Cloud - Schema Registry

Confluent Cloud - Schema Registry dashboard provides details on schema operations, request counts, and traffic types for your Confluent Cloud Schema Registry.

Use this dashboard to:
* Monitor the distribution of schema operations (CREATE, READ, DELETE) to ensure proper usage and identify potential issues with schema management.
* Analyze the ratio of public vs. private traffic to understand access patterns and potential security concerns.
* Track schema count trends over time to manage growth and capacity planning for your Schema Registry.
* Correlate request counts with operation types to identify unusual spikes or drops in schema-related activities.

<img src={useBaseUrl('https://sumologic-app-data-v2.s3.amazonaws.com/dashboards/ConfluentCloud/Confluent-Cloud-Schema-Registry.png')} alt="Confluent Cloud - Schema Registry" style={{border: '1px solid gray'}} width="800" />
6 changes: 6 additions & 0 deletions docs/integrations/saas-cloud/index.md
Original file line number Diff line number Diff line change
Expand Up @@ -129,6 +129,12 @@ Learn about the Sumo Logic apps for SaaS and Cloud applications.
<p>Monitor, detect, and respond to potential data threats on your Code42 Incydr platform.</p>
</div>
</div>
<div className="box smallbox card">
<div className="container">
<a href="/docs/integrations/saas-cloud/confluent-cloud"><img src={useBaseUrl('img/send-data/confluent-cloud-logo.png')} alt="icon" width="100"/><h4>Confluent Cloud</h4></a>
<p>Monitor and analyze streaming activity in Confluent Cloud to ensure secure, reliable, and high-performance data pipelines.</p>
</div>
</div>
<div className="box smallbox card">
<div className="container">
<a href="/docs/integrations/saas-cloud/crowdstrike-falcon-filevantage"><img src={useBaseUrl('img/integrations/security-threat-detection/crowdstrike.png')} alt="icon" width="90"/><h4>CrowdStrike Falcon FileVantage</h4></a>
Expand Down
1 change: 1 addition & 0 deletions sidebars.ts
Original file line number Diff line number Diff line change
Expand Up @@ -2544,6 +2544,7 @@ integrations: [
'integrations/saas-cloud/citrix-cloud',
'integrations/saas-cloud/cloudflare',
'integrations/saas-cloud/code42-incydr',
'integrations/saas-cloud/confluent-cloud',
'integrations/saas-cloud/crowdstrike-falcon-filevantage',
'integrations/saas-cloud/crowdstrike-fdr-host-inventory',
'integrations/saas-cloud/crowdstrike-spotlight',
Expand Down
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.