Skip to content

Updated doc for AWS tag filters and AWSO apps #4827

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 11 commits into from
Jan 24, 2025
Original file line number Diff line number Diff line change
Expand Up @@ -85,10 +85,11 @@ You should only install the AWS Observability apps and alerts the first time you
The below tables displays the response for each text box in this section.

| Prompt | Guideline |
|:--|:--|
| :-- | :-- |
| Select the kind of CloudWatch Metrics Source to create | **Note:** Switching from one type of Metrics Source to another can result in re-computation of your Root Cause Explorer anomaly detection models. This re-computation can take a couple of days to finish and meanwhile you will not get new Events of Interest (EOIs).<ul><li>**CloudWatch Metrics Source** - Creates Sumo Logic AWS CloudWatch Metrics Sources.</li><li>**Kinesis Firehose Metrics Source (Recommended)** - Creates a Sumo Logic AWS Kinesis Firehose for Metrics Source.<br/>**Note:** This new source has cost and performance benefits over the CloudWatch Metrics Source is therefore recommended.</li><li>**None** - Skips the Installation of both the Sumo Logic Sources</li></ul> |
| Sumo Logic AWS Metrics Namespaces | Enter a comma-delimited list of the namespaces which will be used for both AWS CloudWatch Metrics and Inventory Sources.<br/>The default will be AWS/ApplicationELB, AWS/ApiGateway, AWS/DynamoDB, AWS/Lambda, AWS/RDS, AWS/ECS, AWS/ElastiCache, AWS/ELB, AWS/NetworkELB, AWS/SQS, AWS/SNS, and AWS/EC2.<br/> AWS/AutoScaling will be appended to Namespaces for Inventory Sources.<br/>Supported namespaces are based on the type of CloudWatch Metrics Source you have selected above. See the relevant docs for the [Kinesis Firehose Metrics Source](/docs/send-data/hosted-collectors/amazon-aws/aws-kinesis-firehose-metrics-source.md) and the [CloudWatch Metrics Source](/docs/send-data/hosted-collectors/amazon-aws/amazon-cloudwatch-source-metrics.md) for details on which namespaces they support. |
| Existing Sumo Logic Metrics Source API URL | You must supply this URL if you are already collecting CloudWatch Metrics. Provide the existing Sumo Logic Metrics Source API URL. The account field will be added to the Source. For information on how to determine the URL, see [View or Download Source JSON Configuration](/docs/send-data/use-json-configure-sources/local-configuration-file-management/view-download-source-json-configuration.md). |
| Sumo Logic AWS Metrics Namespaces | Enter a comma-delimited list of the namespaces which will be used for both AWS CloudWatch Metrics and Inventory Sources.<br/>The default will be AWS/ApplicationELB, AWS/ApiGateway, AWS/DynamoDB, AWS/Lambda, AWS/RDS, AWS/ECS, AWS/ElastiCache, AWS/ELB, AWS/NetworkELB, AWS/SQS, AWS/SNS, and AWS/EC2. You can provide both AWS as well as custom namespaces. <br/> AWS/AutoScaling will be appended to Namespaces for Inventory Sources.<br/>Supported namespaces are based on the type of CloudWatch Metrics Source you have selected above. See the relevant docs for the [Kinesis Firehose Metrics Source](/docs/send-data/hosted-collectors/amazon-aws/aws-kinesis-firehose-metrics-source.md) and the [CloudWatch Metrics Source](/docs/send-data/hosted-collectors/amazon-aws/amazon-cloudwatch-source-metrics.md) for details on which namespaces they support. |
| Existing Sumo Logic Metrics Source API URL | You must supply this URL if you are already collecting CloudWatch Metrics. Provide the existing Sumo Logic Metrics Source API URL. The account field will be added to the Source. For information on how to determine the URL, see [View or Download Source JSON Configuration](/docs/send-data/use-json-configure-sources/local-configuration-file-management/view-download-source-json-configuration.md).|
| Sumo Logic AWS Metrics Tag Filters | Provide JSON format of the namespaces with its tags values to add filters to your metrics. Use semicolons to separate multiple values for the same tag key. AWS Tag Filters will be added to the Source. See JSON format example: ```json {"AWS/ELB":{"tags":["env=prod;dev"]},"AWS/EC2":{"tags":["env=dev","creator=john"]},"AWS/RDS":{"tags":["env=prod;dev","creator=himan"]},"All":{"tags":["env=dev"]}}``` .<br/> Filters are not supported for custom metrics. |

## Step 6: Sumo Logic AWS ALB Log Source

Expand Down Expand Up @@ -125,6 +126,7 @@ The below tables displays the response for each text box in this section.
| Existing Sumo Logic Lambda CloudWatch Logs Source API URL | Required you already collect AWS Lambda CloudWatch logs. Provide the existing Sumo Logic AWS Lambda CloudWatch Source API URL. The account, region and namespace fields will be added to the Source. For information on how to determine the URL, see [View or Download Source JSON Configuration](/docs/send-data/use-json-configure-sources/local-configuration-file-management/view-download-source-json-configuration.md). |
| Subscribe log groups to destination (lambda or kinesis firehose delivery stream) | <ul><li>**New** - Automatically subscribes new AWS Lambda log groups to Lambda, to send logs to Sumo Logic.</li><li>**Existing** - Automatically subscribes existing log groups to Lambda, to send logs to Sumo Logic.</li><li>**Both** - Automatically subscribes new and existing log groups.</li><li>**None** - Skips automatic subscription of log groups.</li></ul>|
| Regex for AWS Log Groups | Default Value: **aws/(lambda\|apigateway\|rds)** <br/> With default value, log group names matching with lambda or rds will be subscribed and ingesting cloudwatch logs into sumo logic.<br/> Enter a regex for matching log group names. For more information, see [Configuring parameters](/docs/send-data/collect-from-other-data-sources/autosubscribe-arn-destination/#configuringparameters) in the *Auto-Subscribe ARN (Amazon Resource Name) Destination* topic.
| Tags for filtering CloudWatch Log Groups | Enter comma separated key value pairs for filtering logGroups using tags. Ex KeyName1=string,KeyName2=string. This is optional leave it blank if tag based filtering is not needed. Visit https://help.sumologic.com/docs/send-data/collect-from-other-data-sources/autosubscribe-arn-destination/#configuringparameters |

:::note
* Don't use forward slashes (`/`) to encapsulate the regex. While normally they are needed for raw code, it's not necessary here.
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -60,7 +60,7 @@ System Files:

Before you run the Terraform script, perform the following actions on a server machine of your choice:

1. Install [Terraform](https://www.terraform.io/) version [0.13.0](https://releases.hashicorp.com/terraform/) or later. To check the installed Terraform version, run the following command:
1. Install [Terraform](https://www.terraform.io/) version [1.6.0](https://releases.hashicorp.com/terraform/) or later. To check the installed Terraform version, run the following command:
```bash
$ terraform --version
```
Expand Down Expand Up @@ -639,7 +639,7 @@ The following table provides a list of all source parameters and their default v
### Configure collection of CloudWatch metrics

:::note
To migrate CloudWatch Metrics Source to Kinesis Firehose Metrics Source using Terraform, refer to [Migration Strategy using Terraform](/docs/observability/aws/deploy-use-aws-observability/migration-strategy-using-terraform).
To migrate from legacy CloudWatch Metrics Source to Kinesis Firehose Metrics Source using Terraform, refer to [Migration Strategy using Terraform](/docs/observability/aws/deploy-use-aws-observability/migration-strategy-using-terraform).
:::

#### collect_cloudwatch_metrics
Expand Down Expand Up @@ -668,7 +668,7 @@ collect_cloudwatch_metrics = "Kinesis Firehose Metrics Source"

Provide details for the Sumo Logic CloudWatch Metrics source. If not provided, then defaults will be used.

* `limit_to_namespaces`. Enter a comma-delimited list of the namespaces which will be used for both AWS CloudWatch Metrics Source.
* `limit_to_namespaces`. Enter a comma-delimited list of the namespaces which will be used for both AWS CloudWatch Metrics Source. You can provide both AWS and custom namespaces.

Supported namespaces are based on the type of CloudWatch Metrics Source you have selected above. See the relevant docs for the [Kinesis Firehose Metrics Source](/docs/send-data/hosted-collectors/amazon-aws/aws-kinesis-firehose-metrics-source) and the [CloudWatch Metrics Source](/docs/send-data/hosted-collectors/amazon-aws/amazon-cloudwatch-source-metrics) for details on which namespaces they support.

Expand All @@ -695,7 +695,8 @@ Supported namespaces are based on the type of CloudWatch Metrics Source you have
"AWS/NetworkELB",
"AWS/SQS",
"AWS/SNS"
],
],
"tag_filters": [],
"source_category": "aws/observability/cloudwatch/metrics",
"source_name": "CloudWatch Metrics (Region)"
}
Expand All @@ -705,8 +706,8 @@ Supported namespaces are based on the type of CloudWatch Metrics Source you have

The following override example collects only DynamoDB and Lambda namespaces with source_category set to `"aws/observability/cloudwatch/metrics/us-east-1"`:

```json
Cloudwatch_metrics_source_details = {
```json title="cloudwatch_metrics_source_details"
{
"bucket_details": {
"bucket_name": "",
"create_bucket": true,
Expand All @@ -716,13 +717,27 @@ Cloudwatch_metrics_source_details = {
"fields": {},
"limit_to_namespaces": [
"AWS/DynamoDB",
"AWS/Lambda"
],
"AWS/Lambda",
"CWAgent"
],
"tag_filters": [{
"type":"TagFilters",
"namespace" : "AWS/DynamoDB",
"tags": ["env=prod;dev"]
},{
"type": "TagFilters",
"namespace": "AWS/Lambda",
"tags": ["env=prod"]
}],
"source_category": "aws/observability/cloudwatch/metrics/us-east-1",
"source_name": "CloudWatch Metrics us-east-1"
}
```

:::note
All namespaces specified in `tag_filters` must be included in `limit_to_namespaces`. Filters are not supported for custom metrics.
:::

#### cloudwatch_metrics_source_url

Use this parameter if you are already collecting CloudWatch Metrics and want to use an existing Sumo Logic Collector Source. You need to provide the URL of the existing Sumo Logic CloudWatch Metrics Source. If the URL is for a AWS CloudWatch Metrics source, the "account" and "accountid" metadata fields will be added to the Source. If the URL is for the Kinesis Firehose for Metrics source, the "account" field will be added to the Source. For information on how to determine the URL, see [View or Download Source JSON Configuration](/docs/send-data/use-json-configure-sources/local-configuration-file-management/view-download-source-json-configuration).
Expand Down Expand Up @@ -1235,23 +1250,26 @@ auto_enable_logs_subscription="New"

### auto_enable_logs_subscription_options

`filter`. Enter regex for matching logGroups for AWS Lambda only. The regex will check the name. See [Configuring Parameters](/docs/send-data/collect-from-other-data-sources/autosubscribe-arn-destination).
* `filter`. Enter regex for matching logGroups for AWS Lambda only. The regex will check the name. See [Configuring Parameters](/docs/send-data/collect-from-other-data-sources/autosubscribe-arn-destination/#configuringparameters).
* `tags_filter`. Enter comma separated key value pairs for filtering logGroups using tags. Ex KeyName1=string,KeyName2=string. This is optional leave it blank if tag based filtering is not needed. See [Configuring Parameters](/docs/send-data/collect-from-other-data-sources/autosubscribe-arn-destination/#configuringparameters)

**Default value:**

```json
{
"filter": "apigateway|lambda|rds"
"filter": "apigateway|lambda|rds",
"tags_filter": ""
}
```

**Default JSON:**
**Override Example JSON:**

The following example includes all log groups that match `"lambda-cloudwatch-logs"`:

```
auto_enable_logs_subscription_options = {
"filter": "lambda-cloudwatch-logs"
"tags_filter": "Environment=Production,Application=MyApp"
}
```

Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -58,7 +58,7 @@ This section describes the parameters you can configure for the Lambda function.

* **UseExistingLogs**—Controls whether this function will be used to create subscription filters for existing log groups. Select "True" if you want to use the function for subscribing to existing log groups.

* **LogGroupTags**: Enter comma-separated key-value pairs for filtering logGroups using tags. For Example, KeyName1=string,KeyName2=string. Only log groups that match any one of the key-value pairs will be subscribed by this Lambda function. Supported only when UseExistingLogs is set to false which means it works only for new log groups, not existing log groups.
* **LogGroupTags**: Enter comma-separated key-value pairs for filtering logGroups using tags. For Example, KeyName1=string,KeyName2=string. Supported only when UseExistingLogs is set to false.

* **RoleArn:** Provide the AWS Role ARN which has permission to put data into the provided Kinesis Firehose data delivery stream. Keep the value empty, when the destination type is Lambda.

Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -73,7 +73,8 @@ AWS tag filtering is supported for the following AWS namespaces.
* AWS/ES
* AWS/Firehose
* AWS/Inspector
* AWS/Kinesis AWS/KinesisAnalytics
* AWS/Kinesis
* AWS/KinesisAnalytics
* AWS/KinesisVideo
* AWS/KMS
* AWS/Lambda
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -53,6 +53,8 @@ In this step, you create the AWS Kinesis Firehose for Metrics source.
1. Select **AWS Kinesis Firehose** for Metrics.
1. Enter a **Name** for the source.
1. (Optional) Enter a **Description**.
1. (Optional) Enter a **AWS Tag filters**.
**Example** <br/><img src={useBaseUrl('img/send-data/kinesis-aws-tag-filters.png')} alt="kinesis-aws-source.png" style={{border: '1px solid gray'}} width="500"/>
1. For **Source Category**, enter any string to tag the output collected from this Source. Category metadata is stored in a searchable field called `_sourceCategory`.<br/><img src={useBaseUrl('img/send-data/kinesis-aws-source.png')} alt="kinesis-aws-source.png" style={{border: '1px solid gray'}} width="500"/>
1. For **AWS Access** of a Kinesis Metric source, the role requires `tag:GetResources` permission. The Kinesis Log source does not require permissions.
1. Click **Save**.
Expand Down
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading