From 93e060d47a7f75f39724fe76c498ca6720a1060d Mon Sep 17 00:00:00 2001 From: sumoanema Date: Thu, 21 Nov 2024 13:42:18 +0530 Subject: [PATCH 1/4] Doc changes to collect unified audit logs for windows env for oracle otel app --- .../opentelemetry/oracle-opentelemetry.md | 32 +++++++++---------- 1 file changed, 16 insertions(+), 16 deletions(-) diff --git a/docs/integrations/databases/opentelemetry/oracle-opentelemetry.md b/docs/integrations/databases/opentelemetry/oracle-opentelemetry.md index 3c12f39f43..b85c3414a2 100644 --- a/docs/integrations/databases/opentelemetry/oracle-opentelemetry.md +++ b/docs/integrations/databases/opentelemetry/oracle-opentelemetry.md @@ -11,13 +11,15 @@ import TabItem from '@theme/TabItem'; Thumbnail icon Thumbnail icon -The [Oracle](https://docs.oracle.com/database/121/CNCPT/intro.htm#CNCPT001) app is a logs and metrics based app. Preconfigured dashboards and searches provide insight into the listeners, sys/xml audit logs, alerts, performance, and security. It also gives insight around count of rollback, commits, transaction, process, session, hard parse, and DML locks. +The [Oracle](https://docs.oracle.com/database/121/CNCPT/intro.htm#CNCPT001) app is a logs and metrics based app. Preconfigured dashboards and searches provide insight into the listeners, audit logs (traditional - sys/xml and unified), performance, and security. It also gives insight around count of rollback, commits, transaction, process, session, hard parse, and DML locks. This app is tested with the following Oracle versions: - Non-Kubernetes: Oracle Database 23 Release 23.4.0.24.05. -Oracle logs are sent to Sumo Logic through OpenTelemetry [filelog receiver](https://github.com/open-telemetry/opentelemetry-collector-contrib/tree/main/receiver/filelogreceiver) and metrics are collected through [Oracledb receiver](https://github.com/open-telemetry/opentelemetry-collector-contrib/tree/main/receiver/oracledbreceiver) +Oracle logs like listner, alert and traditional audit logs (oracle version <= 19c) are sent to Sumo Logic through OpenTelemetry [filelog receiver](https://github.com/open-telemetry/opentelemetry-collector-contrib/tree/main/receiver/filelogreceiver) whereas unified audit logs (oracle version >= 21c) are collected through [syslog](https://github.com/open-telemetry/opentelemetry-collector-contrib/tree/main/receiver/syslogreceiver) for linux environment and [windowseventlog](https://github.com/open-telemetry/opentelemetry-collector-contrib/blob/main/receiver/windowseventlogreceiver/) receiver for windows environment. + +Metrics are collected through [Oracledb receiver](https://github.com/open-telemetry/opentelemetry-collector-contrib/tree/main/receiver/oracledbreceiver) Schematics @@ -64,15 +66,11 @@ If logging is not enabled, you can configure it by following the steps below. lsnrctl command [listener_name] lsnrctl set log_status on ``` -- **Audit Log**. Traditional auditing is deprecated in Oracle Database 21c. If you are using version below 21c follow [this](https://docs.oracle.com/cd/E11882_01/server.112/e10575/tdpsg_auditing.htm#TDPSG50000) guide to enable Audit Logs. For version 21c and above, follow [this](https://docs.oracle.com/en/database/oracle/oracle-database/19/dbseg/administering-the-audit-trail.html#GUID-662AA54B-D878-4B78-94D3-733256B3F37C) to enable exporting for unified audit trail through syslog for Oracle on Linux. - - :::note - Currently this app does not support collection of unified audit trail logs collected in Oracle on Windows as Windows event logs. - ::: +- **Audit Log**. Traditional auditing is deprecated in Oracle Database 21c. If you are using version 19c or below follow [this](https://docs.oracle.com/cd/E11882_01/server.112/e10575/tdpsg_auditing.htm#TDPSG50000) guide to enable Audit Logs. For version 21c and above, follow [this](https://docs.oracle.com/en/database/oracle/oracle-database/19/dbseg/administering-the-audit-trail.html#GUID-662AA54B-D878-4B78-94D3-733256B3F37C) to enable exporting for unified audit logs through syslog for Oracle on Linux and windows event logs for Oracle on Windows. #### Verify local logs file directories and path -- **Oracle Alert Logs**. For 11g and later releases (12c, 18c, 19c). By default, Oracle logs are stored in +- **Oracle Alert Logs**. For 11g and later releases (12c, 18c, 19c, 21c, 23c). By default, Oracle logs are stored in `$ORACLE_BASE/diag/rdbms/$DB_UNIQUE_NAME/$ORACLE_SID/trace/`. The default directory for log files is stored in `BACKGROUND_DUMP_DEST` parameter. You can query the value of `BACKGROUND_DUMP_DEST`, an initialization parameter, where you can find Oracle alert log by executing the command below: ```sh SQL > show parameter background_dump_dest; @@ -82,7 +80,7 @@ If logging is not enabled, you can configure it by following the steps below. [oracle@sumolab alert]$ lsnrctl status ``` - **Oracle Audit Logs**. - - **For Oracle version below 21c**. By default, Oracle logs are stored in + - **For Oracle version 19c and below**. By default, Oracle logs are stored in ``` $ORACLE_BASE/app/oracle/admin/orcl/adump ``` @@ -92,7 +90,7 @@ If logging is not enabled, you can configure it by following the steps below. The location of these logs will be required when you set up the app through the app catalog. - - **For Oracle version 21c and above**. Once unified audit trail is redirected to syslog, it will start getting written to destination set in `syslog.conf`. + - **For Oracle version 21c and above**. Based on [Unified audit policy](https://docs.oracle.com/en/database/oracle/oracle-database/23/sqlrf/AUDIT-Unified-Auditing.html) configuered, audit logs exported will direclty be ingested to sumo using syslog or windows event log. For linux environment instead of redirecting audit logs to a file we need to directly redirect it to local port. In the next step, otel collector can be configured to listen to this port to send log to sumo. #### Performance metrics script setup @@ -181,11 +179,13 @@ import SetupColl from '../../../reuse/apps/opentelemetry/set-up-collector.md'; In this step, you will configure the yaml required for Oracle Collection. Below are the inputs required: -- **`Endpoint (no default)`**. Endpoint used to connect to the OracleDB server. Must be in the format of `host:port`. - **`Alert Logs`**. Path of the log file configured to capture oracle alert logs. - **`Listener Logs`**. Path of the log file configured to capture oracle listener logs . -- **`Audit Logs`**. Path of the log file configured to capture oracle audit logs. -- **`Performance metric script-based logs`**. Path of the log file configured to capture log generated through script. +- **`Audit Logs`**. + - For **Traditional Audit Logs** - Path of the log file configured to capture oracle audit logs. + - For **Unified Audit Logs** - You need to select the OS where the oracle is setup - windows or linux. For Oracle on windows machine no parameter is required. Only respective event id will be ingested. For linux environment you need to give the **port** (configured in prerequisite steps for linux environment) where the syslog receiver should listen to, for sending the logs to sumo. +- **`Performance metric script-based logs`**. Path of the log file configured to capture log generated through script can be configured in other logs. +- **`Endpoint (no default)`**. Endpoint used to connect to the OracleDB server. Must be in the format of `host:port`. - **`username`**. Username for the OracleDB connection. - **`password`**. Password for the OracleDB connection. Special characters are allowed. - **`service`**. OracleDB Service that the receiver should connect to. @@ -321,8 +321,8 @@ sumo.datasource=oracle metric=oracledb.sessions.usage deployment.environment=* ## Viewing Oracle dashboards :::note -- **Oracle - Sys Audit Log**, **Oracle - Sys Audit Log - Logon Analysis**, **Oracle - XML Audit Log - Logon Analysis**, and **Oracle - XML Audit Log - SQL Statement Analysis** dashboards will be populated only when collecting logs for Oracle database version below 21c. -- **Oracle - Unified Audit Syslogs** dashboard will populate for audit log collected for Oracle database version 21c and above. +- **Oracle - Sys Audit Log**, **Oracle - Sys Audit Log - Logon Analysis**, **Oracle - XML Audit Log - Logon Analysis**, and **Oracle - XML Audit Log - SQL Statement Analysis** dashboards will be populated only when collecting logs for Oracle database version 19c and below. +- **Oracle - Unified Audit Syslogs** dashboard will populate for unified audit log collected for Oracle database version 21c and above. ::: ### Overview @@ -549,7 +549,7 @@ Recent Jobs in the database. A table of information about recent database jobs, ### Unified Audit Syslog Monitor Performance by DB Script -See information derived from the syslog audit trail, including successful and failed activities, successful and failed logon attempts. In addition to this dashboard gives insight around logon status trend, top current and database users. +See information derived from the syslog audit trail, including successful and failed activities, successful and failed logon attempts. In addition to this dashboard gives insight around logon status trend, top current and database users. This dashboard work with unified audit logs exported from both windows and linux environment. ### Performance Details From f11e77dff8b9d0b61edc8e9c7663e73f3726cd2e Mon Sep 17 00:00:00 2001 From: sumoanema Date: Thu, 21 Nov 2024 14:02:23 +0530 Subject: [PATCH 2/4] Adding example to configure syslog server to redirect logs to a port --- .../databases/opentelemetry/oracle-opentelemetry.md | 5 ++++- 1 file changed, 4 insertions(+), 1 deletion(-) diff --git a/docs/integrations/databases/opentelemetry/oracle-opentelemetry.md b/docs/integrations/databases/opentelemetry/oracle-opentelemetry.md index b85c3414a2..7201f801c9 100644 --- a/docs/integrations/databases/opentelemetry/oracle-opentelemetry.md +++ b/docs/integrations/databases/opentelemetry/oracle-opentelemetry.md @@ -90,7 +90,10 @@ If logging is not enabled, you can configure it by following the steps below. The location of these logs will be required when you set up the app through the app catalog. - - **For Oracle version 21c and above**. Based on [Unified audit policy](https://docs.oracle.com/en/database/oracle/oracle-database/23/sqlrf/AUDIT-Unified-Auditing.html) configuered, audit logs exported will direclty be ingested to sumo using syslog or windows event log. For linux environment instead of redirecting audit logs to a file we need to directly redirect it to local port. In the next step, otel collector can be configured to listen to this port to send log to sumo. + - **For Oracle version 21c and above**. Based on [Unified audit policy](https://docs.oracle.com/en/database/oracle/oracle-database/23/sqlrf/AUDIT-Unified-Auditing.html) configuered, audit logs exported will direclty be ingested to sumo using syslog or windows event log. + For linux environment instead of redirecting audit logs to a file we need to directly redirect it to local port. In the next step, otel collector can be configured to listen to this port to send log to sumo. This can be done using a below configuration in rsyslog.conf : + ```local7.info @@127.0.0.1:10514``` + This will redirect all the unified audit logs to localhost port 10514. #### Performance metrics script setup From 109b3135f36009972e174ea89dc981c119c74118 Mon Sep 17 00:00:00 2001 From: Alekh Nema <91047769+sumoanema@users.noreply.github.com> Date: Thu, 21 Nov 2024 15:51:21 +0530 Subject: [PATCH 3/4] Apply suggestions from code review Co-authored-by: Jagadisha V <129049263+JV0812@users.noreply.github.com> --- .../opentelemetry/oracle-opentelemetry.md | 14 +++++++------- 1 file changed, 7 insertions(+), 7 deletions(-) diff --git a/docs/integrations/databases/opentelemetry/oracle-opentelemetry.md b/docs/integrations/databases/opentelemetry/oracle-opentelemetry.md index 7201f801c9..7c28b9322e 100644 --- a/docs/integrations/databases/opentelemetry/oracle-opentelemetry.md +++ b/docs/integrations/databases/opentelemetry/oracle-opentelemetry.md @@ -17,9 +17,9 @@ This app is tested with the following Oracle versions: - Non-Kubernetes: Oracle Database 23 Release 23.4.0.24.05. -Oracle logs like listner, alert and traditional audit logs (oracle version <= 19c) are sent to Sumo Logic through OpenTelemetry [filelog receiver](https://github.com/open-telemetry/opentelemetry-collector-contrib/tree/main/receiver/filelogreceiver) whereas unified audit logs (oracle version >= 21c) are collected through [syslog](https://github.com/open-telemetry/opentelemetry-collector-contrib/tree/main/receiver/syslogreceiver) for linux environment and [windowseventlog](https://github.com/open-telemetry/opentelemetry-collector-contrib/blob/main/receiver/windowseventlogreceiver/) receiver for windows environment. +Oracle logs, such as listener, alert, and traditional audit logs (oracle version <= 19c) are sent to Sumo Logic through OpenTelemetry [filelog receiver](https://github.com/open-telemetry/opentelemetry-collector-contrib/tree/main/receiver/filelogreceiver) whereas, unified audit logs (oracle version >= 21c) are collected through [syslog](https://github.com/open-telemetry/opentelemetry-collector-contrib/tree/main/receiver/syslogreceiver) for the Linux environment and [windowseventlog](https://github.com/open-telemetry/opentelemetry-collector-contrib/blob/main/receiver/windowseventlogreceiver/) receiver for windows environment. -Metrics are collected through [Oracledb receiver](https://github.com/open-telemetry/opentelemetry-collector-contrib/tree/main/receiver/oracledbreceiver) +Metrics are collected through [Oracledb receiver](https://github.com/open-telemetry/opentelemetry-collector-contrib/tree/main/receiver/oracledbreceiver). Schematics @@ -90,8 +90,8 @@ If logging is not enabled, you can configure it by following the steps below. The location of these logs will be required when you set up the app through the app catalog. - - **For Oracle version 21c and above**. Based on [Unified audit policy](https://docs.oracle.com/en/database/oracle/oracle-database/23/sqlrf/AUDIT-Unified-Auditing.html) configuered, audit logs exported will direclty be ingested to sumo using syslog or windows event log. - For linux environment instead of redirecting audit logs to a file we need to directly redirect it to local port. In the next step, otel collector can be configured to listen to this port to send log to sumo. This can be done using a below configuration in rsyslog.conf : + - **For Oracle version 21c and above**. Based on the [Unified audit policy](https://docs.oracle.com/en/database/oracle/oracle-database/23/sqlrf/AUDIT-Unified-Auditing.html) configured, audit logs exported will be directly ingested to Sumo Logic using syslog or windows event log. +For the Linux environment, instead of redirecting audit logs to a file, we need to redirect them to the local port. In the next step, the OTel collector can be configured to listen to this port and then send the log to Sumo Logic. This can be done using the below configuration in the `rsyslog.conf`. : ```local7.info @@127.0.0.1:10514``` This will redirect all the unified audit logs to localhost port 10514. @@ -185,8 +185,8 @@ Below are the inputs required: - **`Alert Logs`**. Path of the log file configured to capture oracle alert logs. - **`Listener Logs`**. Path of the log file configured to capture oracle listener logs . - **`Audit Logs`**. - - For **Traditional Audit Logs** - Path of the log file configured to capture oracle audit logs. - - For **Unified Audit Logs** - You need to select the OS where the oracle is setup - windows or linux. For Oracle on windows machine no parameter is required. Only respective event id will be ingested. For linux environment you need to give the **port** (configured in prerequisite steps for linux environment) where the syslog receiver should listen to, for sending the logs to sumo. + - For **Traditional Audit Logs**, path of the log file is configured to capture oracle audit logs. + - For **Unified Audit Logs**, you need to select the OS where the Oracle is setup - Windows or Linux. For Oracle on Windows machine, no parameter is required. Only respective event IDs will be ingested. For the Linux environment, you need to provide the **port** (configured in pre-requisite steps for the Linux environment) which the syslog receiver should listen to for sending the logs to Sumo Logic. - **`Performance metric script-based logs`**. Path of the log file configured to capture log generated through script can be configured in other logs. - **`Endpoint (no default)`**. Endpoint used to connect to the OracleDB server. Must be in the format of `host:port`. - **`username`**. Username for the OracleDB connection. @@ -552,7 +552,7 @@ Recent Jobs in the database. A table of information about recent database jobs, ### Unified Audit Syslog Monitor Performance by DB Script -See information derived from the syslog audit trail, including successful and failed activities, successful and failed logon attempts. In addition to this dashboard gives insight around logon status trend, top current and database users. This dashboard work with unified audit logs exported from both windows and linux environment. +See information derived from the syslog audit trail, including successful and failed activities, successful and failed logon attempts. Additionally, this dashboard also provides insight around logon status trend, top current, and database users. This dashboard work with unified audit logs exported from both Windows and Linux environment. ### Performance Details From 785bcd74038f468ac1423aec86eb8dbe539574d8 Mon Sep 17 00:00:00 2001 From: Alekh Nema <91047769+sumoanema@users.noreply.github.com> Date: Thu, 21 Nov 2024 17:10:26 +0530 Subject: [PATCH 4/4] Changes to make automation pass, which was failing due to less that/ greater than condition symbols --- .../databases/opentelemetry/oracle-opentelemetry.md | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/docs/integrations/databases/opentelemetry/oracle-opentelemetry.md b/docs/integrations/databases/opentelemetry/oracle-opentelemetry.md index 7c28b9322e..f871182d15 100644 --- a/docs/integrations/databases/opentelemetry/oracle-opentelemetry.md +++ b/docs/integrations/databases/opentelemetry/oracle-opentelemetry.md @@ -17,7 +17,7 @@ This app is tested with the following Oracle versions: - Non-Kubernetes: Oracle Database 23 Release 23.4.0.24.05. -Oracle logs, such as listener, alert, and traditional audit logs (oracle version <= 19c) are sent to Sumo Logic through OpenTelemetry [filelog receiver](https://github.com/open-telemetry/opentelemetry-collector-contrib/tree/main/receiver/filelogreceiver) whereas, unified audit logs (oracle version >= 21c) are collected through [syslog](https://github.com/open-telemetry/opentelemetry-collector-contrib/tree/main/receiver/syslogreceiver) for the Linux environment and [windowseventlog](https://github.com/open-telemetry/opentelemetry-collector-contrib/blob/main/receiver/windowseventlogreceiver/) receiver for windows environment. +Oracle logs, such as listener, alert, and traditional audit logs (Oracle version 19c and below) are sent to Sumo Logic through OpenTelemetry [filelog receiver](https://github.com/open-telemetry/opentelemetry-collector-contrib/tree/main/receiver/filelogreceiver) whereas, unified audit logs (Oracle version 21c and above) are collected through [syslog](https://github.com/open-telemetry/opentelemetry-collector-contrib/tree/main/receiver/syslogreceiver) for the Linux environment and [windowseventlog](https://github.com/open-telemetry/opentelemetry-collector-contrib/blob/main/receiver/windowseventlogreceiver/) receiver for windows environment. Metrics are collected through [Oracledb receiver](https://github.com/open-telemetry/opentelemetry-collector-contrib/tree/main/receiver/oracledbreceiver). @@ -558,4 +558,4 @@ See information derived from the syslog audit trail, including successful and fa Monitor Performance by DB Script The Oracle - Performance Details dashboard gives insight about - count of rollback, commits, transaction, process, session. -In addition to this it helps monitoring physical and logical reads, PGA allocated. This dashboard is based on the [metrics collected by Oracle DB opentelemetry receiver](https://github.com/open-telemetry/opentelemetry-collector-contrib/blob/main/receiver/oracledbreceiver/documentation.md). \ No newline at end of file +In addition to this it helps monitoring physical and logical reads, PGA allocated. This dashboard is based on the [metrics collected by Oracle DB opentelemetry receiver](https://github.com/open-telemetry/opentelemetry-collector-contrib/blob/main/receiver/oracledbreceiver/documentation.md).