Could not connect to RAC DB outside the Host through SQL Developer with CMAN

[srivallisudha]

Followed the documentation https://github.com/oracle/docker-images/blob/main/OracleDatabase/RAC/OracleRealApplicationClusters/docs/developers/README.md
to setup the rac db with 19c.

[root@rac-samples containerfiles]# podman ps -a
CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES
0a107e854d3b localhost/oracle/rac-dnsserver:latest /bin/sh -c exec $... 4 hours ago Up 4 hours (healthy) rac-dnsserver
3cf5709f6793 localhost/oracle/rac-storage-server:latest 4 hours ago Up 4 hours (healthy) racnode-storage
2f72e18fd687 localhost/oracle/database-rac:19c 4 hours ago Up 4 hours (healthy) racnodep1
85e280b1e5e3 localhost/oracle/database-rac:19c 4 hours ago Up 4 hours (healthy) racnodep2
8575c0856506 localhost/oracle/client-cman:19.3.0 /bin/sh -c exec $... 2 hours ago Up 2 hours (healthy) 0.0.0.0:1521->1521/tcp racnodepc1-cman

able to connect inside the cman pod
[oracle@racnodepc1-cman ~]$ sqlplus sys/***@//racnodepc1-scan.example.info:1521/ORCLCDB as sysdba

SQL*Plus: Release 19.0.0.0.0 - Production on Wed Mar 26 08:17:35 2025
Version 19.3.0.0.0

Copyright (c) 1982, 2019, Oracle. All rights reserved.

Connected to:
Oracle Database 19c Enterprise Edition Release 19.0.0.0.0 - Production
Version 19.25.0.0.0

SQL> exit

From sql developer tried to connect with HOST - ip address of the machine
Are there any configuration changes required to access from outside

one observation is inside the cman pod status is below.

[oracle@racnodepc1-cman /]$ lsnrctl status

LSNRCTL for Linux: Version 19.0.0.0.0 - Production on 26-MAR-2025 09:04:34

Copyright (c) 1991, 2019, Oracle. All rights reserved.

Connecting to (DESCRIPTION=(ADDRESS=(PROTOCOL=TCP)(HOST=57891aa261a6)(PORT=1521)))
TNS-12545: Connect failed because target host or object does not exist
TNS-12560: TNS:protocol adapter error
TNS-00515: Connect failed because target host or object does not exist
Linux Error: 2: No such file or directory
Connecting to (DESCRIPTION=(ADDRESS=(PROTOCOL=IPC)(KEY=EXTPROC1521)))
TNS-12541: TNS:no listener
TNS-12560: TNS:protocol adapter error
TNS-00511: No listener
Linux Error: 2: No such file or directory

Inside the racnodep1 updated the remote_listener but still not able to access.
[oracle@racnodep1 ~]$ sqlplus / as sysdba

SQL*Plus: Release 19.0.0.0.0 - Production on Wed Mar 26 08:30:34 2025
Version 19.25.0.0.0

Copyright (c) 1982, 2024, Oracle. All rights reserved.

Connected to:
Oracle Database 19c Enterprise Edition Release 19.0.0.0.0 - Production
Version 19.25.0.0.0

SQL> SHOW PARAMETER remote_listener;

NAME TYPE VALUE

---

remote_listener string racnodepc1-scan:1521,(DESCRIPT
ION=(ADDRESS_LIST=(ADDRESS=(PR
OTOCOL=TCP)(HOST=racnodepc1-cm
an)(PORT=1521))))
SQL> alter system set remote_listener='racnodepc1-scan:1521,racnodepc1-cman:1521' scope=both;

System altered.

SQL> alter system register;

System altered.

Please let me know if any changes required to access this rac db from outside

[psaini79]

Please share following details:

1. Are you able to ping racnodepc1-scan from cman machine?
2. Login to CMAN and using cmctl show services

[srivallisudha]

1. yes it is able to ping the racnodepc1-scan
   [oracle@racnodepc1-cman ~]$ ping racnodepc1-scan
   PING racnodepc1-scan.example.info (10.0.20.237) 56(84) bytes of data.
   64 bytes from racnodepc1-scan.example.info (10.0.20.237): icmp_seq=1 ttl=64 time=0.024 ms
   64 bytes from racnodepc1-scan.example.info (10.0.20.237): icmp_seq=2 ttl=64 time=0.040 ms
   64 bytes from racnodepc1-scan.example.info (10.0.20.237): icmp_seq=3 ttl=64 time=0.035 ms
   64 bytes from racnodepc1-scan.example.info (10.0.20.237): icmp_seq=4 ttl=64 time=0.059 ms

2.[oracle@racnodepc1-cman ~]$ cmctl show services

CMCTL for Linux: Version 19.0.0.0.0 - Production on 09-APR-2025 12:29:59

Copyright (c) 1996, 2019, Oracle. All rights reserved.

Current instance CMAN_racnodepc1-cman.example.info is already started
Connecting to (DESCRIPTION=(address=(protocol=tcp)(host=racnodepc1-cman.example.info)(port=1521)))
Services Summary...
Proxy service "cmgw" has 1 instance(s).
Instance "cman", status READY, has 2 handler(s) for this service...
Handler(s):
"cmgw001" established:0 refused:0 current:0 max:256 state:ready
<machine: localhost, pid: 125>
(ADDRESS=(PROTOCOL=ipc)(KEY=#125.1)(KEYPATH=/var/tmp/.oracle_5432100))
"cmgw000" established:0 refused:0 current:0 max:256 state:ready
<machine: localhost, pid: 123>
(ADDRESS=(PROTOCOL=ipc)(KEY=#123.1)(KEYPATH=/var/tmp/.oracle_5432100))
Service "cmon" has 1 instance(s).
Instance "cman", status READY, has 1 handler(s) for this service...
Handler(s):
"cmon" established:2 refused:0 current:1 max:4 state:ready
<machine: localhost, pid: 117>
(ADDRESS=(PROTOCOL=ipc)(KEY=#117.1)(KEYPATH=/var/tmp/.oracle_5432100))
The command completed successfully.

CMCTL:CMAN_racnodepc1-cman.example.info> show all
listener_address | (DESCRIPTION=(address=(protocol=tcp)(host=racnodepc1-cman.example.info)(port=1521)))
aso_authentication_filter | OFF
connection_statistics | ON
event_group | OFF
log_directory | /u01/app/oracle/diag/netcman/racnodepc1-cman/cman_racnodepc1-cman.example.info/alert
log_level | USER
max_connections | 256
idle_timeout | 0
inbound_connect_timeout | 60
session_timeout | 0
outbound_connect_timeout | 0
max_gateway_processes | 16
min_gateway_processes | 2
max_cmctl_sessions | 4
password | OFF
remote_admin | OFF
trace_directory | /u01/app/oracle/diag/netcman/racnodepc1-cman/cman_racnodepc1-cman.example.info/trace
trace_level | USER
trace_timestamp | ON
trace_filelen | 0
trace_fileno | 0
Number of filtering rules currently in effect: 3
(rule_list=
(rule=
(src=)
(dst=racnodepc1-scan)
(srv=)
(act=accept)
(action_list=(aut=off)(moct=0)(mct=0)(mit=0)(conn_stats=on))
)
(rule=
(src=)
(dst=racnodep1)
(srv=)
(act=accept)
(action_list=(aut=off)(moct=0)(mct=0)(mit=0)(conn_stats=on))
)
(rule=
(src=racnodepc1-cman.example.info)
(dst=127.0.0.1)
(srv=cmon)
(act=accept)
)
)
The command completed successfully.
CMCTL:CMAN_racnodepc1-cman.example.info>

[psaini79]

Let us try to ease the filtering rules and make it work. Once it is working, if you want, we can add filtering rules based on your environment.

Please execute following steps on corresponding containers and share the status:

Login to CMAN container and execute following:

1. Stop the CMAN cmctl shutdown
2. change dir to cd $ORACLE_HOME/network/admin
3. Copy original cman.ora to cman.orag cp cman.ora cman.org
4. Open cman.ora, remove the previous contents and replace with following:

CMAN_racnodepc1-cman.example.info =
(configuration=
  (address=(protocol=tcp)(host=racnodepc1-cman.example.info)(port=1521))
  (parameter_list =
    (connection_statistics=yes)
    (valid_node_checking_registration=on)
    (registration_invited_nodes=*)
  )
  (rule_list=
    (rule=
       (src=*)(dst=*)(srv=*)(act=accept)
       (action_list=(aut=off)(moct=0)(mct=0)(mit=0)(conn_stats=on))
    )
    (rule=(src=racnodepc1-cman.example.info)(dst=127.0.0.1)(srv=cmon)(act=accept))
  )
)

4. cmctl startup

5. Login to RAC Containers, change user to Oracle user and execute following:

sqlplus "/as sysdba"
alter system set remote_listener="racnodepc1-scan:1521,racnodepc1-cman:1521" sid='*';
show parameter remote_listener
alter system register
alter system register

6. Switch back to CMAN container and execute cmctl show services
7. You should see database services are registered with CMAN. Once the services are registered try to connect using sql developer or sqlplus.

Let me know if you are seeing any issue.

[srivallisudha]

[oracle@racnodepc1-cman ~]$ cmctl shutdown

CMCTL for Linux: Version 19.0.0.0.0 - Production on 16-APR-2025 09:00:55

Copyright (c) 1996, 2019, Oracle. All rights reserved.

Current instance CMAN_racnodepc1-cman.example.info is already started
Connecting to (DESCRIPTION=(address=(protocol=tcp)(host=racnodepc1-cman.example.info)(port=1521)))
The command completed successfully.

[oracle@racnodepc1-cman admin]$ cat cman.ora

CMAN_racnodepc1-cman.example.info =
(configuration=
(address=(protocol=tcp)(host=racnodepc1-cman.example.info)(port=1521))
(parameter_list =
(connection_statistics=yes)
(valid_node_checking_registration=on)
(registration_invited_nodes=)
)
(rule_list=
(rule=
(src=)(dst=)(srv=)(act=accept)
(action_list=(aut=off)(moct=0)(mct=0)(mit=0)(conn_stats=on))
)
(rule=(src=racnodepc1-cman.example.info)(dst=127.0.0.1)(srv=cmon)(act=accept))
)
)
[oracle@racnodepc1-cman admin]$ cmctl startup

CMCTL for Linux: Version 19.0.0.0.0 - Production on 16-APR-2025 09:08:34

Copyright (c) 1996, 2019, Oracle. All rights reserved.
Current instance CMAN_racnodepc1-cman.example.info is not yet started
Connecting to (DESCRIPTION=(address=(protocol=tcp)(host=racnodepc1-cman.example.info)(port=1521)))
Starting Oracle Connection Manager instance CMAN_racnodepc1-cman.example.info. Please wait...
CMAN for Linux: Version 19.0.0.0.0 - Production
Status of the Instance

Instance name cman_racnodepc1-cman.example.info
Version CMAN for Linux: Version 19.0.0.0.0 - Production
Start date 16-APR-2025 09:08:34
Uptime 0 days 0 hr. 0 min. 9 sec
Num of gateways started 2
Average Load level 0
Log Level SUPPORT
Trace Level OFF
Instance Config file /u01/app/oracle/product/19.3.0/client_1/network/admin/cman.ora
Instance Log directory /u01/app/oracle/diag/netcman/racnodepc1-cman/cman_racnodepc1-cman.example.info/alert
Instance Trace directory /u01/app/oracle/diag/netcman/racnodepc1-cman/cman_racnodepc1-cman.example.info/trace
The command completed successfully.
[oracle@racnodepc1-cman admin]$

[oracle@racnodep1 ~]$ sqlplus / as sysdba

SQL*Plus: Release 19.0.0.0.0 - Production on Wed Apr 16 12:13:55 2025
Version 19.25.0.0.0

Copyright (c) 1982, 2024, Oracle. All rights reserved.

Connected to:
Oracle Database 19c Enterprise Edition Release 19.0.0.0.0 - Production
Version 19.25.0.0.0

SQL> show parameter remote_listener;

NAME TYPE VALUE

---

remote_listener string racnodepc1-scan:1521,racnodepc
1-cman:1521
SQL> alter system set remote_listener="racnodepc1-scan:1521,racnodepc1-cman:1521" sid='*';

System altered.

SQL> show parameter remote_listener

NAME TYPE VALUE

---

remote_listener string racnodepc1-scan:1521,racnodepc
1-cman:1521
SQL> alter system register
2 ;

System altered.

SQL> alter system register;

System altered.

SQL> show parameter remote_listener;

NAME TYPE VALUE

---

remote_listener string racnodepc1-scan:1521,racnodepc
1-cman:1521
SQL>

[oracle@racnodepc1-cman ~]$ cmctl show services

CMCTL for Linux: Version 19.0.0.0.0 - Production on 16-APR-2025 12:21:24

Copyright (c) 1996, 2019, Oracle. All rights reserved.

Current instance CMAN_racnodepc1-cman.example.info is already started
Connecting to (DESCRIPTION=(address=(protocol=tcp)(host=racnodepc1-cman.example.info)(port=1521)))
Services Summary...
Proxy service "cmgw" has 1 instance(s).
Instance "cman", status READY, has 2 handler(s) for this service...
Handler(s):
"cmgw001" established:0 refused:0 current:0 max:256 state:ready
<machine: localhost, pid: 58909>
(ADDRESS=(PROTOCOL=ipc)(KEY=#58909.1)(KEYPATH=/var/tmp/.oracle_5432100))
"cmgw000" established:0 refused:0 current:0 max:256 state:ready
<machine: localhost, pid: 58907>
(ADDRESS=(PROTOCOL=ipc)(KEY=#58907.1)(KEYPATH=/var/tmp/.oracle_5432100))
Service "cmon" has 1 instance(s).
Instance "cman", status READY, has 1 handler(s) for this service...
Handler(s):
"cmon" established:2 refused:0 current:1 max:4 state:ready
<machine: localhost, pid: 58901>
(ADDRESS=(PROTOCOL=ipc)(KEY=#58901.1)(KEYPATH=/var/tmp/.oracle_5432100))
The command completed successfully.

After this as well i could not connect to the DB outside

[srivallisudha]

CMAN README is having wrong remote listener. The CMAN README mentioned the IP to be used is 10.0.20.15 but DNS container resolve the racnodepc1-cman to 10.0.20.166 and because of that ping was failing from RAC to CMAN.

SQL> show parameter remote_list

NAME TYPE VALUE

---

remote_listener string racnodepc1-scan:1521,racnodepc
1-cman:1521
SQL> alter system set remote_listener="racnodepc1-scan:1521,10.0.20.15:1521" sid='*';
...................................................
CMCTL:CMAN_racnodepc1-cman.example.info> show services
Services Summary...
Proxy service "cmgw" has 1 instance(s).
Instance "cman", status READY, has 2 handler(s) for this service...
Handler(s):
"cmgw001" established:0 refused:0 current:0 max:256 state:ready
<machine: localhost, pid: 58909>
(ADDRESS=(PROTOCOL=ipc)(KEY=#58909.1)(KEYPATH=/var/tmp/.oracle_5432100))
"cmgw000" established:2 refused:0 current:0 max:256 state:ready
<machine: localhost, pid: 58907>
(ADDRESS=(PROTOCOL=ipc)(KEY=#58907.1)(KEYPATH=/var/tmp/.oracle_5432100))
Service "3256dced0f17201ce063ab14000a6d3a" has 2 instance(s).
Instance "ORCLCDB1", status READY, has 1 handler(s) for this service...
Handler(s):
"DEDICATED" established:0 refused:0 state:ready
REMOTE SERVER
(ADDRESS=(PROTOCOL=TCP)(HOST=10.0.20.200)(PORT=1521))
Instance "ORCLCDB2", status READY, has 1 handler(s) for this service...
Handler(s):
"DEDICATED" established:2 refused:0 state:ready
REMOTE SERVER
(ADDRESS=(PROTOCOL=TCP)(HOST=10.0.20.201)(PORT=1521))
Service "ORCLCDB" has 2 instance(s).
Instance "ORCLCDB1", status READY, has 1 handler(s) for this service...
Handler(s):
"DEDICATED" established:0 refused:0 state:ready
REMOTE SERVER
(ADDRESS=(PROTOCOL=TCP)(HOST=10.0.20.200)(PORT=1521))
Instance "ORCLCDB2", status READY, has 1 handler(s) for this service...
Handler(s):
"DEDICATED" established:2 refused:0 state:ready
REMOTE SERVER
(ADDRESS=(PROTOCOL=TCP)(HOST=10.0.20.201)(PORT=1521))
Service "ORCLCDBXDB" has 2 instance(s).
Instance "ORCLCDB1", status READY, has 1 handler(s) for this service...
Handler(s):
"D000" established:0 refused:0 current:0 max:1022 state:ready
DISPATCHER <machine: racnodep1, pid: 2143150>
(ADDRESS=(PROTOCOL=tcp)(HOST=racnodep1.example.info)(PORT=44707))
Instance "ORCLCDB2", status READY, has 1 handler(s) for this service...
Handler(s):
"D000" established:0 refused:0 current:0 max:1022 state:ready
DISPATCHER <machine: racnodep2, pid: 1842138>
(ADDRESS=(PROTOCOL=tcp)(HOST=racnodep2.example.info)(PORT=44261))
Service "cmon" has 1 instance(s).
Instance "cman", status READY, has 1 handler(s) for this service...
Handler(s):
"cmon" established:9 refused:0 current:2 max:4 state:ready
<machine: localhost, pid: 58901>
(ADDRESS=(PROTOCOL=ipc)(KEY=#58901.1)(KEYPATH=/var/tmp/.oracle_5432100))
Service "orclpdb" has 2 instance(s).
Instance "ORCLCDB1", status READY, has 1 handler(s) for this service...
Handler(s):
"DEDICATED" established:0 refused:0 state:ready
REMOTE SERVER
(ADDRESS=(PROTOCOL=TCP)(HOST=10.0.20.200)(PORT=1521))
Instance "ORCLCDB2", status READY, has 1 handler(s) for this service...
Handler(s):
"DEDICATED" established:2 refused:0 state:ready
REMOTE SERVER
(ADDRESS=(PROTOCOL=TCP)(HOST=10.0.20.201)(PORT=1521))
The command completed successfully.
CMCTL:CMAN_racnodepc1-cman.example.info>

Thanks a lot @psaini79

[psaini79]

@srivallisudha , thanks for confirming it worked. We will fix the README.md and update you to grab the changes.

[s1saurabh]

@psaini79 This has fix, pls merge this #2961

[s1saurabh]

@psaini79 This can me closed. Fix is already merged