Troubleshooting Inter-Node Network Issues in Oracle RAC

When working with Oracle Real Application Clusters (RAC), reliable network connectivity between nodes is critical. If communication between nodes becomes unstable or fails, one of the first steps is to test connectivity at both the public and private network levels.

Let’s walk through a practical approach to test node-to-node connectivity using the ping command with custom packet size and source IP options.

🔍 Scenario Setup

Assume we have a two-node Oracle RAC setup:

• Node A: Hostname racnode1
• Node B: Hostname racnode2

Each node has both public and private IP configurations:

• Public IPs: 192.168.100.1 (Node A) and 192.168.100.2 (Node B)
• Private IPs: 172.16.0.1 (Node A) and 172.16.0.2 (Node B)

🔧 Testing Public Network Connectivity

On Node A:

[grid@racnode1 ~]$ ping -s 9000 -c 2 -I 192.168.100.1 192.168.100.2

On Node B:

[grid@racnode2 ~]$ ping -s 9000 -c 2 -I 192.168.100.2 192.168.100.1

🔧 Testing Private Network Connectivity

On Node A:

[grid@racnode1 ~]$ ping -s 9000 -c 2 -I 172.16.0.1 172.16.0.2

On Node B:

[grid@racnode2 ~]$ ping -s 9000 -c 2 -I 172.16.0.2 172.16.0.1

🛠️ Understanding the Ping Options

• -s 9000: Specifies the packet size (in bytes). Larger packets can help detect MTU (Maximum Transmission Unit) issues.
• -c 2: Sends 2 packets only — enough to test without overwhelming the network.
• -I [source IP]: Forces the use of a specific source IP, crucial when multiple interfaces exist.

❗ Observing the Results

If you see:

2 packets transmitted, 0 packets received, 100% packet loss

This indicates a complete communication failure between the source and destination IPs.

🧩 Common Causes of Connectivity Failures

• 🔥 Firewall restrictions: Ensure that port filtering or packet blocking isn’t applied between the nodes.
• 🔒 Security appliances: IPS/IDS systems might block unusual or large packets.
• 📶 Physical or switch-level issues: Check cables and port configurations.
• ⚙️ Interface misconfiguration: Ensure IP bindings and routes are correct on both nodes.

✅ Final Tip

Always double-check whether any changes to network security policies (e.g., firewall rules, ACLs, or intrusion detection systems) have been implemented recently. These can silently disrupt RAC node communication without immediate visibility.