CGW Troubleshooting
This document provides steps for troubleshooting and diagnosing CGW configurations. It also contains verification methods for different traffic modes, response validation, and log analysis.
Common Symptoms and Causes
| Symptom | Possible Cause |
|---|---|
| No traffic passing through CGW | Incorrect mode selection or configuration error |
| Traffic not routing as expected | Misconfigured routing rules or incorrect mode setting |
| Unable to access certain domains/IPs | Blocked or unallowed IPs/domains in selective mode |
| Unexpected traffic behavior | Misconfigured policy settings or local IP routing issues |
Troubleshooting Steps
- Cloud
- UCI
- Run-Time
- Testing
- Log
Cloud Configuration Verification
Access the CE Terminal
Log in to the CE (Customer Edge) terminal and gain superuser privileges:
sudo su -
Check the Last Configuration Response
To confirm the last applied configuration response, run:
cat /tmp/last_config_response.json |jq .gatewayConfig
This command displays the JSON configuration, allowing you to verify settings like gatewayMode, allowedDomains, blockedIPAddresses, and defaultPolicy. For example:
The given one is just an example output; when this command is run, it will show something like this.
Example Response:
Global
[
{
"translationAddress": "masquerade",
"ceDeviceIp": "100.100.0.77/30",
"allowedDomains": [
"www.google.com"
],
"localSeqId": 19,
"peDeviceIp": "100.100.0.78/30",
"gatewayMode": "GLOBAL",
"blockedIPSubnets": [],
"blockedIPAddresses": [
"13.13.13.13"
],
"blockedDomains": [
"www.facebook.com"
],
"mode": "CE_PEGW",
"defaultPolicy": "ALLOW",
"sourceSubnet": [
"172.30.3.0/24"
],
"zone": "cn",
"id": 1,
"allowedIPAddresses": [
"12.12.12.12"
],
"allowedIPSubnets": [],
"singleArm": null,
"cid": 38
}
]
Full
[
{
"translationAddress": "masquerade",
"ceDeviceIp": "100.100.0.77/30",
"allowedDomains": [
"www.google.com"
],
"localSeqId": 19,
"peDeviceIp": "100.100.0.78/30",
"gatewayMode": "FULL",
"blockedIPSubnets": [],
"blockedIPAddresses": [
"13.13.13.13"
],
"blockedDomains": [
"www.facebook.com"
],
"mode": "CE_PEGW",
"defaultPolicy": "ALLOW",
"sourceSubnet": [
"172.30.3.0/24"
],
"zone": "cn",
"id": 1,
"allowedIPAddresses": [
"12.12.12.12"
],
"allowedIPSubnets": [],
"singleArm": null,
"cid": 38
}
]
Selected
[
{
"translationAddress": "masquerade",
"ceDeviceIp": "100.100.0.77/30",
"allowedDomains": [
"www.google.com"
],
"localSeqId": 19,
"peDeviceIp": "100.100.0.78/30",
"gatewayMode": "SELECTED",
"blockedIPSubnets": [],
"blockedIPAddresses": [
"13.13.13.13"
],
"blockedDomains": [
"www.facebook.com"
],
"mode": "CE_PEGW",
"defaultPolicy": "BLOCK",
"sourceSubnet": [
"172.30.3.0/24"
],
"zone": "cn",
"id": 1,
"allowedIPAddresses": [
"12.12.12.12"
],
"allowedIPSubnets": [],
"singleArm": null,
"cid": 38
}
]
Q:1 What does gatewayMode represent?
Q:2 What does defaultPolicy do?
Q:3 What are the things to keep in mind while verifying JSON output?
UCI Configration Verification
Checking CGW Network Rules
uci show network | grep CGW is used to check the name, IP, gateway etc. details of CGW interface in Single Arm structure.
uci show Network | grep CGW
The given one is just an example output; when this command is run, it will show something like this.
Example Response
network.br38_cgw=interface
network.br38_cgw.ipaddr='100.100.0.93'
network.br38_cgw.netmask='255.255.255.252'
network.br38_cgw.device='@br38'
network.br38_cgw.proto='static'
network.br38_cgw.metric='3'
network.br38_cgw.ip4table='3'
network.br38_cgw.gateway='100.100.0.94'
Checking CGW Firewall Rules
The uci show firewall | grep CGW command is used to check the CGW firewall zone and its rules
uci show firewall | grep CGW
The given one is just an example output; when this command is run, it will show something like this.
Example Response
firewall.CGW_ALLOWED_IPADDRESS=ipset
firewall.CGW_ALLOWED_IPADDRESS.name='CGW_ALLOWED_IPADDRESS'
firewall.CGW_ALLOWED_IPADDRESS.match='dst_net'
firewall.CGW_ALLOWED_IPADDRESS.storage='hash'
firewall.CGW_ALLOWED_IPADDRESS.enabled='1'
firewall.CGW_BLOCKED_IPADDRESS=ipset
firewall.CGW_BLOCKED_IPADDRESS.name='CGW_BLOCKED_IPADDRESS'
firewall.CGW_BLOCKED_IPADDRESS.match='dst_net'
firewall.CGW_BLOCKED_IPADDRESS.storage='hash'
firewall.CGW_BLOCKED_IPADDRESS.enabled='1'
firewall.CGW_ALLOWED_IPSUBNETS=ipset
firewall.CGW_ALLOWED_IPSUBNETS.name='CGW_ALLOWED_IPSUBNETS'
firewall.CGW_ALLOWED_IPSUBNETS.match='dst_net'
firewall.CGW_ALLOWED_IPSUBNETS.storage='hash'
firewall.CGW_ALLOWED_IPSUBNETS.enabled='1'
firewall.CGW_BLOCKED_IPSUBNETS=ipset
firewall.CGW_BLOCKED_IPSUBNETS.name='CGW_BLOCKED_IPSUBNETS'
firewall.CGW_BLOCKED_IPSUBNETS.match='dst_net'
firewall.CGW_BLOCKED_IPSUBNETS.storage='hash'
firewall.CGW_BLOCKED_IPSUBNETS.enabled='1'
firewall.CGW_ALLOWED_DOMAIN=ipset
firewall.CGW_ALLOWED_DOMAIN.name='CGW_ALLOWED_DOMAIN'
firewall.CGW_ALLOWED_DOMAIN.match='dst_net'
firewall.CGW_ALLOWED_DOMAIN.storage='hash'
firewall.CGW_ALLOWED_DOMAIN.enabled='1'
firewall.CGW_BLOCKED_DOMAIN=ipset
firewall.CGW_BLOCKED_DOMAIN.name='CGW_BLOCKED_DOMAIN'
firewall.CGW_BLOCKED_DOMAIN.match='dst_net'
firewall.CGW_BLOCKED_DOMAIN.storage='hash'
firewall.CGW_BLOCKED_DOMAIN.enabled='1'
firewall.CGW_ALLOWED=ipset
firewall.CGW_ALLOWED.name='CGW_ALLOWED'
firewall.CGW_ALLOWED.match='dst_set'
firewall.CGW_ALLOWED.storage='list'
firewall.CGW_ALLOWED.enabled='1'
firewall.CGW_ALLOWED.entry='CGW_ALLOWED_IPADDRESS' 'CGW_ALLOWED_IPSUBNETS' 'CGW_ALLOWED_DOMAIN'
firewall.CGW_BLOCKED=ipset
firewall.CGW_BLOCKED.name='CGW_BLOCKED'
firewall.CGW_BLOCKED.match='dst_set'
firewall.CGW_BLOCKED.storage='list'
firewall.CGW_BLOCKED.enabled='1'
firewall.CGW_BLOCKED.entry='CGW_BLOCKED_IPADDRESS' 'CGW_BLOCKED_IPSUBNETS' 'CGW_BLOCKED_DOMAIN'
Firewall-rules query "CGW" should return any firewall-rules containing the word "CGW," thus allowing inspection and confirmation. If the firewall rules are not defined correctly, it can close or misroute the traffic.
Run time Configuration Verification
Check openvpn Service Status
To check the status of the openvpn network, run the following command.
/etc/init.d/openvpn status
When this command is run, an output similar to the example below appears.
Example Response
running
To manually start or stop the network service, use the following commands.
/etc/init.d/openvpn start
/etc/init.d/openvpn stop
Q:1 What should I check in the CGW_ALLOWED rule?
Q:2 What happens if CGW rules are not defined correctly?
Q:3 How to confirm a specific IP/domain is allowed or blocked?
Testing Verification
Verify Traffic Routing Based on CGW Mode
Global Mode
In global mode, all traffic from the CE device should be routed through the CGW except for traffic destined for the local domain. To test this, perform the following traceroutes:
traceroute -n x.x.x.x
Expected Result: The traceroute to the global IP address should show hops traversing the CGW. The traceroute to the local DNS server IP should not traverse the CGW.
Problem: The configuration is correct if traffic to the global IP address does not pass through the CGW or if traffic from the local domain does pass through the CGW.
Full Mode
In full mode, all traffic by global, selective (if set up), and local domain should be sent over the CGW. Use these commands:
traceroute -n x.x.x.x
Expected Result: Both traceroutes should contain hops crossing over the CGW.
Problem: If any of the traffic passes the CGW, either the mode selection or routing rule needs adjustment.
Selective Mode
Selective mode routes traffic according to defined rules, such as allowed/blocked domains, IPs, and subnets. To test, and execute traceroutes to both allowed and blocked destinations:
traceroute -n x.x.x.x
Expected Result: Traffic to the allowed destinations must be routed over the CGW. Traffic to the blocked destinations should not traverse the CGW.
Problem: If blocked IPs/domains can be reached via the CGW, or allowed traffic is routed around the CGW, then the filtering rules need to be investigated. Selective mode relies on dnsmasq for its functionality.
Q:1 Why check traffic in Global Mode?
Q:2 How to check traffic in Full Mode?
Q:3 How to check allowed and blocked traffic in Selective Mode?
Log Verification
Check System Logs
If the unexpected behavior persists, review the system log for errors or anomalies using:
logread | grep cgw
The given one is just an example output; when this command is run, it will show something like this.
Exmpale Response
Jun 24 07:00:01 manual-testing netifd: Interface 'br38_cgw' is enabled
Jun 24 07:00:01 manual-testing netifd: Interface 'br38_cgw' has link connectivity
Jun 24 07:00:01 manual-testing netifd: Interface 'br38_cgw' is setting up now
Jun 24 07:00:01 manual-testing netifd: Interface 'br38_cgw' is now up
Jun 24 07:00:02 manual-testing mwan3-hotplug[19424]: hotplug called on br38_cgw before mwan3 has been set up
Jun 24 07:00:05 manual-testing firewall: Reloading firewall due to ifup of br38_cgw (br38)
Jun 24 07:00:07 manual-testing firewall: Reloading network due to ifup of br38_cgw (br38)
Jun 24 07:00:10 manual-testing mwan3-hotplug[21818]: Execute ifup event on interface br38_cgw (br38)
Jun 24 07:00:13 manual-testing mwan3track[22215]: Check (ping) success for target "100.100.0.94" on interface br38_cgw (br38). Current score: 0
Jun 24 07:00:13 manual-testing mwan3track[22215]: Interface br38_cgw (br38) is connecting
Jun 24 07:00:19 manual-testing mwan3track[22215]: Check (ping) success for target "100.100.0.94" on interface br38_cgw (br38). Current score: 1
Jun 24 07:00:25 manual-testing mwan3track[22215]: Check (ping) success for target "100.100.0.94" on interface br38_cgw (br38). Current score: 2
Jun 24 07:00:25 manual-testing mwan3track[22215]: Interface br38_cgw (br38) is online
Jun 24 07:00:25 manual-testing mwan3-hotplug[23371]: Execute connected event on interface br38_cgw (br38)
Jun 24 07:00:53 manual-testing mwan3track[22215]: Stopping mwan3track for interface "br38_cgw". Status was "online"
Jun 24 07:00:54 manual-testing mwan3-hotplug[28815]: Execute ifup event on interface br38_cgw (br38)
Jun 24 07:00:56 manual-testing mwan3track[29193]: Check (ping) success for target "100.100.0.94" on interface br38_cgw (br38). Current score: 0
Jun 24 07:00:56 manual-testing mwan3track[29193]: Interface br38_cgw (br38) is connecting
Jun 24 07:01:02 manual-testing mwan3track[29193]: Check (ping) success for target "100.100.0.94" on interface br38_cgw (br38). Current score: 1
Jun 24 07:01:08 manual-testing mwan3track[29193]: Check (ping) success for target "100.100.0.94" on interface br38_cgw (br38). Current score: 2
Jun 24 07:01:08 manual-testing mwan3track[29193]: Interface br38_cgw (br38) is online
Jun 24 07:01:08 manual-testing mwan3-hotplug[30395]: Execute connected event on interface br38_cgw (br38)
Analyze logs to identify problems such as policy misconfigurations, connectivity errors, or authentication failures.