Version: v24.12.13

CGW Troubleshooting

This document provides steps for troubleshooting and diagnosing CGW (Carrier Gateway) configurations. It also contains verification methods for different traffic modes, response validation, and log analysis.

Common Symptoms and Causes

Symptom	Possible Cause
No traffic passing through CGW	Incorrect mode selection or configuration error
Traffic not routing as expected	Misconfigured routing rules or incorrect mode setting
Unable to access certain domains/IPs	Blocked or unallowed IPs/domains in selective mode
Unexpected traffic behavior	Misconfigured policy settings or local IP routing issues

Troubleshooting Steps

Access the CE Terminal

sudo su -

Check the Last Configuration Response

To confirm the last applied configuration response, run:

cat /tmp/last_config_response.json | jq

This command displays the JSON configuration, allowing you to verify settings like gatewayMode, allowedDomains, blockedIPAddresses, and defaultPolicy. For example:

Example Response:

Global

cat /tmp/last_config_response.json |jq .gatewayConfig
[
  {
    "translationAddress": "masquerade",
    "ceDeviceIp": "100.100.0.77/30",
    "allowedDomains": [
      "www.google.com"
    ],
    "localSeqId": 19,
    "peDeviceIp": "100.100.0.78/30",
    "gatewayMode": "GLOBAL",
    "blockedIPSubnets": [],
    "blockedIPAddresses": [
      "13.13.13.13"
    ],
    "blockedDomains": [
      "www.facebook.com"
    ],
    "mode": "CE_PEGW",
    "defaultPolicy": "ALLOW",
    "sourceSubnet": [
      "172.30.3.0/24"
    ],
    "zone": "cn",
    "id": 1,
    "allowedIPAddresses": [
      "12.12.12.12"
    ],
    "allowedIPSubnets": [],
    "singleArm": null,
    "cid": 38
  }
]

Full

root@jaymin-openWRT:~# cat /tmp/last_config_response.json |jq .gatewayConf
ig
[
  {
    "translationAddress": "masquerade",
    "ceDeviceIp": "100.100.0.77/30",
    "allowedDomains": [
      "www.google.com"
    ],
    "localSeqId": 19,
    "peDeviceIp": "100.100.0.78/30",
    "gatewayMode": "FULL",
    "blockedIPSubnets": [],
    "blockedIPAddresses": [
      "13.13.13.13"
    ],
    "blockedDomains": [
      "www.facebook.com"
    ],
    "mode": "CE_PEGW",
    "defaultPolicy": "ALLOW",
    "sourceSubnet": [
      "172.30.3.0/24"
    ],
    "zone": "cn",
    "id": 1,
    "allowedIPAddresses": [
      "12.12.12.12"
    ],
    "allowedIPSubnets": [],
    "singleArm": null,
    "cid": 38
  }
]

Selected

root@jaymin-openWRT:/tmp# cat last_config_response.json |jq .gatewayConfig
[
  {
    "translationAddress": "masquerade",
    "ceDeviceIp": "100.100.0.77/30",
    "allowedDomains": [
      "www.google.com"
    ],
    "localSeqId": 19,
    "peDeviceIp": "100.100.0.78/30",
    "gatewayMode": "SELECTED",
    "blockedIPSubnets": [],
    "blockedIPAddresses": [
      "13.13.13.13"
    ],
    "blockedDomains": [
      "www.facebook.com"
    ],
    "mode": "CE_PEGW",
    "defaultPolicy": "BLOCK",
    "sourceSubnet": [
      "172.30.3.0/24"
    ],
    "zone": "cn",
    "id": 1,
    "allowedIPAddresses": [
      "12.12.12.12"
    ],
    "allowedIPSubnets": [],
    "singleArm": null,
    "cid": 38
  }
]

Verify Traffic Routing Based on CGW Mode

Global Mode

In global mode, all traffic from the CE device should be routed through the CGW except for traffic destined for the local domain. To test this, perform the following traceroutes:

traceroute -n x.x.x.x

Expected Result: The traceroute to the global IP address should show hops traversing the CGW. The traceroute to the local DNS server IP should not traverse the CGW.
Problem: The configuration is correct if traffic to the global IP address does not pass through the CGW or if traffic from the local domain does pass through the CGW.

Full Mode

In full mode, all traffic by global, selective (if set up), and local domain should be sent over the CGW. Use these commands:

traceroute -n x.x.x.x

Expected Result: Both traceroutes should contain hops crossing over the CGW.
Problem: If any of the traffic passes the CGW, either the mode selection or routing rule needs adjustment.

Selective Mode

Selective mode routes traffic according to defined rules, such as allowed/blocked domains, IPs, and subnets. To test, and execute traceroutes to both allowed and blocked destinations:

traceroute -n x.x.x.x

Expected Result: Traffic to the allowed destinations must be routed over the CGW. Traffic to the blocked destinations should not traverse the CGW.
Problem: If blocked IPs/domains can be reached via the CGW, or allowed traffic is routed around the CGW, then the filtering rules need to be investigated. Selective mode relies on dnsmasq for its functionality.

Check System Logs

If the unexpected behavior persists, review the system log for errors or anomalies using:

logread

Analyze logs to identify problems such as policy misconfigurations, connectivity errors, or authentication failures.

Verify Network Connectivity:

Make sure the CE device has a valid IP address, subnet mask, and gateway. Use the following command to check network configuration:

uci show network

(Example Response):

root@Backup_node:~# uci show network
network.loopback=interface
network.loopback.device='lo'
network.loopback.proto='static'
network.loopback.ipaddr='127.0.0.1'
network.loopback.netmask='255.0.0.0'
network.@globals[0]=globals
network.@globals[0].packet_steering='1'
network.eth0=interface
network.eth0.device='eth0'
network.eth0.default_wan='1'
network.eth0.disabled='0'
network.eth0.proto='static'
network.eth0.ipaddr='172.20.10.8'
network.eth0.netmask='255.255.255.0'
network.eth0.dns='172.20.10.1'
network.eth3=interface
network.eth3.device='eth3'
network.eth3.proto='static'
network.eth3.netmask='255.255.255.0'
network.eth3.disabled='0'
network.eth3.ipaddr='172.30.1.1'
network.@rule[0]=rule
network.@rule[0].priority='901'
network.@rule[0].lookup='main'
network.wlm0=interface
network.wlm0.disabled='1'
network.wlm0.proto='3g'
network.wlm0.pppname='wlm0'
network.wlm0.device='ttyUSB0'
network.wlm0.apn='comgt'
network.wlm0.ipv6='0'
network.wlm0.delegate='0'
network.wlm0.metric='2'
network.wlm0.ip4table='2'
network.f85c71f21c3040bdb4abcd168fa8e900=route
network.f85c71f21c3040bdb4abcd168fa8e900.target='172.30.2.0'
network.f85c71f21c3040bdb4abcd168fa8e900.netmask='255.255.255.0'
network.f85c71f21c3040bdb4abcd168fa8e900.gateway='172.31.0.2'
network.f85c71f21c3040bdb4abcd168fa8e900.table='main'
network.f85c71f21c3040bdb4abcd168fa8e900.proto='static'
network.f85c71f21c3040bdb4abcd168fa8e900.metric='1'
network.f85c71f21c3040bdb4abcd168fa8e900.interface='br25'
network.1777530465de4eafada07376f1239abf=route
network.1777530465de4eafada07376f1239abf.target='172.30.1.0'
network.1777530465de4eafada07376f1239abf.netmask='255.255.255.0'
network.1777530465de4eafada07376f1239abf.gateway='172.31.0.1'
network.1777530465de4eafada07376f1239abf.table='main'
network.1777530465de4eafada07376f1239abf.proto='static'
network.1777530465de4eafada07376f1239abf.metric='1'
network.eth1=interface
network.eth1.proto='static'
network.eth1.device='eth1'
network.eth1.ipaddr='100.100.100.2'
network.eth1.netmask='255.255.255.0'
network.eth1.dns='172.20.10.1'

This output contains information about network settings such as IP address, netmask and DNS server for each interface pay particular attention to the interface used for WAN connectivity ( in this example, probably eth0). It must have a valid IP address and configured DNS server too. In any case, wrong network settings can block the access of the essential for accessing various networks. Ensure that the routes required to access the hiCloud server are properly configured.

Check hiCLOUDS Configuration:

The configuration on the CE device can be verified using the following command.

uci show hiclouds

(Example Response):

root@Backup_node:~# uci show hiclouds
hiclouds.globals=hiclouds
hiclouds.globals.hub='hub.hi-clouds.com'
hiclouds.globals.uri='deviceApi/getEndpointUrl'
hiclouds.globals.request_timeout='30'
hiclouds.globals.machine_id='33b01673-94a7-343e-b403-8c47cabac692'
hiclouds.globals.router_id='08:00:27:82:fc:73'
hiclouds.globals.endpoint='dev.hi-clouds.com'
hiclouds.globals.device_id='677e6da4ab30043a8492148a'
hiclouds.globals.auth_token='eyJhbGciOiJIUzUxMiJ9.eyJzdWIiOiI2NzdlNmRhNGFiMzAwNDNhODQ5MjE0OGE6OmNlOjozM2IwMTY3My05NGE3LTM0M2UtYjQwMy04YzQ3Y2FiYWM2OTI6OjA4OjAwOjI3OjgyOmZjOjczIiwiaWF0IjoxNzM4NjcxOTg5LCJleHAiOjE3Mzg2NzU1ODl9.JU0WBSCzpCbS8PTPypXoCvjka01IKcuKVm54YJnPjtt9jUZdF_5fTyraDc3oirSh67ZzKU7MbYo7Osy84AmoeA'
hiclouds.register=hiclouds
hiclouds.register.interval='30'
hiclouds.register.uri='api/deviceApi/v1/devices/register'
hiclouds.status=hiclouds
hiclouds.status.keepalive_interval='10'
hiclouds.status.status_interval='30'
hiclouds.status.uri='api/deviceApi/v1/devices'
hiclouds.config=hiclouds
hiclouds.config.uri='api/deviceApi/v1/devices'
hiclouds.config.revision='64bd463d-f6ae-4cfc-b89d-c4d74b63a382'
hiclouds.edge=hiclouds_edge
hiclouds.edge.type='CE'
hiclouds.hiclouds=hiclouds
hiclouds.hiclouds.version='22.03.5'
hiclouds.hiclouds.build='b75'
root@Backup_node:~# uci show hiclouds
hiclouds.globals=hiclouds
hiclouds.globals.hub='hub.hi-clouds.com'
hiclouds.globals.uri='deviceApi/getEndpointUrl'
hiclouds.globals.request_timeout='30'
hiclouds.globals.machine_id='33b01673-94a7-343e-b403-8c47cabac692'
hiclouds.globals.router_id='08:00:27:82:fc:73'
hiclouds.globals.endpoint='dev.hi-clouds.com'
hiclouds.globals.device_id='677e6da4ab30043a8492148a'
hiclouds.globals.auth_token='eyJhbGciOiJIUzUxMiJ9.eyJzdWIiOiI2NzdlNmRhNGFiMzAwNDNhODQ5MjE0OGE6OmNlOjozM2IwMTY3My05NGE3LTM0M2UtYjQwMy04YzQ3Y2FiYWM2OTI6OjA4OjAwOjI3OjgyOmZjOjczIiwiaWF0IjoxNzM4NjcxOTg5LCJleHAiOjE3Mzg2NzU1ODl9.JU0WBSCzpCbS8PTPypXoCvjka01IKcuKVm54YJnPjtt9jUZdF_5fTyraDc3oirSh67ZzKU7MbYo7Osy84AmoeA'
hiclouds.register=hiclouds
hiclouds.register.interval='30'
hiclouds.register.uri='api/deviceApi/v1/devices/register'
hiclouds.status=hiclouds
hiclouds.status.keepalive_interval='10'
hiclouds.status.status_interval='30'
hiclouds.status.uri='api/deviceApi/v1/devices'
hiclouds.config=hiclouds
hiclouds.config.uri='api/deviceApi/v1/devices'
hiclouds.config.revision='64bd463d-f6ae-4cfc-b89d-c4d74b63a382'
hiclouds.edge=hiclouds_edge
hiclouds.edge.type='CE'
hiclouds.hiclouds=hiclouds
hiclouds.hiclouds.version='22.03.5'
hiclouds.hiclouds.build='b75'

This output displays the hiCLOUDS-related configurations, such as the hub address, endpoint, deviceId, authToken, and the keepalive_interval. Check whether the hub and endpoint are correct and if the authToken is valid and non-expired. The keepalive_interval specifies how often the device is set to send Keepalive requests. A very short interval may cause unnecessary overloading of the network, while a too big interval may cause the device to be determined as offline in case connectivity is temporarily lost

Checking CGW Firewall Rules

Check that firewall rules for CGW are defined properly:

 uci show firewall | grep CGW

Example Response

root@Backup_node:/tmp# uci show firewall | grep CGW
firewall.CGW_ALLOWED_IPADDRESS=ipset
firewall.CGW_ALLOWED_IPADDRESS.name='CGW_ALLOWED_IPADDRESS'
firewall.CGW_ALLOWED_IPADDRESS.match='dst_net'
firewall.CGW_ALLOWED_IPADDRESS.storage='hash'
firewall.CGW_ALLOWED_IPADDRESS.enabled='1'
firewall.CGW_BLOCKED_IPADDRESS=ipset
firewall.CGW_BLOCKED_IPADDRESS.name='CGW_BLOCKED_IPADDRESS'
firewall.CGW_BLOCKED_IPADDRESS.match='dst_net'
firewall.CGW_BLOCKED_IPADDRESS.storage='hash'
firewall.CGW_BLOCKED_IPADDRESS.enabled='1'
firewall.CGW_ALLOWED_IPSUBNETS=ipset
firewall.CGW_ALLOWED_IPSUBNETS.name='CGW_ALLOWED_IPSUBNETS'
firewall.CGW_ALLOWED_IPSUBNETS.match='dst_net'
firewall.CGW_ALLOWED_IPSUBNETS.storage='hash'
firewall.CGW_ALLOWED_IPSUBNETS.enabled='1'
firewall.CGW_BLOCKED_IPSUBNETS=ipset
firewall.CGW_BLOCKED_IPSUBNETS.name='CGW_BLOCKED_IPSUBNETS'
firewall.CGW_BLOCKED_IPSUBNETS.match='dst_net'
firewall.CGW_BLOCKED_IPSUBNETS.storage='hash'
firewall.CGW_BLOCKED_IPSUBNETS.enabled='1'
firewall.CGW_ALLOWED_DOMAIN=ipset
firewall.CGW_ALLOWED_DOMAIN.name='CGW_ALLOWED_DOMAIN'
firewall.CGW_ALLOWED_DOMAIN.match='dst_net'
firewall.CGW_ALLOWED_DOMAIN.storage='hash'
firewall.CGW_ALLOWED_DOMAIN.enabled='1'
firewall.CGW_BLOCKED_DOMAIN=ipset
firewall.CGW_BLOCKED_DOMAIN.name='CGW_BLOCKED_DOMAIN'
firewall.CGW_BLOCKED_DOMAIN.match='dst_net'
firewall.CGW_BLOCKED_DOMAIN.storage='hash'
firewall.CGW_BLOCKED_DOMAIN.enabled='1'
firewall.CGW_ALLOWED=ipset
firewall.CGW_ALLOWED.name='CGW_ALLOWED'
firewall.CGW_ALLOWED.match='dst_set'
firewall.CGW_ALLOWED.storage='list'
firewall.CGW_ALLOWED.enabled='1'
firewall.CGW_ALLOWED.entry='CGW_ALLOWED_IPADDRESS' 'CGW_ALLOWED_IPSUBNETS' 'CGW_ALLOWED_DOMAIN'
firewall.CGW_BLOCKED=ipset
firewall.CGW_BLOCKED.name='CGW_BLOCKED'
firewall.CGW_BLOCKED.match='dst_set'
firewall.CGW_BLOCKED.storage='list'
firewall.CGW_BLOCKED.enabled='1'
firewall.CGW_BLOCKED.entry='CGW_BLOCKED_IPADDRESS' 'CGW_BLOCKED_IPSUBNETS' 'CGW_BLOCKED_DOMAIN'

Firewall-rules query "CGW" should return any firewall-rules containing the word "CGW," thus allowing inspection and confirmation. If the firewall rules are not defined correctly, it can close or misroute the traffic.

CGW Troubleshooting

Common Symptoms and Causes​

Troubleshooting Steps​

Access the CE Terminal​

Check the Last Configuration Response​

Example Response:​

Global​

Full​

Selected​

Verify Traffic Routing Based on CGW Mode​

Global Mode​

Full Mode​

Selective Mode​

Check System Logs​

Verify Network Connectivity:​

(Example Response):​

Check hiCLOUDS Configuration:​

(Example Response):​

Checking CGW Firewall Rules​

Example Response​

Common Symptoms and Causes

Troubleshooting Steps

Access the CE Terminal

Check the Last Configuration Response

Example Response:

Global

Full

Selected

Verify Traffic Routing Based on CGW Mode

Global Mode

Full Mode

Selective Mode

Check System Logs

Verify Network Connectivity:

(Example Response):

Check hiCLOUDS Configuration:

(Example Response):

Checking CGW Firewall Rules

Example Response