Routing Troubleshooting for PE (Provider Edge) Devices
This document explains how to troubleshoot routing issues on PE (Provider Edge) devices. PE devices are critical components in a service provider's network, acting as a bridge between the customer's network and the service provider's core network.
Troubleshooting Steps Overview
Follow these steps in the prescribed order to efficiently narrow down the problem. Each section builds upon the previous one, ensuring a thorough investigation.
- Cloud Configuration Verification: Validate the intended configuration as it exists in the centralized management system.
- UCI Configuration Verification: Confirm that the centralized configuration has been correctly translated and applied to the device's local configuration.
- Runtime Configuration Verification: Check the active state of routes in the device's forwarding table and ensure critical network services are operational.
- Test Verification & Log Analysis: Perform connectivity tests and review system logs for errors or warnings that indicate the problem.
Issue: Routing Configuration Not Working
Symptom
A problem exists in the routing configuration on the PE device, preventing it from functioning as intended. This typically results in customers losing connectivity to specific parts of the network, or in internet traffic failing to reach its destination. The issue is evident when connectivity tests such as ping or traceroute fail while attempting to reach a customer's IP address from the PE device.
- Cloud
- UCI
- Run-time
- Test
- Logs
Cloud Configuration Verification
The first step is to verify that the desired routing configuration is correctly defined within this central system. This ensures that the instructions sent to the PE device are accurate.
Access the Terminal
Log in to the CE terminal with superuser privileges to perform diagnostic checks.
sudo su -
Check Route Configuration File
This command is used to check the configuration file received by the PE device from the cloud management system.
cat /tmp/last_config_response.json | jq .protocols.staticRoutes
Q:1 Why is cloud configuration verification necessary?
Q:2 Which command is useful for checking routing configuration?
Q:3 What to do if the routing configuration is inaccurate?
UCI Configuration Verification
When the cloud configuration is checked, the next step is to make sure it's successfully applied to the device's local configuration.
Verify Network Configuration Details
To view the static routes set up on the PE device, use the uci show network | grep route command. This command filters and displays only the static route information from the network settings.
uci show network | grep route
Q:1 How can I check the network configuration on a device?
Q:2 What does the uci show network | grep route command do?
Q:3 What steps should be taken after UCI Verification?
Runtime Configuration Verification
Verify Current Routes
Use the ip route show or ip route list command to display the active routing table on the PE device.
ip route show
or
ip route list
Check Network Service Status
To check the status of the route network, run the following command.
/etc/init.d/network status
To manually start or stop the network service, use the following commands.
/etc/init.d/network start
/etc/init.d/network stop
Q:1 How to check the status of network services?
Q:2 Can the ip route show and /etc/init.d/network status commands be used interchangeably?
Test Verification & Log Analysis
Test Connectivity
Use the ping command to test connectivity to the target IP address on the client network. This will confirm whether the route is operational and traffic can successfully flow to the destination.
ping `target-ip`
Replace target-ip with the destination IP to be reached (e.g., a client PE LAN IP). A successful ping indicates that the route is active and basic IP connectivity is established.
Q:1 Which command is used to test connectivity?
Q:2 When is Test Connectivity useful?
Check Logs for Errors
Review the system log for any routing-related errors, warnings, or informational messages.
logread | grep route
This command will filter the system logs for keywords like "error".
These logs can reveal configuration syntax errors, conflicts with existing routes or policies, or problems with the network interfaces that prevent the route from being added to the routing table or from functioning correctly.