Creation GRE tunnel
Introduction
A GRE tunnel was already established between router-001 and windriver. The objective was to create additional GRE tunnels for redundancy testing:
- Create a GRE tunnel between router-002 and windriver
- Create a GRE tunnel between router-003 and windriver
Incidents
What happened:
- Creating a GRE tunnel on top of an existing GRE tunnel caused a cut connection between router-001 and windriver, and direct access to the router was unavailable,
- The network interface (natif) in windriver was restarted without restoring the routing table.
As a result, the server windriver did not restart properly, and network services were unavailable.
Actions taken:
- Accessed the server via KVM to restore the routing table.
- Performed a controlled network interface restart.
Okey but, how about the link between router-001 and windriver that was broken ?
Technical analysis on windriver:
dmesg showed the following error: gre0: if_output recursively called too many times --> this was the cause of the cut connections between router-001 and windriver. this error comes from the FreeBSD kernel file sys/net/if.c, in the function if_tunnel_check_nesting().
Step by step:
1. A packet needs to reach windriver.
2. The system checks the routing table.
3. It says: “Send it via gre0.”
4. gre0 encapsulates the packet.
5. But to send that encapsulated packet out… The system again decides to use gre0.
6. gre0 encapsulates it again.
7. And again.
8. And again.
Recommandations
- Do not create GRE-over-GRE in the current infrastructure to avoid recursions.
- Changes the network is ok while having access to KVM : always have KVM prepared and the VM itself to prevent any long server routage.
- if restart is needed, always restart natif along with the routing table with one command using "&&"
Alternative Design
We will then create a GRE tunnel between router-002 and router-003 to test the creation of a GRE tunnel with IPsec.
And if everything goes well, we can automate the configurations with Salt and deploy the code to Windriver and the two routers.
Linked to T2201
