Creation GRE tunnel

Introduction

A GRE tunnel was already established between router-001 and windriver. The objective was to create additional GRE tunnels for redundancy testing:

- Create a GRE tunnel between router-002 and windriver

- Create a GRE tunnel between router-003 and windriver

Incidents

What happened:

• Creating a GRE tunnel on top of an existing GRE tunnel caused a cut connection between router-001 and windriver, and direct access to the router was unavailable,
• The network interface (natif) in windriver was restarted without restoring the routing table.

As a result, the server windriver did not restart properly, and network services were unavailable.

Actions taken:

• Accessed the server via KVM to restore the routing table.
• Performed a controlled network interface restart.

Okey but, how about the link between router-001 and windriver that was broken ?

Technical analysis on windriver:

dmesg showed the following error: gre0: if_output recursively called too many times --> this was the cause of the cut connections between router-001 and windriver. this error comes from the FreeBSD kernel file sys/net/if.c, in the function if_tunnel_check_nesting().

Step by step:

1. A packet needs to reach windriver.

2. The system checks the routing table.

3. It says: “Send it via gre0.”

4. gre0 encapsulates the packet.

5. But to send that encapsulated packet out… The system again decides to use gre0.

6. gre0 encapsulates it again.

7. And again.

8. And again.

Recommandations

• Do not create GRE-over-GRE in the current infrastructure to avoid recursions.

• Changes the network is ok while having access to KVM : always have KVM prepared and the VM itself to prevent any long server routage.

• if restart is needed, always restart natif along with the routing table with one command using "&&"