DN42

Executive summary of this article

This article explains a possible setup for your participation in the DN42 network, using OpenVPN tap for layer 2 tunnels, HP VSR1000 as routers as well as further Linux servers for testing and providing services.

Note: Example instructions for OpenVPN TUN and Quagga are e.g. available here http://dn42.volcanis.me/initenv/wiki/HowToPeer.html (external link)

 

DN42 intro

dn42 (https://dn42.net) is a big dynamic VPN, which employs several Internet technologies like IPv4, IPv6, BGP, whois and DNS, connected via tunnel technology like OpenVPN, GRE, IPSec. It also is interconnected with other, somewhat similar network like Freifunk and ChaosVPN. Primary scope is to enable learning networking stuff like routing using a cool somewhat Internet like setup (for those that cannot learn on real internet connected devices or for experiments that you can’t or shouldn’t do on the Internet), provide a hands-on testing ground, gain experience, and have fun while experimenting with routing.

 

Requirements

  • Internet access
  • Either a static IP or a dyndns solution, to enable peers to reliably know your IP address
  • A VMWare or KVM based environment able to host several virtual machines (for example equivalent to my HP ProLiant N36L MicroServer with ESX as described here). Limitation to ESXi and KVM is relates to the supported hypervisors for the HP VSR. E.g. running it on Virtual Box is not possible.

 

Example setup

A dedicated OpenVPN Linux installation is used to build the OpenVPN tunnels to the peers.

Using two different AS numbers, logically build two sites; a head quarter site (HQ) and a branch site. Each site will get its own core router based on HP VSR.

Both sites will get connected to different peers on DN42, building a BGP peering via L2 OpenVPN tunnel, this way they are not directly connected but only via the DN42 network simulating the Internet.

At the end of the exercise optionally build a GRE/IPSec tunnel between the routers to interconnect the branch site with the HQ.

 

Note on possible changes to the example setup:

The same results can of course be achieved in various ways also using other products than the HP VSR, e.g. Cisco VSR, Cisco XRV, Cisco ASA/PIX, Junipers, or any other routing product that supports dynamic routing protocol BGP. Also Quagga or Bird could be used, those could even be installed (at least one of two instances in this example) on the OpenVPN hosting system. Further also simulators could be used, e.g. GNS3 and HP HCL (a.k.a HP Cloud Lab a.k.a HNS a.k.a HP Network Simulator a.k.a Simware). However keep in mind, that if your peer sees your system offline most of the time, the peering might get cancelled, so preferably build a setup that is online all/most of the time.

 

Register on DN42

Use instructions of the DN42 getting started page https://dn42.net/howto/Getting-started to register a maintainer, person, organization, AS numbers, IPv4 and IPv6 subnets. Registry: https://io.nixnodes.net/?registry

For this example setup, you should register two AS numbers (one for headquarter, one for the branch). 

Further for IPv4, the smallest subnet that can be advertised is a /26 (compared to /24 on the Internet). So even if you split them further to smaller subnet like a /28 and some /30 as transfer subnets, the /26 should be advertised as an aggregate. Consequently you should assign two /26 or a /25 as a minimum. For my setup I used a /24 and split it into 2x/26 for the two sites (HQ & branch) and further /26-/30 for loopbacks, transfer networks, etc. 

Also register an IPv6 range as described there.

 

Plan the setup and prepare environment

To enable smooth connection to DN42, create a good plan on your setup, e.g. which site uses which ASN, which subnets will be advertised to DN42, which subnets will be used to connect servers, which subnets can be used for transfer networks and for loopback addresses/router IDs.

Further how can OpenVPN tunnel traffic reach the OpenVPN linux system setup later (e.g. which UDP ports can be forwarded), and which VLANs can be used to interconnect OpenVPN instance, routers and DN42 servers.

Example setup:

  • OpenVPN Linux system is setup with IP 10.0.4.101, three tunnels to DN42 peers shall be created (one to branch site, two to the HQ site (multihomed))
  • Internet router forwards ports udp/5001-5003 to OpenVPN 10.0.4.101
  • OpenVPN ends tunnel and bridges tunnel #1 to VLAN 201, tunnel #2 to VLAN 202, tunnel #3 to VLAN 203
  • HQ server VM placed in VLAN 211
  • Branch server VM placed in VLAN 212
  • HQ-VSR router connected to VLAN 201, 202 (transfer) and 211 (server)
  • branch VSR router connected to VLAN 203 (transfer) and 212 (server)
  • Consequently HQ ASN must be used in request for tunnel #1 & #2, branch ASN must be used in request for tunnel #3.
  • IPv4: one /24 split as follows
    • /26 used for HQ servers
    • /26 used for branch servers
    • /28 loopback addresses / router IDs
    • Multiple /30 as transfer subnets (one per peer, further used within own AS for further extensions with other devices)
  • IPv6: one /48 subnet, e.g. take last /64 networks as transfer

 

At the end of the planning phase, to request peering with any other DN42 participant later, you should be able to fill all peering details like in the example below:

Peering request to peer eBGP IPv4 & IPv6 via OpenVPN Tap/L2 tunnel
OpenVPN data:
Tunnel type: tap (L2 tunnel)
Protocol: udp
Other specs: mode p2p, comp-lzo, persist-key, persist-tun
My IP address: <static IP or dyndns hostname like somehost.dyndns.org> 
My UDP port: <any port that can be redirected on router to OpenVPN VM, e.g. 5001>
Peer IP: <to be filled by peer during his response>
Peer UDP port: <to be filled by peer during his response>
Shared-key: <attach file with shared key, or agree on a secure way of exchange> 
 
eBGP IPv4 data:
Own ASN: <your ASN, e.g. 4242429999>
Peer ASN: <to be filled by peer during his response>
Proposal transfersubnet: <transfersubnet and mask/CIDR, e.g. 172.23.22.4/30)
Proposal own BGP IP: <from the transfersubnet, e.g. 172.23.22.5>
Proposal peer BGP IP: <from the transfersubnet, e.g. 172.23.22.6>
BGP features enabled: <e.g. BGP graceful restart enabled, BFD enabled>
Info: Own router ID: <router ID, e.g. 172.23.22.129; can be helpful for troubleshooting>
 
eBGP IPv6 data (ASN, router ID and BGP features same as IPv4 above)
Proposal transfersubnet: <transfersubnet and mask/CIDR, e.g. fd42:0:0:ffff::/64)
Proposal own BGP IP: <from the transfersubnet, e.g. fd42:0:0:ffff::1>
Proposal peer BGP IP: <from the transfersubnet, e.g. fd42:0:0:ffff::2>

 

Identify good peers

Using the DN42 peerfinder webpage http://peerfinder.polyno.me/ will test latency to your ISP connection and based on the responsetime give you a list of potential peers that match best.

Prepare the peering request, will be send out later after OpenVPN installation, because generation of a shared secret key is required first.

 

Setup OpenVPN

Create a new VM

usually 384 MB memory should be sufficient. Install a minimal Linux system, e.g. using Ubuntu 14.04 LTS.

Hint: If you e.g. use ESXi as the hypervisor, export the VM now, so it can later be used as a template for further VMs by simply importing the image and changing IP/hostname. If you use Oracle Virtual Box, it is possible to clone systems as a delta of an existing system, this way enables easy duplication of the system with only few additionally required diskspace (as the base system is the same).

On the ESX network configuration, create the VLANs planned above, e.g. 201-203 and 211-212.

Also create a trunk interface (assigning VLAN number 4095 makes the LAN carrying all VLANs).

Update the system config by adding a couple of network interfaces, connected to the different VLANs needed. (Alternatively one could use the trunk interface and work with dot1q subinterfaces in Linux).

 

Startup the VM, Install OpenVPN and some tools

apt-get install openvpn, traceroute, iptraf, tcpdump, bridge-utils

Generate shared secrets for the peering request.

Create a new shared key file within directory /etc/openvpn/. The name doesn’t matter, suggesting to use dn42-AS<targetASN>-shared.key.

Generate a shared key easily using: 

openvpn --genkey --secret /etc/openvpn/dn42-ASxxx-shared.key

Configure the network

Note: Because the local peer will be the VSR router, there is no need to have a local IP address on the tunnels. In case you e.g. want to run Quagga/Bird on the same VM as OpenVPN, change the bridges setup from manual to static and assign the local peers IP address to it. Do not assign IP to tap or eth interface.

root@openvpn:/etc/network# more interfaces
# The loopback network interface
auto lo
iface lo inet loopback
 
# The primary network interface
# This interface also has access to the Internet and incoming tunnels are forwarded to this interface.
auto eth0
iface eth0 inet static
      address <VMs LAN IP where it connects to the Internet, in example 10.0.4.101>
      netmask <netmask, e.g. 255.255.255.0>
      gateway <LAN gateway = typically your Internet facing router>
      dns-nameservers <DNS server = typically your Internet facing router, e.g. 8.8.8.8 also possible>
      dns-search <your domain, e.g. .local>
 
# Now setup interfaces for use with DN42
# For each tunnel, we require an eth interface connected to the target VLAN towards the VSR,
# a tap interface to be used with OpenVPN,
# and a bridge interface interconnecting the tap and eth.
# The tap will be created via openvpn as a placeholder during bridge setup
 
auto eth1
iface eth1 inet manual
 
auto br1
iface br1 inet manual
  pre-up openvpn --mktun --dev tap1
  pre-up ip link set tap1 up
  pre-up ip link set eth1 up
  pre-up brctl addbr br1
  pre-up brctl addif br1 eth1 tap1
  post-down ip link set tap1 down
  post-down ip link set eth1 down
  post-down ip link set br1 down
  post-down brctl delif br1 eth1 tap1
  post-down brctl delbr br1

As needed clone the setup of eth1 for eth2-n to enable additional tunnels. 

Note: Sometimes the order of the Ethernet interfaces is not the same order than in ESXi VM setup. Compare the MAC addresses to find out which eth<x> device is which NIC and thus connected to which VLAN. You can configure Linux to make a specific MAC address a specific eth<n> interface, see good hint at http://ask.xmodulo.com/change-network-interface-names-permanently-linux.html for details (vi /etc/udev/rules.d/70-persistent-net.rules).

 

Request peering

Now your peering request is complete (all data from above, and now with shared secret file). Send it out according to the contact details of the peer and wait for a hopefully positive response.

 Once you received a positive feedback to your peering request, setup the tunnels according to your peering request (including the IP/port information provided by the peer).

To create a new tunnel, create a new config file within directory /etc/openvpn/. The name doesn’t matter, suggesting to use dn42-AS<targetASN>.conf.

Create a config based on example below

remote      <remoteHostnameOrIP e.g. 6.6.6.6>
port        <remotePort e.g. 5001>
proto       udp
mode        p2p
dev-type    tap
dev         <tap device that via bridge connects to correct eth and thus VLAN, e.g. tap1>
comp-lzo
persist-key
persist-tun
secret /etc/openvpn/dn42-ASxxx-shared.key

Once setup is complete, restart the openvpn:

service openvpn restart

This will load all configs and open the tunnels.

Check the syslog (/var/log/syslog) for details/errors.

Check if local port is open/listen as expected: netstat -ltnup

 

Setup VSR for HQ & branch

Install VSR and do base config

Setup 2 VSR instances as described in this article. So at the end you have ssh access to two preconfigured VSRs and you can use cut & paste to apply (a modified version of) below example config.

Setup the peering interface

Assuming the tunnel from your the peer terminates on your OpenVPN instance, and the OpenVPN instance passes traffic from the tunnel to an interface in VLAN 201;

Setup a local interface in VLAN 201 using the IP addresses agreed in the peering request.

interface GigabitEthernet1/0.201
 description transfer to AS xxx
 ip address 172.22.22.21 255.255.255.252
 vlan-type dot1q vid 201
 ipv6 address FD42:0:0:FFFF::1/64

Setup BGP peering

bgp 4242424242
graceful-restart
router-id 172.22.22.1
peer 172.22.22.22 as-number 4242424243
peer 172.22.22.22 bfd
peer FD42:0:0:FFFF::2 as-number 4242424243
peer FD42:0:0:FFFF::2 bfd
address-family ipv4 unicast
  peer 172.22.22.22 enable
address-family ipv6 unicast
  peer FD42:0:0:FFFF::2 enable

Setup a good prefix list filtering networks

Filter incoming prefixes

# We do not want conflicts with our local LAN range, so block any conflicting networks
ip prefix-list dn42-in deny <your local LAN, e.g. 10.0.4.0 24> less-equal 32
# allow all private subnets
ip prefix-list dn42-in permit 10.0.0.0 8 less-equal 32
ip prefix-list dn42-in permit 172.16.0.0 12 less-equal 32
ip prefix-list dn42-in permit 192.168.0.0 16 less-equal 32
# in DN42, block all other, e.g. all public internet blocks as well as the default route
ip prefix-list dn42-in deny 0.0.0.0 0 less-equal 32 
 

Filter advertised networks

# if you do not want to be a transit AS, allow your own subnet only and block all others
ip prefix-list dn42-out permit 172.22.22.0 24 greater-equal 24
ip prefix-list dn42-out deny 0.0.0.0 0 less-equal 32

# Else if transit is ok, you might simply allow all

ip prefix-list dn42-out permit 0.0.0.0 0 less-equal 32

 

Apply filters inside BGP context

bgp <ASN>
address-family ipv4 unicast
  peer <peer IP> prefix-list dn42-out export
  peer <peer IP> prefix-list dn42-in import

 

Alternative concept: Use in-list, but instead of out-list use a route-map allowing your own subnet unmodified and all others with increased AS path only… so your AS can be a backup path solution, but because of long AS path usually your AS is not used, if alternative paths exist.

ip prefix-list dn42-own-prefixes permit 172.23.109.0 24 less-equal 32
ip prefix-list dn42-own-prefixes deny 0.0.0.0 0 less-equal 32
 
ip prefix-list dn42-all-prefixes permit 10.0.0.0 8 less-equal 32
ip prefix-list dn42-all-prefixes permit 172.16.0.0 12 less-equal 32
ip prefix-list dn42-all-prefixes permit 192.168.0.0 16 less-equal 32
ip prefix-list dn42-all-prefixes deny 0.0.0.0 0 less-equal 32 

# own prefixes without modification
route-policy AS-PATH-EXTEND permit node 10
 if-match ip address prefix-list dn42-own-prefixes

# other prefixes add own ASN 4 times in addition – do not use the “replace” option!
route-policy AS-PATH-EXTEND permit node 20
 if-match ip address prefix-list dn42-all-prefixes
 apply as-path 4242422999 4242422999 4242422999 4242422999

# we should have covered all now… if anything not covered by prefix lists, block it.
route-policy AS-PATH-EXTEND deny node 99

Apply the route policy and prefix lists to the peers.

address-family ipv4 unicast
  peer <peer> route-policy AS-PATH-EXTEND export
  peer <peer> prefix-list dn42-in import

 

Optionally limit speed of DN42

If your bandwidth is limited e.g. to 2 Mbit uplink, you might want to limit the maximum bandwidth used by DN42 network. Following setting simply limits the speed to ~1 Mbit on the transfer VLAN.

interface GigabitEthernet1/0.201
qos lr outbound cir 1000

 

Check peering status

First check of course: ping the peer IP addresses to ensure tunnel is up and traffic reaches the other side.

ping 172.22.22.22
ping ipv6 FD42:0:0:FFFF::2

Second check the BGP peer status

dis bgp peer ipv4
dis bgp peer ipv6

Finally check the routing tables, they should contain lots of IPv4 and IPv6 routes now.

display ip routing-table
display ipv6 routing-table
display bgp routing-table ipv4
display bgp routing-table ipv6

 

Summary

You've registered at DN42, planned a network setup and it's BGP peering, installed OpenVPN and VSR routers, established tunnels with other peers using OpenVPN TAP tunnel and finally established BGP sessions to peers including route filtering etc.

Now you can play around with BGP, routing, route modifications, route redistribution, etc. Also you can setup e.g. a webserver in your DN42 address space and make content available to other members. Of course DNS setup would make it easier to access your system... this will be covered in a later article :)

 

Add comment


Security code
Refresh