High Performance Load Balancing Using DSR on a Raspberry Pi 2

TL;DR: Direct Server Return allows a Raspberry Pi or other low-end equipment to balance load at performance impossible to achieve using other methods.

Test conditions

Pi directly connected to laptop with Ethernet.

Laptop running Debian VM in Virtual Box with adapter 1 on NAT and adapter 2 on Ethernet (bridged)

Two more VM:s in VirtualBox on the same laptop to be used as test targets, connected in the same way as the first.

One tricky aspect of this test is that DSR requires a dedicated network interface, and the Pi only has one. This means that everything needs to be set up with the interface configured normally, and then the interface must be reconfigured and the test controlled from the console.

The Debian VM is temporarily set up as gateway to the outside world.

root@debian:~# ifconfig eth1 192.168.100.1/24
root@debian:~# echo 1 > /proc/sys/net/ipv4/ip_forward
root@debian:~# iptables -t nat -A POSTROUTING -o eth0 -j MASQUERADE

And the Pi is temporarily set up with this configuration:

Pi eth0 = 192.168.100.2/24
GW = 192.168.100.1

ping 8.8.8.8 from pi

Works!

Install Pen

Make sure Raspbian on the Pi is up to date.

apt-get update
apt-get upgrade

Continue with instructions from the Wiki.

apt-get install automake autoconf gcc git
mkdir Git
cd Git
git clone https://github.com/UlricE/pen.git
cd pen
aclocal
automake --add-missing
autoconf
./configure
make

Verify the installation

./pen -dfU 53 8.8.8.8:53

And from the Debian VM:

root@debian:~# dig @192.168.100.2 +short siag.nu
194.9.95.65

Pen is now confirmed to work.

Let’s try one more. There is an Apache server running on the Debian VM.

root@raspberrypi:~/Git/pen# ./pen -df 80 192.168.100.1

root@debian:~# lynx -dump http://192.168.100.2/
It works!

This is the default web page for this server.

The web server software is running but no content has been added, yet.

Configure the Pi for DSR

Everything seems good to go. We can now reconfigure eth0 on the Pi.

ifconfig eth0 0.0.0.0

./pen -df -O "dsr_if eth0" -r 192.168.100.10:0 192.168.100.3 192.168.100.4

This means that we intend to forward any TCP traffic with destination address 192.168.100.10 to the two servers 192.168.100.3 and 192.168.100.4, load balanced using round robin.

Those two addresses exist on two additional Debian VM:s on the same laptop. Like the first one they each have eth0 connected to NAT and eth1 connected to wired ethernet.

Set up test targets

Both VM:s need a loopback interface configured with the virtual address
which they must mot tell anyone about:

ifconfig lo:1 192.168.100.10/32
echo 2 > /proc/sys/net/ipv4/conf/all/arp_announce
echo 1 > /proc/sys/net/ipv4/conf/all/arp_ignore

Restart Apache to make sure it listens on the new address:

service apache2 restart

Finally verify that we can access Apache on all addresses:

root@debian:~# lynx -dump http://192.168.100.3/
It works!

This is the default web page for this server.

The web server software is running but no content has been added, yet.
root@debian:~# lynx -dump http://192.168.100.4/
It works!

This is the default web page for this server.

The web server software is running but no content has been added, yet.
root@debian:~# lynx -dump http://192.168.100.10/
It works!

This is the default web page for this server.

The web server software is running but no content has been added, yet.

And here is one of the main reasons for wanting to use DSR:

root@debian:~# ab -n 1000 -c 20 http://192.168.100.10/1000k
This is ApacheBench, Version 2.3 <$Revision: 1604373 $>
Copyright 1996 Adam Twiss, Zeus Technology Ltd, http://www.zeustech.net/
Licensed to The Apache Software Foundation, http://www.apache.org/

Benchmarking 192.168.100.10 (be patient)
Completed 100 requests
Completed 200 requests
Completed 300 requests
Completed 400 requests
Completed 500 requests
Completed 600 requests
Completed 700 requests
Completed 800 requests
Completed 900 requests
Completed 1000 requests
Finished 1000 requests

Server Software: Apache/2.2.22
Server Hostname: 192.168.100.10
Server Port: 80

Document Path: /1000k
Document Length: 1024000 bytes

Concurrency Level: 20
Time taken for tests: 13.427 seconds
Complete requests: 1000
Failed requests: 0
Total transferred: 1024235000 bytes
HTML transferred: 1024000000 bytes
Requests per second: 74.48 [#/sec] (mean)
Time per request: 268.532 [ms] (mean)
Time per request: 13.427 [ms] (mean, across all concurrent requests)
Transfer rate: 74496.13 [Kbytes/sec] received

Connection Times (ms)
min mean[+/-sd] median max
Connect: 1 7 8.3 6 239
Processing: 74 258 139.7 210 1081
Waiting: 0 11 22.4 8 239
Total: 77 265 140.0 217 1087

Percentage of the requests served within a certain time (ms)
50% 217
66% 239
75% 269
80% 390
90% 439
95% 517
98% 691
99% 782
100% 1087 (longest request)

The document 1000k is a dummy file containing 1024000 zeroes. Fetching it at 74.48 requests per second corresponds to a bandwidth of 610 Mbps, a speed physically impossible to achieve through the Pi’s Fast Ethernet interface, but easily achieved using DSR since the return traffic bypasses the load balancer completely. CPU usage on the Pi hovered at 15-20% during the test.

Facebooktwittergoogle_plusredditpinterestlinkedinmail

Leave a Reply

Your email address will not be published. Required fields are marked *