Pen 0.34.0 released

Available here:

http://siag.nu/pub/pen/

And also here:

https://sourceforge.net/projects/penloadbalancer/files/Source/

Sander van Burken noticed that it wasn’t possible to specify a listening address
in the configuration file when using UDP: it was created as TCP even if -U was
used on the command line.

Harry G. Coin found that Pen would use the CARP address as local address in
upstream connections. An expected behaviour, but undesirable when CARP is used
with two instances of Pen. A new configuration option allows another source
address to be specified.

Vincent Bernat added support for OpenSSL 1.1.0. OpenSSL 1.0.2 is still supported.

Full list of changes:

161028 Merged pull request from Vincent Bernat for OpenSSL 1.1.0 compatibility.
This fixes issue #28.

161024 Allow setting local address for upstream connections. This fixes issue #31.
New penctl command "source" to set this option.

160914 Fixed issue #30: UDP not working in combination with a configuration file.

Facebooktwittergoogle_plusredditpinterestlinkedinmail

Load balancing load balancers and playing with affinity

Starting with Pen 0.33.0, it is possible to run several Pen instances on the same server and listening on the same IP address and port. The kernel takes care of distributing incoming connections to the instances.

To test this, start Pen like this. The VM test1 has two CPUs:

ulric@test1:~/Git/pen$ taskset -c 0 ./pen -ddf 5001 192.168.2.2 > log0 2>&1
ulric@test1:~/Git/pen$ taskset -c 0 ./pen -ddf 5001 192.168.2.3 > log1 2>&1

From a test client, run iperf in client mode:

ulric@debtest:~/Git/pen$ iperf -c test1 -P 20
------------------------------------------------------------
Client connecting to test1, TCP port 5001
TCP window size: 85.0 KByte (default)
------------------------------------------------------------
[ 22] local 192.168.1.1 port 37596 connected with 192.168.1.2 port 5001
[ 13] local 192.168.1.1 port 37587 connected with 192.168.1.2 port 5001
[ 4] local 192.168.1.1 port 37578 connected with 192.168.1.2 port 5001
[ 10] local 192.168.1.1 port 37584 connected with 192.168.1.2 port 5001
[ 6] local 192.168.1.1 port 37580 connected with 192.168.1.2 port 5001
[ 9] local 192.168.1.1 port 37583 connected with 192.168.1.2 port 5001
[ 7] local 192.168.1.1 port 37581 connected with 192.168.1.2 port 5001
[ 3] local 192.168.1.1 port 37577 connected with 192.168.1.2 port 5001
[ 14] local 192.168.1.1 port 37588 connected with 192.168.1.2 port 5001
[ 12] local 192.168.1.1 port 37586 connected with 192.168.1.2 port 5001
[ 15] local 192.168.1.1 port 37589 connected with 192.168.1.2 port 5001
[ 8] local 192.168.1.1 port 37582 connected with 192.168.1.2 port 5001
[ 5] local 192.168.1.1 port 37579 connected with 192.168.1.2 port 5001
[ 11] local 192.168.1.1 port 37585 connected with 192.168.1.2 port 5001
[ 17] local 192.168.1.1 port 37591 connected with 192.168.1.2 port 5001
[ 16] local 192.168.1.1 port 37590 connected with 192.168.1.2 port 5001
[ 18] local 192.168.1.1 port 37592 connected with 192.168.1.2 port 5001
[ 19] local 192.168.1.1 port 37593 connected with 192.168.1.2 port 5001
[ 20] local 192.168.1.1 port 37594 connected with 192.168.1.2 port 5001
[ 21] local 192.168.1.1 port 37595 connected with 192.168.1.2 port 5001
[ ID] Interval Transfer Bandwidth
[ 12] 0.0-10.0 sec 39.8 MBytes 33.3 Mbits/sec
[ 6] 0.0-10.0 sec 52.8 MBytes 44.2 Mbits/sec
[ 17] 0.0-10.0 sec 55.2 MBytes 46.3 Mbits/sec
[ 21] 0.0-10.0 sec 53.2 MBytes 44.6 Mbits/sec
[ 7] 0.0-10.0 sec 65.2 MBytes 54.6 Mbits/sec
[ 14] 0.0-10.0 sec 63.0 MBytes 52.8 Mbits/sec
[ 5] 0.0-10.0 sec 53.2 MBytes 44.6 Mbits/sec
[ 22] 0.0-10.0 sec 56.2 MBytes 47.0 Mbits/sec
[ 15] 0.0-10.0 sec 50.9 MBytes 42.6 Mbits/sec
[ 16] 0.0-10.0 sec 42.5 MBytes 35.6 Mbits/sec
[ 4] 0.0-10.0 sec 60.9 MBytes 50.9 Mbits/sec
[ 10] 0.0-10.0 sec 51.2 MBytes 42.8 Mbits/sec
[ 3] 0.0-10.0 sec 65.0 MBytes 54.3 Mbits/sec
[ 8] 0.0-10.0 sec 51.4 MBytes 42.9 Mbits/sec
[ 18] 0.0-10.0 sec 37.0 MBytes 30.9 Mbits/sec
[ 9] 0.0-10.0 sec 57.6 MBytes 48.1 Mbits/sec
[ 11] 0.0-10.0 sec 57.5 MBytes 48.0 Mbits/sec
[ 19] 0.0-10.1 sec 39.1 MBytes 32.6 Mbits/sec
[ 13] 0.0-10.1 sec 54.4 MBytes 45.3 Mbits/sec
[ 20] 0.0-10.1 sec 51.6 MBytes 42.9 Mbits/sec
[SUM] 0.0-10.1 sec 1.03 GBytes 880 Mbits/sec

192.168.2.2 and .3 run iperf in server mode.

[ 4] local 192.168.2.2 port 5001 connected with 192.168.2.1 port 47827
[ 18] local 192.168.2.2 port 5001 connected with 192.168.2.1 port 47826
[ 9] local 192.168.2.2 port 5001 connected with 192.168.2.1 port 47837
[ 8] local 192.168.2.2 port 5001 connected with 192.168.2.1 port 47832
[ 7] local 192.168.2.2 port 5001 connected with 192.168.2.1 port 47831
[ 6] local 192.168.2.2 port 5001 connected with 192.168.2.1 port 47829
[ 5] local 192.168.2.2 port 5001 connected with 192.168.2.1 port 47828
[ 10] local 192.168.2.2 port 5001 connected with 192.168.2.1 port 47843
[ 11] local 192.168.2.2 port 5001 connected with 192.168.2.1 port 47844
[ 5] 0.0-10.1 sec 53.2 MBytes 44.2 Mbits/sec
[ 10] 0.0-10.1 sec 51.6 MBytes 42.8 Mbits/sec
[ 8] 0.0-10.1 sec 57.5 MBytes 47.6 Mbits/sec
[ 7] 0.0-10.1 sec 57.6 MBytes 47.7 Mbits/sec
[ 4] 0.0-10.1 sec 60.9 MBytes 50.3 Mbits/sec
[ 18] 0.0-10.2 sec 65.0 MBytes 53.6 Mbits/sec
[ 11] 0.0-10.2 sec 56.2 MBytes 46.4 Mbits/sec
[ 6] 0.0-10.2 sec 65.2 MBytes 53.7 Mbits/sec
[ 9] 0.0-10.4 sec 63.0 MBytes 51.0 Mbits/sec
[SUM] 0.0-10.4 sec 530 MBytes 430 Mbits/sec

and

[ 6] local 192.168.2.3 port 5001 connected with 192.168.2.1 port 59344
[ 5] local 192.168.2.3 port 5001 connected with 192.168.2.1 port 59343
[ 4] local 192.168.2.3 port 5001 connected with 192.168.2.1 port 59342
[ 9] local 192.168.2.3 port 5001 connected with 192.168.2.1 port 59339
[ 12] local 192.168.2.3 port 5001 connected with 192.168.2.1 port 59350
[ 11] local 192.168.2.3 port 5001 connected with 192.168.2.1 port 59349
[ 10] local 192.168.2.3 port 5001 connected with 192.168.2.1 port 59348
[ 8] local 192.168.2.3 port 5001 connected with 192.168.2.1 port 59347
[ 7] local 192.168.2.3 port 5001 connected with 192.168.2.1 port 59345
[ 14] local 192.168.2.3 port 5001 connected with 192.168.2.1 port 59354
[ 13] local 192.168.2.3 port 5001 connected with 192.168.2.1 port 59351
[ 12] 0.0-10.1 sec 37.0 MBytes 30.8 Mbits/sec
[ 10] 0.0-10.1 sec 42.5 MBytes 35.3 Mbits/sec
[ 13] 0.0-10.1 sec 39.1 MBytes 32.5 Mbits/sec
[ 5] 0.0-10.1 sec 51.2 MBytes 42.5 Mbits/sec
[ 8] 0.0-10.1 sec 50.9 MBytes 42.1 Mbits/sec
[ 4] 0.0-10.1 sec 51.4 MBytes 42.5 Mbits/sec
[ 9] 0.0-10.1 sec 52.8 MBytes 43.6 Mbits/sec
[ 7] 0.0-10.2 sec 54.4 MBytes 44.9 Mbits/sec
[ 6] 0.0-10.2 sec 39.8 MBytes 32.8 Mbits/sec
[ 11] 0.0-10.2 sec 55.2 MBytes 45.5 Mbits/sec
[ 14] 0.0-10.2 sec 53.2 MBytes 43.9 Mbits/sec
[SUM] 0.0-10.2 sec 528 MBytes 435 Mbits/sec

9 on one and 11 on the other, not perfect balance but pretty good. We can see the from the logs as well:

ulric@test1:~/Git/pen$ wc -l log[01]
95962 log0
96737 log1
192699 total

Now try this with the Pen instances on separate CPUs.

ulric@test1:~/Git/pen$ taskset -c 0 ./pen -ddf 5001 192.168.2.2 > log0 2>&1
ulric@test1:~/Git/pen$ taskset -c 1 ./pen -ddf 5001 192.168.2.3 > log1 2>&1

ulric@debtest:~$ iperf -c test1 -P 20
------------------------------------------------------------
Client connecting to test1, TCP port 5001
TCP window size: 85.0 KByte (default)
------------------------------------------------------------
[ 22] local 192.168.1.1 port 37658 connected with 192.168.1.2 port 5001
[ 12] local 192.168.1.1 port 37648 connected with 192.168.1.2 port 5001
[ 14] local 192.168.1.1 port 37650 connected with 192.168.1.2 port 5001
[ 9] local 192.168.1.1 port 37645 connected with 192.168.1.2 port 5001
[ 4] local 192.168.1.1 port 37640 connected with 192.168.1.2 port 5001
[ 13] local 192.168.1.1 port 37649 connected with 192.168.1.2 port 5001
[ 8] local 192.168.1.1 port 37644 connected with 192.168.1.2 port 5001
[ 5] local 192.168.1.1 port 37641 connected with 192.168.1.2 port 5001
[ 17] local 192.168.1.1 port 37653 connected with 192.168.1.2 port 5001
[ 11] local 192.168.1.1 port 37647 connected with 192.168.1.2 port 5001
[ 3] local 192.168.1.1 port 37639 connected with 192.168.1.2 port 5001
[ 15] local 192.168.1.1 port 37651 connected with 192.168.1.2 port 5001
[ 10] local 192.168.1.1 port 37646 connected with 192.168.1.2 port 5001
[ 7] local 192.168.1.1 port 37643 connected with 192.168.1.2 port 5001
[ 18] local 192.168.1.1 port 37654 connected with 192.168.1.2 port 5001
[ 6] local 192.168.1.1 port 37642 connected with 192.168.1.2 port 5001
[ 19] local 192.168.1.1 port 37655 connected with 192.168.1.2 port 5001
[ 16] local 192.168.1.1 port 37652 connected with 192.168.1.2 port 5001
[ 20] local 192.168.1.1 port 37656 connected with 192.168.1.2 port 5001
[ 21] local 192.168.1.1 port 37657 connected with 192.168.1.2 port 5001
[ ID] Interval Transfer Bandwidth
[ 5] 0.0-10.0 sec 61.9 MBytes 51.9 Mbits/sec
[ 17] 0.0-10.0 sec 59.2 MBytes 49.7 Mbits/sec
[ 4] 0.0-10.0 sec 80.6 MBytes 67.6 Mbits/sec
[ 19] 0.0-10.0 sec 68.5 MBytes 57.4 Mbits/sec
[ 16] 0.0-10.0 sec 61.1 MBytes 51.2 Mbits/sec
[ 21] 0.0-10.0 sec 52.8 MBytes 44.2 Mbits/sec
[ 3] 0.0-10.0 sec 81.1 MBytes 67.8 Mbits/sec
[ 15] 0.0-10.0 sec 65.9 MBytes 55.1 Mbits/sec
[ 10] 0.0-10.0 sec 50.5 MBytes 42.2 Mbits/sec
[ 7] 0.0-10.0 sec 56.5 MBytes 47.2 Mbits/sec
[ 8] 0.0-10.0 sec 64.1 MBytes 53.6 Mbits/sec
[ 14] 0.0-10.1 sec 62.4 MBytes 51.9 Mbits/sec
[ 12] 0.0-10.1 sec 8.62 MBytes 7.17 Mbits/sec
[ 18] 0.0-10.1 sec 7.25 MBytes 6.02 Mbits/sec
[ 9] 0.0-10.1 sec 8.88 MBytes 7.36 Mbits/sec
[ 13] 0.0-10.1 sec 9.12 MBytes 7.56 Mbits/sec
[ 11] 0.0-10.1 sec 6.75 MBytes 5.60 Mbits/sec
[ 6] 0.0-10.1 sec 5.50 MBytes 4.56 Mbits/sec
[ 20] 0.0-10.1 sec 6.38 MBytes 5.29 Mbits/sec
[ 22] 0.0-10.1 sec 9.25 MBytes 7.66 Mbits/sec
[SUM] 0.0-10.1 sec 826 MBytes 684 Mbits/sec

One iperf server sees less traffic than the other:

[ 13] local 192.168.2.2 port 5001 connected with 192.168.2.1 port 47869
[ 5] local 192.168.2.2 port 5001 connected with 192.168.2.1 port 47874
[ 4] local 192.168.2.2 port 5001 connected with 192.168.2.1 port 47871
[ 7] local 192.168.2.2 port 5001 connected with 192.168.2.1 port 47876
[ 6] local 192.168.2.2 port 5001 connected with 192.168.2.1 port 47875
[ 9] local 192.168.2.2 port 5001 connected with 192.168.2.1 port 47883
[ 8] local 192.168.2.2 port 5001 connected with 192.168.2.1 port 47881
[ 10] local 192.168.2.2 port 5001 connected with 192.168.2.1 port 47884
[ 13] 0.0-10.1 sec 5.50 MBytes 4.56 Mbits/sec
[ 5] 0.0-10.1 sec 6.75 MBytes 5.60 Mbits/sec
[ 9] 0.0-10.1 sec 6.38 MBytes 5.28 Mbits/sec
[ 6] 0.0-10.1 sec 8.62 MBytes 7.14 Mbits/sec
[ 8] 0.0-10.1 sec 7.25 MBytes 6.01 Mbits/sec
[ 4] 0.0-10.1 sec 8.88 MBytes 7.34 Mbits/sec
[ 7] 0.0-10.1 sec 9.12 MBytes 7.55 Mbits/sec
[ 10] 0.0-10.1 sec 9.25 MBytes 7.65 Mbits/sec
[SUM] 0.0-10.1 sec 61.8 MBytes 51.0 Mbits/sec

and

[ 9] local 192.168.2.3 port 5001 connected with 192.168.2.1 port 59386
[ 10] local 192.168.2.3 port 5001 connected with 192.168.2.1 port 59388
[ 8] local 192.168.2.3 port 5001 connected with 192.168.2.1 port 59382
[ 7] local 192.168.2.3 port 5001 connected with 192.168.2.1 port 59381
[ 6] local 192.168.2.3 port 5001 connected with 192.168.2.1 port 59379
[ 5] local 192.168.2.3 port 5001 connected with 192.168.2.1 port 59377
[ 4] local 192.168.2.3 port 5001 connected with 192.168.2.1 port 59376
[ 15] local 192.168.2.3 port 5001 connected with 192.168.2.1 port 59375
[ 11] local 192.168.2.3 port 5001 connected with 192.168.2.1 port 59387
[ 12] local 192.168.2.3 port 5001 connected with 192.168.2.1 port 59394
[ 14] local 192.168.2.3 port 5001 connected with 192.168.2.1 port 59391
[ 13] local 192.168.2.3 port 5001 connected with 192.168.2.1 port 59389
[ 10] 0.0-10.1 sec 61.1 MBytes 51.0 Mbits/sec
[ 12] 0.0-10.1 sec 52.8 MBytes 44.0 Mbits/sec
[ 13] 0.0-10.1 sec 59.2 MBytes 49.4 Mbits/sec
[ 8] 0.0-10.1 sec 50.5 MBytes 42.0 Mbits/sec
[ 6] 0.0-10.1 sec 56.5 MBytes 47.0 Mbits/sec
[ 4] 0.0-10.1 sec 80.6 MBytes 67.1 Mbits/sec
[ 7] 0.0-10.1 sec 64.1 MBytes 53.3 Mbits/sec
[ 11] 0.0-10.1 sec 65.9 MBytes 54.7 Mbits/sec
[ 14] 0.0-10.1 sec 68.5 MBytes 57.0 Mbits/sec
[ 15] 0.0-10.1 sec 81.1 MBytes 67.4 Mbits/sec
[ 9] 0.0-10.1 sec 62.4 MBytes 51.8 Mbits/sec
[ 5] 0.0-10.1 sec 61.9 MBytes 51.4 Mbits/sec
[SUM] 0.0-10.1 sec 765 MBytes 635 Mbits/sec

Quite unbalanced:

ulric@test1:~/Git/pen$ wc -l log[01]
13711 log0
377374 log1
391085 total

Facebooktwittergoogle_plusredditpinterestlinkedinmail

Pen 0.33.0 released

Available here:

http://siag.nu/pub/pen/

And also here:

https://sourceforge.net/projects/penloadbalancer/files/Source/

Pen 0.33.0 has been released. UDP streams are now treated as such and not
broken up into individual datagrams.

Issue #22 has been fixed.

Full list of changes from 0.32.0:

160407 Cleaned up code residue surrounded by “#if 0”.
Released 0.33.0.

160407 Added CS_HALFDEAD for UDP streams that haven’t seen traffic in a while.

160321 Bug in pending_and_closing: don’t modify the list we’re looping over.

160318 Updated pen manpage.
Deprecated -Q option (it didn’t do anything since kqueue was already the
default where it was available).
Fixed error handling in epoll support.

160217 Added transparent UDP test case to testsuite.sh.

160128 Contribution from Talik Eichinger: add X-Forwarded-Proto when doing
SSL decryption.

Facebooktwittergoogle_plusredditpinterestlinkedinmail

Pen 0.32.0 released

Available here:

http://siag.nu/pub/pen/

And also here:

https://sourceforge.net/projects/penloadbalancer/files/Source/

Pen 0.32.0 adds tarpit functionality to the Direct Server Return mode. The purpose of tarpitting is to make network scanning harder by producing lots of false positives.

Full list of changes from 0.31.1:

151123 Released 0.32.0.

151120 Added tarpit test case to testsuite.sh.

151117 Tarpit functionality to be used with the DSR mode.

151112 pen.1: removed obsolete -S option, updated defaults for -x and -L.

Facebooktwittergoogle_plusredditpinterestlinkedinmail

Tarpit support in Pen

Pen 0.32 will have built-in tarpit support in its Direct Server Return mode. The feature is enabled by specifying an access control list against which incoming requests are matched. Matching destination addresses will make Pen do two things:

1. Reply to ARP requests to such addresses.
2. Reply to TCP SYN with SYN+ACK.

The idea behind tarpitting is to slow down network scanning by giving lots of false positives. Pen does this with very little load and without having to manage any state.

Here is an example command line.

pen -df -O “acl 1 permit 192.168.2.11” -O “tarpit_acl 1” -O “dsr_if eth1” 192.168.2.10:80 192.168.2.2 192.168.2.3

Let’s go through that option by option.

acl 1 permit 192.168.2.11 creates an entry in access list 1 which matches IP address 192.168.2.11. All other IP addresses will be rejected.

tarpit_acl 1 makes Pen use access list 1, the one with 192.168.2.11 as its sole entry, to match destination addresses.

dsr_if eth1 makes Pen use eth1 as the network interface where all direct server return processing is performed.

192.168.2.10:80 is the address and port where Pen listens for legitimate requests. They will be forwarded to the backend servers.

192.168.2.2 and 192.168.2.3 are the backend servers. They have web servers listening on port 80 and IP address 192.168.2.10 configured on a loopback interface. See the Wiki.

Let’s try making a legitimate request.

ulric@debtest:~/Git/pen$ curl http://192.168.2.10/cgi-bin/remote_addr
192.168.1.1

That worked fine. Frames from us go to Pen, Pen forwards them to one of the web servers, the web server replies directly to us. In Wireshark we see:

dsr

But what happens when we try the same thing on a tarpitted address?

ulric@debtest:~/Git/pen$ curl http://192.168.2.11/cgi-bin/remote_addr
^C

It just hangs. We send SYN, Pen replies with SYN+ACK, we send ACK and think that the TCP handshake is done. So we send the HTTP request, which Pen ignores. We send it again. Pen ignores it again, and so on. Here’s what that looks like in Wireshark:

tar

Access control lists are a very flexible way to control the tarpit functionality in Pen and have it tarpit every address in a subnet (except those that shouldn’t). As an example, think of a network with the following hosts:

192.168.2.1 gateway
192.168.2.2 web server 1
192.168.2.3 web server 2
192.168.2.10 load balanced address

The corresponding ACL would be created like this:

acl 1 deny 192.168.2.1
acl 1 deny 192.168.2.2
acl 1 deny 192.168.2.3
acl 1 deny 192.168.2.10

Anything Pen sees that is not destined for one of these addresses will be tarpitted.

Facebooktwittergoogle_plusredditpinterestlinkedinmail

Pen 0.31.1 released

Available here:

http://siag.nu/pub/pen/

And also here:

https://sourceforge.net/projects/penloadbalancer/files/Source/

Two bugfixes. The first addresses failover, see issue #19 on Github.
The second addresses how the emergency server is used.

Full list of changes from 0.31.0:

151105 Released 0.31.1.

151103 In failover_server: sanity checks to failover routine.

151102 In add_client: add the initial server to .client as well as .initial.

151029 In failover_server: changed abuse_server to ABUSE_SERVER and emerg_server
to EMERG_SERVER, to handle their default NO_SERVER values.
See issue #19 on Github.

Facebooktwittergoogle_plusredditpinterestlinkedinmail