Next Previous Contents

2. Getting Files

Go to the software link on the LVS homepage. Get the latest patch tarball for the director (available in 2.2.x and 2.4.x flavors). This will contain the kernel patch and source code for various programs and utilities.

2.1 Director Code

You have to patch the standard kernel with the matching ip_vs patch.

The director operates in one (or several) forwarding methods; LVS-NAT (network address translation); LVS-DR (direct routing) and LVS-Tun (tunneling). The mode is chosen individually for each realserver/service by running the user level program ipvsadm.

The director code is well tested on Intel. As well several people are running on Alpha hardware (Flavio Pescuma edtfopa@malmen.ericsson.se on a mailserver, and Philip Copeland copeland@redhat.com who is using it alongside an Intel director to test director failover).

You have to edit the code to run it on Alpha: from Philip (this has been fixed in recent code).

remember use socklen_t NOT size_t for network calls (See page 27 of R.Stevens UNIX Net. prog. vol1 ed 2) and everything suddenly falls into place. Certainly socklen_t is the correct type to use for portability between 32/64 machines, you'll need to update any other code to use socklen_t thats 64 bit eg ia64 sparc64 etc

2.2 RealServer Code

The realservers must be configured appropriately for the LVS forwarding method. You must

Here's this list of OS's that have been tested with the forwarding methods. (we expect all OS's can be made to work one way or another.)

The most common way to handle the arp problem is to hide the VIP from arp requests

Although not used very much (I don't know why), another simple method is to put an extra NIC into the realservers for the VIP and not connect it to the network. The NIC doesn't handle any packets, it's just a way of putting the VIP onto the realserver. The NIC can be an old 10Mbps ISA card. The cost of some 100Mbps PCI tulip cards now is less than the salary you'd pay for the time to recompile a 2.4.x kernel with the hidden patch.

All methods of handling the arp problem work, have about the same performance (throughput, latency) and are about equally easy/difficult to setup.

The method of hiding the devices from arp requests is the closest to the standard NOARP unix behaviour and is the method most commonly used on Linux realservers.

2.3 Configure Script(s)

You can set up the LVS by hand with ipvsadm. This is somewhat tedious and error prone. While do-able for a single LVS configuration, this is not the way to go for setting up lots of different LVS configurations. Some configure tools are available.

The configure script was designed to set up LVS's quickly so I could do testing. The current version (0.9.x) does a wide range of checks, hopefully catching the usual errors. This only sets up a single director and cannot handle director failover.

The configure script uses mon to handle failure of services on realservers. For director failover try Ultra Monkey.

2.4 Priority Routing and ifconfig

The standard network tools (eg ifconfig and route/netstat) aren't capable of setting up some of the features used in newer LVSs. For this we use iproute2, which allows routing based on almost any of the parameters of a packet (src, dest, proto, tos...). iproute2 is available at iproute2-current.tar.gz. iproute2 implements similar functionality to cisco's IOS. If there is only one possible route for packets, then ifconfig and route are just fine. If multiple routes exist then iproute2 is needed.

Example:

In a normally functioning LVS-DR, with routing setup by "route" the realservers will be sending packets with the following routing

In LVS-DR a packet leaving the realserver can exit via the default gw or the director. In the standard setup, packets with dst_addr=RIPnetwork are put onto the local network and all other packets are sent to the default gw.

If instead the routing is setup by "iproute2", packets with src_addr=VIP are sent to the default gw, while packets with src_addr=RIP are put onto the local network. The realservers will be sending packets with the following routing

The result for a normal working LVS, will be the same (i.e. the LVS will still work). However with the standard setup, packets with scr_addr=RIP cannot get to the outside world (the director does not have a default route to 0/0, or else will not forward packets from RIP network). If a process needs this (e.g. the operator needs to telnet out or the realserver needs DNS), then those packets from the RIP can be NAT'ed out via the director. For security all packets from the VIP have to go out the default gw (including those to say the DIP, which will be dropped by rules on the default gw). This prevents spoofing.

iproute2 doesn't use ip_aliases (e.g. eth0:110) and just attaches all addresses to the machine. (ip_tables is based on the same underlying code and also doesn't recognise ip_aliases. If you want to see the network as ip_tables sees it, you need the iproute2 tools.) Presumably ip_aliases are in 2.4 for compatibility with 2.2 scripts

Presumably routing in Linux and the setup of LVS will move more toward using iproute2. The configure script will use the iproute2 package to do some configuration if you have it installed.

Note: iproute2 is not compatible with ifconfig/route/netstat. The entries added by the iproute2 tools are not seen by ifconfig/route etc and the output of ifconfig/route etc will be incorrect. You can't tell from looking at the output of ifconfig/route whether iproute2 commands have been run - you just have to know. The iproute2 tools correctly interpret the results of ifconfig/route commands and will give the correct state of the network.

Unfortunately the user interface to iproute2 is not easy. The documentation is not easy to read (although it was all Julian needed). Ratz suggested "Policy Routing Using Linux" by Matthew G. Marsh, Pub Sams 2001, ISBN 0-672-32052-5, to get you started (it helped me). The output from the commands is difficult to parse (see the configure script for more details).

2.5 Go setup an LVS

If you're just reading through the HOWTO and have got to here, you should now setup an LVS using the LVS-mini-HOWTO.


Next Previous Contents