Next Previous Contents

22. Monitoring director throughput

Much of this is relatively recent (end 2001).

22.1 ipvsadmm

The number of active/inactive connections are available from the output of ipvsadm.

Julian 22 May 2001

Conns is a counter and is incremented when a new connection is created. It is not incremented when a client re-uses a port to make a new connection (Joe, - this is common with Linux).

director:/etc/lvs# ipvsadm
IP Virtual Server version 0.2.12 (size=16384)                  
Prot LocalAddress:Port Scheduler Flags                         
  -> RemoteAddress:Port             Forward Weight ActiveConn InActConn
TCP  lvs2.mack.net:0 rr persistent 360
  -> bashfull.mack.net:0            Route   1      0          0         
  -> sneezy.mack.net:0              Route   1      0          0         
TCP  lvs2.mack.net:telnet rr
  -> bashfull.mack.net:telnet       Route   1      0          0         
  -> sneezy.mack.net:telnet         Route   1      0          0  

22.2 /proc system (originally /proc/net/ip_vs_stats)

Cyril Bouthors

Where can I get the info originally in /proc/net/ip_vs_stats and removed since 0.9.4?

Wensong Zhang wensong@gnuchina.org 20 Nov 2001

/proc/net/ip_vs_stats is now for global stats.

You can get per-service statistics by ipvsadm -Ln --stats -t|u|f service-address

If you want to program to get statistics info, you can let your program to use libipvs. It is easier to get into in this way.

Here's the write that went with the original code.

Packet throughput (in 64 bit integers) is in /proc/net/ip_vs_stats or /proc/net/ip_masq/vs_stats. The counters are not resetable, you have to keep the previous reading and substract. Output is in hexadecimal.

kernel 2.4:#cat /proc/net/ip_vs_stats
kernel 2.2:#cat /proc/net/ip_masq/vs_stats 

Here's the statistics

director:# more /proc/net/ip_vs_stats
TotalConns   InPkts  OutPkts          InBytes         OutBytes 
      98F9 13945999 13609E49        613AD3B2F       4F90FE6F9E

Virtual Service
Pro VirtService      Conns   InPkts  OutPkts          InBytes         OutBytes                
TCP C0A8026E:0000        4       12        0        00000043B        000000000
TCP C0A8026E:0017        7      3A9        0        00000C3A5        000000000
Real Service
Pro VirtService   RealService      Conns   InPkts  OutPkts          InBytes         OutBytes
TCP C0A8026E:0000 C0A8010C:0000        4       14        0        0000004B4        000000000
TCP C0A8026E:0000 C0A8010B:0000        1        3        0        0000000B4        000000000
TCP C0A8026E:0017 C0A8010C:0017        4       A2        0        00000232A        000000000
TCP C0A8026E:0017 C0A8010B:0017        4      32A        0        00000A827        000000000

Joe

Can I zero out these counters if I want to get rates, or should I store the last count?

Ratz, May 2001

There was a recent (2 months ago) talk about zeroing in-kernel counters and I'm not so sure if all the kernel hacker gurus agreed but:

You must not zero a counter in the kernel!

I didn't really understand the arguments against or pro zeroing counters so I'm not a big help here, but if others agree we certainly can add this feature. It would be ipvsadm -Z as an analogy to ip{chains|tables}. BTW, we are proud of haveing 64 bit counters in the kernel :)

Storing ... there are different approaches to this (complexity order):

22.3 MRTG and LVSGSP

Alexandre Cassen alexandre.cassen@canal-plus.com, the author of keepalived has produced a package, LVSGSP that runs with MRTG to output LVS status information. Currently active and inactive connections are plotted (html/png).

The LVSGSP package includes directions for installing and a sample mrtg.cfg file for monitoring one service. The mrtg.cfg file can be expanded to multiple services

WorkDir: /usr/local/mrtg
IconDir: /usr/local/mrtg/images/

# VS1 10.10.10.2:1358
Target[VS1]: `/usr/local/bin/LVSGSP 10.10.10.2 1358`
Directory[VS1]: LVS
MaxBytes[VS1]: 150
.
.

# VS2 10.10.10.2:8080
Target[VS2]: `/usr/local/bin/LVSGSP 10.10.10.2 8080`
Directory[VS2]: LVS
MaxBytes[VS2]: 150
.
.

A note from Alexandre

Concerning the use of MRTG directly onto the director, we must take care of the computing CPU time monopolised by the MRTG graph generation. On a very overloaded director, the MRTG processing can degrade LVS performance.

22.4 MIB

A MIB has been written for LVS by Romeo Benzoni rb@ssn.tp (Nov 2001). It's available as code and documentation.


Next Previous Contents