Much of this is relatively recent (end 2001).
The number of active/inactive connections are available from the output of ipvsadm.
Julian 22 May 2001Conns is a counter and is incremented when a new connection is created. It is not incremented when a client re-uses a port to make a new connection (Joe, - this is common with Linux).
director:/etc/lvs# ipvsadm IP Virtual Server version 0.2.12 (size=16384) Prot LocalAddress:Port Scheduler Flags -> RemoteAddress:Port Forward Weight ActiveConn InActConn TCP lvs2.mack.net:0 rr persistent 360 -> bashfull.mack.net:0 Route 1 0 0 -> sneezy.mack.net:0 Route 1 0 0 TCP lvs2.mack.net:telnet rr -> bashfull.mack.net:telnet Route 1 0 0 -> sneezy.mack.net:telnet Route 1 0 0
Cyril Bouthors
Where can I get the info originally in /proc/net/ip_vs_stats and removed since 0.9.4?
Wensong Zhangwensong@gnuchina.org
20 Nov 2001/proc/net/ip_vs_stats is now for global stats.
You can get per-service statistics by ipvsadm -Ln --stats -t|u|f service-address
If you want to program to get statistics info, you can let your program to use libipvs. It is easier to get into in this way.
Here's the write that went with the original code.
Packet throughput (in 64 bit integers) is in /proc/net/ip_vs_stats or /proc/net/ip_masq/vs_stats. The counters are not resetable, you have to keep the previous reading and substract. Output is in hexadecimal.
kernel 2.4:#cat /proc/net/ip_vs_stats kernel 2.2:#cat /proc/net/ip_masq/vs_stats
Here's the statistics
director:# more /proc/net/ip_vs_stats TotalConns InPkts OutPkts InBytes OutBytes 98F9 13945999 13609E49 613AD3B2F 4F90FE6F9E Virtual Service Pro VirtService Conns InPkts OutPkts InBytes OutBytes TCP C0A8026E:0000 4 12 0 00000043B 000000000 TCP C0A8026E:0017 7 3A9 0 00000C3A5 000000000 Real Service Pro VirtService RealService Conns InPkts OutPkts InBytes OutBytes TCP C0A8026E:0000 C0A8010C:0000 4 14 0 0000004B4 000000000 TCP C0A8026E:0000 C0A8010B:0000 1 3 0 0000000B4 000000000 TCP C0A8026E:0017 C0A8010C:0017 4 A2 0 00000232A 000000000 TCP C0A8026E:0017 C0A8010B:0017 4 32A 0 00000A827 000000000
Joe
Can I zero out these counters if I want to get rates, or should I store the last count?
Ratz, May 2001There was a recent (2 months ago) talk about zeroing in-kernel counters and I'm not so sure if all the kernel hacker gurus agreed but:
You must not zero a counter in the kernel!
I didn't really understand the arguments against or pro zeroing counters so I'm not a big help here, but if others agree we certainly can add this feature. It would be ipvsadm -Z as an analogy to ip{chains|tables}. BTW, we are proud of haveing 64 bit counters in the kernel :)
Storing ... there are different approaches to this (complexity order):
- Use a script that extracts the info and writes it flat to a file
- Use MRTG or rrdtool since I reckon you wanted to use the stats to generate some graphics anyway. These tools handle the problem for you. MRTG requires SNMP, but you can have a slightly modified snmpd.conf and execute a script that parses /proc/net/ip_masq/vs_stats and writes it into a file. The advantage of this over the first one is, that you can write the current number into one file and mrtg will know how to draw the graph. I give you an example: We have a customer named plx. Now he has only one service and 2 realserver. We extended the snmpd.conf with following lines:
exec lbsessions /bin/sh /opt/tac/snmp/lbsessions exec lbsessions.plx.total /bin/sh /opt/tac/snmp/lbsessions.plx.total exec lbsessions.plx.web-web1 /bin/sh /opt/tac/snmp/lbsessions.plx.web-web1 exec lbsessions.plx.web-web2 /bin/sh /opt/tac/snmp/lbsessions.plx.web-web2The scripts are awk scripts that get the information accordingly to the service or the realserver. You can then do a table walk of the OID 1.3.6.1.4.1.2021.8 to see what your values are: snmpwalk $IP $COMMUNITY .1.3.6.1.4.1.2021.8 Example output if everything is ok:enterprises.ucdavis.extTable.extEntry.extNames.1 = lbsessions enterprises.ucdavis.extTable.extEntry.extNames.2 = lbsessions.plx.total enterprises.ucdavis.extTable.extEntry.extNames.3 = lbsessions.plx.web-web1 enterprises.ucdavis.extTable.extEntry.extNames.4 = lbsessions.plx.web-web2 enterprises.ucdavis.extTable.extEntry.extCommand.1 = /bin/sh /opt/tac/snmp/lbsessions enterprises.ucdavis.extTable.extEntry.extCommand.2 = /bin/sh /opt/tac/snmp/lbsessions.plx.total enterprises.ucdavis.extTable.extEntry.extCommand.3 = /bin/sh /opt/tac/snmp/lbsessions.plx.web-web1 enterprises.ucdavis.extTable.extEntry.extCommand.4 = /bin/sh /opt/tac/snmp/lbsessions.plx.web-web2 enterprises.ucdavis.extTable.extEntry.extResult.1 = 0 enterprises.ucdavis.extTable.extEntry.extResult.2 = 0 enterprises.ucdavis.extTable.extEntry.extResult.3 = 0 enterprises.ucdavis.extTable.extEntry.extResult.4 = 0 enterprises.ucdavis.extTable.extEntry.extOutput.1 = 292 enterprises.ucdavis.extTable.extEntry.extOutput.2 = -1 enterprises.ucdavis.extTable.extEntry.extOutput.3 = -1 enterprises.ucdavis.extTable.extEntry.extOutput.4 = -1Here you see that the total amount of sessions of the load balancer serving about 8 customers is 292 currently and that customer plx has no connections so far.- Write a MIB for LVS stats.
Alexandre Cassen alexandre.cassen@canal-plus.com
, the author of
keepalived
has produced a package,
LVSGSP
that runs with
MRTG
to output LVS status information.
Currently
active and inactive connections are plotted (html/png).
The LVSGSP package includes directions for installing and a sample mrtg.cfg file for monitoring one service. The mrtg.cfg file can be expanded to multiple services
WorkDir: /usr/local/mrtg IconDir: /usr/local/mrtg/images/ # VS1 10.10.10.2:1358 Target[VS1]: `/usr/local/bin/LVSGSP 10.10.10.2 1358` Directory[VS1]: LVS MaxBytes[VS1]: 150 . . # VS2 10.10.10.2:8080 Target[VS2]: `/usr/local/bin/LVSGSP 10.10.10.2 8080` Directory[VS2]: LVS MaxBytes[VS2]: 150 . .
A note from Alexandre
Concerning the use of MRTG directly onto the director, we must take care of the computing CPU time monopolised by the MRTG graph generation. On a very overloaded director, the MRTG processing can degrade LVS performance.
A MIB has been written for LVS by Romeo Benzoni rb@ssn.tp
(Nov 2001).
It's available as
code and documentation.