ip_conntrack problem with memcached
Hansflug (at) freenet
hansflug at freenet.de
Mon Nov 13 09:50:54 UTC 2006
thx for your answer. i tell this your administrator ;) and i write which
solutions we are take and which was successful ;)
<< hans
this.Am Montag, den 13.11.2006, 10:33 +0100 schrieb NTPT:
> stupid me .second rule should be
> iptables -t raw -I OUTPUT -s ip_addrsss of your memcached box --sport port of your memcached daemon -j NOTRACK
> (without -i)
>
> > ------------ Původní zpráva ------------
> > Od: NTPT <NTPT at seznam.cz>
> > Předmět: Re: ip_conntrack problem with memcached
> > Datum: 13.11.2006 10:31:07
> > ----------------------------------------
> > This is because kernel try to track all your memcached connections - ie watch
> > all tcp/ip connection that you made to your memcache. and because conntrack
> > table have a limited size , and it must hold a connection information for
> > relative long time, when this table get overloaded, packets are dropped .
> >
> >
> > there are general tree possible solutions
> >
> > 1: Do not use Ip_conntrack feature of the kernel . ie if you do not need to
> > perform a nat or some fancy packet mangling, SPI or something on your box, do
> > not load (modporbe or insmod ) ip_conntrack module to the kernal .
> >
> > 2: Increase a size of the conntrack table like this
> >
> > echo "yoursize" > /proc/sys/net/ipv4/ip_conntrack_max
> >
> > where "yoursize" is a number of tcp/ip connections to be tracked. something
> > about 32000 - 128000. Work well, but eat up memory and CPU time, with is a big
> > disadvantage on busy sites (and if you get conntrack_table full message, you
> > are a busy site ) because it lower the amounth of available memory for
> > memchached too.
> >
> >
> > 3: Compile your kernel in netfilter section with "RAW table" support and
> > "NOTRACK" target for iptables. With this feature you can (via iptables )
> > easily disable connection tracking for connection to memchached while leaving
> > other connection tracking mechanism for nat, spi and other etc. so you box can
> > do a nat., púacket mangling or SPI , but ip conntrack mechanism in kernal wil
> > not be burdened with memcached related traffic. For future detail consult
> > iptables manual
> >
> > the command wil be something like this (without warranty - for detail man
> > iptables)
> >
> > iptables -t raw -I PREROUTING -i interface to your memchached clients -s
> > ip_addrsss of your memcached clients -d ip_address of your memcached server
> > --dport port of your memcached daemon -j NOTRACK
> >
> > iptables -t raw -I OUTPUT -i interface to your memchached clients -s ip_addrsss
> > of your memcached box --sport port of your memcached daemon -j NOTRACK
> >
> > first rule is for not track incoming traffic to memcached , second is for not
> > tracking of outgoing traffic from memcached.
> >
> >
> > Note thad disablink connection tracking for memcached generated trafic may be
> > generally a very good idea because conntrac mechanism of the kernal eat up
> > memory and cpu time and may be it can improve overall performance of memcached
> > over tcp/ip . real benchmark needed .
> >
> >
> >
> >
> >
> >
> >
> >
> >
> >
> >
> >
> >
> >
> >
> >
> >
> > > ------------ Původní zpráva ------------
> > > Od: Hansflug (at) freenet <hansflug at freenet.de>
> > > Předmět: ip_conntrack problem with memcached
> > > Datum: 13.11.2006 09:39:10
> > > ----------------------------------------
> > > Hello,
> > >
> > > i have a Problem with my Connection to memcached.
> > > in peaktime the kernel log show that i have a ip_conntrack problem ?
> > > i use php5 and memcachelib for save db results.
> > >
> > > What tuning i can do for my problem ????
> > >
> > > here information about my memcache daemon.....
> > > the server no swapping or have no high load. but i have many
> > > connections !!
> > >
> > > server:
> > > 2 CPU but memcache use only one....
> > > 2GB RAM
> > > load average: 0.08, 0.09, 0.09
> > > Cpu(s): 1.3% us, 2.5% sy, 0.0% ni, 94.3% id, 0.0% wa, 0.3% hi,
> > > 1.5% si
> > > Mem: 2077052k total, 1943424k used, 133628k free, 100344k buffers
> > > Swap: 3148720k total, 0k used, 3148720k free, 285896k cached
> > >
> > >
> > > kernel.log
> > >
> > > kernel: ip_conntrack: table full, dropping packet.
> > > last message repeated 9 times
> > > kernel: printk: 3815 messages suppressed.
> > > ...................
> > > ./memcached-tool localhost:11516 display
> > > 6 B 0 s 0 yes
> > > 7 B 0 s 0 yes
> > > 8 256 B 44405 s 8 yes
> > > 9 512 B 48126 s 12 yes
> > > 10 1 kB 51820 s 1 yes
> > > 11 2 kB 38461 s 130 yes
> > > 12 4 kB 42341 s 117 yes
> > > 13 8 kB 39478 s 229 yes
> > > 14 16 kB 59783 s 2 yes
> > > 15 32 kB 833888 s 1 no
> > > 16 B 0 s 0 yes
> > > 17 B 0 s 0 yes
> > >
> > > startparameter
> > > /usr/bin/memcached -m 500 -p 11516 -u root -c 4500
> > >
> > > statsinfo
> > > telnet localhost 11516
> > > Trying 127.0.0.1...
> > > Connected to localhost.
> > > Escape character is '^]'.
> > > stats
> > > STAT pid 9639
> > > STAT uptime 1532210
> > > STAT time 1163406727
> > > STAT version 1.1.12
> > > STAT rusage_user 16376.415406
> > > STAT rusage_system 28586.777149
> > > STAT curr_items 184292
> > > STAT total_items 125758879
> > > STAT bytes 343284616
> > > STAT curr_connections 547
> > > STAT total_connections 43200011
> > > STAT connection_structures 3253
> > > STAT cmd_get 346041673
> > > STAT cmd_set 125758879
> > > STAT get_hits 304561195
> > > STAT get_misses 41480478
> > > STAT bytes_read 90234189974
> > > STAT bytes_written 402886887964
> > > STAT limit_maxbytes 524288000
> > >
> > >
> > >
> > >
> > >
> > >
> >
> >
> >
More information about the memcached
mailing list