"SERVER_ERROR out of memory" without -M

Mon Jan 2 01:52:09 UTC 2006

We don't specify -M on our servers, so we shouldn't receive the
"SERVER_ERROR out of memory" error, correct?

I'm having the problem that our 128b slabs are filling up and instead of
kicking out an old item, I'm getting the "SERVER_ERROR out of memory"
message.  My understanding based on the documentation and the code, is
that it would simply kick off the first item at the end of the list that
isn't being accessed, but that doesn't appear to be the case here.

To verify this, I tried to manually create an item of just one byte in
one of the exhausted pools (this works fine in the other pools):

set testing 0 0 1
ERROR
set testing 0 0 1
SERVER_ERROR out of memory
set testing 0 0 1
ERROR
set testing 0 0 1
SERVER_ERROR out of memory

What's interesting is how it isn't every single item, but instead every
other.   Any ideas why I'd be alternating between them?

The stats on one of the memcached instances (wouldn't it be useful if
stats included an argv line?):

stats
STAT pid 1041
STAT uptime 4179949
STAT time 1136166523
STAT version 1.1.11
STAT rusage_user 3195:890000
STAT rusage_system 18467:130000
STAT curr_items 615754
STAT total_items 10758554
STAT bytes 1522770483
STAT curr_connections 15
STAT total_connections 32001085
STAT connection_structures 320
STAT cmd_get 153620631
STAT cmd_set 10757775
STAT get_hits 124470604
STAT get_misses 29150027
STAT bytes_read 27982254387
STAT bytes_written 128254256777
STAT limit_maxbytes 2147483648
END

stats items
STAT items:7:number 212992
STAT items:7:age 884240
STAT items:8:number 8192
STAT items:8:age 1279441
STAT items:9:number 147456
STAT items:9:age 662595
STAT items:10:number 88064
STAT items:10:age 770302
STAT items:11:number 29183
STAT items:11:age 768640
STAT items:12:number 44799
STAT items:12:age 877318
STAT items:13:number 53120
STAT items:13:age 756741
STAT items:14:number 10432
STAT items:14:age 932763
STAT items:15:number 17184
STAT items:15:age 1009452
STAT items:16:number 2384
STAT items:16:age 626366
STAT items:17:number 1392
STAT items:17:age 896439
STAT items:18:number 444
STAT items:18:age 888790
STAT items:19:number 60
STAT items:19:age 894845
STAT items:20:number 51
STAT items:20:age 445419
END

stats slabs
STAT 7:chunk_size 128
STAT 7:chunks_per_page 8192
STAT 7:total_pages 26
STAT 7:total_chunks 212992
STAT 7:used_chunks 212992
STAT 7:free_chunks 0
STAT 7:free_chunks_end 0
STAT 8:chunk_size 256
STAT 8:chunks_per_page 4096
STAT 8:total_pages 2
STAT 8:total_chunks 8192
STAT 8:used_chunks 8192
STAT 8:free_chunks 0
STAT 8:free_chunks_end 0
STAT 9:chunk_size 512
STAT 9:chunks_per_page 2048
STAT 9:total_pages 72
STAT 9:total_chunks 147456
STAT 9:used_chunks 147456
STAT 9:free_chunks 0
STAT 9:free_chunks_end 0
STAT 10:chunk_size 1024
STAT 10:chunks_per_page 1024
STAT 10:total_pages 86
STAT 10:total_chunks 88064
STAT 10:used_chunks 88064
STAT 10:free_chunks 0
STAT 10:free_chunks_end 0
STAT 11:chunk_size 2048
STAT 11:chunks_per_page 512
STAT 11:total_pages 57
STAT 11:total_chunks 29184
STAT 11:used_chunks 29183
STAT 11:free_chunks 1
STAT 11:free_chunks_end 0
STAT 12:chunk_size 4096
STAT 12:chunks_per_page 256
STAT 12:total_pages 175
STAT 12:total_chunks 44800
STAT 12:used_chunks 44799
STAT 12:free_chunks 1
STAT 12:free_chunks_end 0
STAT 13:chunk_size 8192
STAT 13:chunks_per_page 128
STAT 13:total_pages 415
STAT 13:total_chunks 53120
STAT 13:used_chunks 53119
STAT 13:free_chunks 1
STAT 13:free_chunks_end 0
STAT 14:chunk_size 16384
STAT 14:chunks_per_page 64
STAT 14:total_pages 163
STAT 14:total_chunks 10432
STAT 14:used_chunks 10432
STAT 14:free_chunks 0
STAT 14:free_chunks_end 0
STAT 15:chunk_size 32768
STAT 15:chunks_per_page 32
STAT 15:total_pages 537
STAT 15:total_chunks 17184
STAT 15:used_chunks 17184
STAT 15:free_chunks 0
STAT 15:free_chunks_end 0
STAT 16:chunk_size 65536
STAT 16:chunks_per_page 16
STAT 16:total_pages 149
STAT 16:total_chunks 2384
STAT 16:used_chunks 2384
STAT 16:free_chunks 0
STAT 16:free_chunks_end 0
STAT 17:chunk_size 131072
STAT 17:chunks_per_page 8
STAT 17:total_pages 174
STAT 17:total_chunks 1392
STAT 17:used_chunks 1392
STAT 17:free_chunks 0
STAT 17:free_chunks_end 0
STAT 18:chunk_size 262144
STAT 18:chunks_per_page 4
STAT 18:total_pages 111
STAT 18:total_chunks 444
STAT 18:used_chunks 444
STAT 18:free_chunks 0
STAT 18:free_chunks_end 0
STAT 19:chunk_size 524288
STAT 19:chunks_per_page 2
STAT 19:total_pages 30
STAT 19:total_chunks 60
STAT 19:used_chunks 60
STAT 19:free_chunks 0
STAT 19:free_chunks_end 0
STAT 20:chunk_size 1048576
STAT 20:chunks_per_page 1
STAT 20:total_pages 51
STAT 20:total_chunks 51
STAT 20:used_chunks 51
STAT 20:free_chunks 0
STAT 20:free_chunks_end 0
STAT active_slabs 14
STAT total_malloced 2147483648
END

--
Adrian Ziemkowski 
Application Development Manager 
Revelex Corporation 