memory fragmentation issues in 1.2?

[Paul T]

--- Rev Lebaredian <iflassman at gmail.com> wrote:

> >
> > You know that implementations of malloc are not
> casted
> > in stone and the fact that you can bring down one
> > malloc implementation only proves the weakness of
> > particular malloc implementation, right?
> >
> 
> The fact that the malloc implementation cannot be
> depended to behave a
> particular way, makes it a good idea to write one's
> own allocator for
> applications like memcached that stress memory
> allocators in extreme ways.
> I don't want to have to test the implementation of
> malloc on every system I
> install such an application on.

Comes with a price. There are some articles out there
- google for 'custom malloc implementation' - here is
one example: 

http://www.cs.umass.edu/~emery/pubs/berger-oopsla2002.pdf

> 
> By the way - univca (that does many things what
> > memecached does) is using standard malloc with no
> > pooling, runs for months, serves billions of hits
> and
> > never got any 'memory fragmentation' issues.
> >
> 
> Can you say this with certainty for all
> implementations of malloc out there?

Nope. I can not.

That is what open source is for. To test the stuff on
different hardware / software combinations, openly
share the observations e t.c. That is what open source
used to be when I was young ;-) When there was no
money in it, you know ;-)

Rgds.Paul.
 
> Rev
> 
> 
> On 12/7/06, Paul T <pault12345 at yahoo.com> wrote:
> >
> >
> > --- George Schlossnagle <george at omniti.com> wrote:
> >
> > > Paul T wrote:
> > > > Turn off the OOM killer and see what happens?
> > >
> > > I could.  I would expect it to actually OOM and
> die
> > > when it exceeds
> > > addressable memory space.
> >
> > All I'm saying is that it could be that OOM gets
> > nervous too early. If I would post you the cacti
> > pictures from my servers you would see than the
> amount
> > of 'free RAM' goes down for a while. And then at
> some
> > point Linux re-claims it. It's OK behaviour.
> >
> > > >
> > > > If you are running Linux, you might be
> surprised
> > > with
> > > > the behaviour of it's memory manager on
> memcached
> > > node
> > > > dedicated to memcached.
> > >
> > > To be clear, I'm not seeing a performance issue,
> as
> > > I'm not actively
> > > swapping (even when my RSS exceeds phys mem),
> I'm
> > > seeing a loss of
> > > addressable memory issue, which is both
> generally
> > > troubling and
> > > eventually terminal.
> >
> > Are you complaining about the amount of 'free RAM'
> > going down on Linux? If that is what you call
> > 'addressabe RAM' - I see no reason to be troubled.
> >
> > > Memory fragmentation isn't a myth.  I can create
> you
> > > an app quite easily
> > > that does not leak memory but will eventually
> run
> > > out of addressable
> > > memory space and eventually fail in malloc.
> >
> > Please do. I will run it on my build of Linux and
> tell
> > you the results.
> >
> > You know that implementations of malloc are not
> casted
> > in stone and the fact that you can bring down one
> > malloc implementation only proves the weakness of
> > particular malloc implementation, right?
> >
> > > I'm
> > > happy for that not to
> > > be the case here, but that behavior is certainly
> > > realizable (and I've
> > > seen it and had to work around it in large
> > > commercial server
> > > applications as well - so it's not academic
> either).
> >
> > When was that?
> > A lot have changed in Linux codebase in last 10
> years.
> >
> > Rgds.Paul.
> >
> > > George
> > >
> > >
> >
> >
> > __________________________________________________
> > Do You Yahoo!?
> > Tired of spam?  Yahoo! Mail has the best spam
> protection around
> > http://mail.yahoo.com
> >
> 



 
____________________________________________________________________________________
Do you Yahoo!?
Everyone is raving about the all-new Yahoo! Mail beta.
http://new.mail.yahoo.com