Compression: client or server?

[Justin Matlock]

This is a multi-part message in MIME format.

------=_NextPart_000_005E_01C361C1.5FF72280
Content-Type: text/plain;
	charset="iso-8859-1"
Content-Transfer-Encoding: 7bit

Doh. Corrected patch.  gzipuncompress was failing in very rare cases (null
on the end of the compressed data stream).

If you don't want to patch, just remove the ",$len" from the end of the
gzuncompress line. :)

----- Original Message ----- 
From: "Justin Matlock" <jmat@shutdown.net>
To: <memcached@lists.danga.com>
Sent: Wednesday, August 13, 2003 5:24 PM
Subject: Re: Compression: client or server?


> You guys read my mind.  :)
>
> I've been doing this for a few days now...  in tests, it takes 400k text
> (the linux words file), and takes it down to 100k.  The extra time it took
> to compress (a few milliseconds) was insignificant compared to the speed
> increase from the size reduction.  As far as CPU load goes -- I didn't
> notice a major increase in CPU, but it's somewhat hard to measure with
PHP.
> My client machine is a Linux box on a AMD Athlon 700, 256mb RAM, so your
> results may vary.
>
> Attached patch to make the PHP API use gzip compression; it also updates
the
> Documentation file about the new $options['compress'] configuration
> variable, which allows complete disabling of compression, or setting a
> minimum threshold to activate compression (I recommend 10240 from
testing).
>
> Justin
>
>
> ----- Original Message ----- 
> From: "Brad Fitzpatrick" <brad@danga.com>
> To: <memcached@lists.danga.com>
> Sent: Wednesday, August 13, 2003 3:53 PM
> Subject: Re: Compression: client or server?
>
>
> > I agree.
> >
> > The issue then is stats, though.  It'd be interesting to know how much
> > physical vs. logical data is in memcache.  Based on that, you could
adjust
> > the compression threshold on the clients.  I imagine adding an option to
> > the Perl MemCachedClient API that says, "Compress after this size".
> >
> > At lunch just now Evan and I realized another point to add it to the
> > server, though:  what if somebody had more memcache servers than web
> > nodes?  But we don't buy this argument, because in our experience you
> > can't just _buy_ a memcache machine.  We were unable to find a system or
> > parts vendor who could get us a machine with a crap processor and tons
of
> > RAM.
> >
> > So, yeah:  compression definitely client-side, but the server should do
> > stats by the client sending more data about how much it shrank it to.
See
> > any problem with that?
> >
> >
> > On Wed, 13 Aug 2003, Lisa Marie Seelye wrote:
> >
> > > On Wed, 2003-08-13 at 14:44, Brad Fitzpatrick wrote:
> > > > Thoughts?
> > >
> > > What CPU resources are required to compress the data? How much of a
lag
> > > will it introduce to the System?
> > >
> > > The best way to go about storing compressed data is for the client to
> > > _send the data in a compressed format_ and then know well enough to
> > > decompress it when they get it back.
> > >
> > > The server should store whatever the client sends them and don't mess
> > > with it -- put the burden on the user, not the server.
> > >
> > >
> > > --
> > > Regards,
> > > -Lisa
> > > <Vix ulla tam iniqua pax, quin bello vel aequissimo sit potior>
> > >
> >
>

------=_NextPart_000_005E_01C361C1.5FF72280
Content-Type: application/octet-stream;
	name="php-memcached-gzip2.patch"
Content-Transfer-Encoding: quoted-printable
Content-Disposition: attachment;
	filename="php-memcached-gzip2.patch"

diff -f php-memcached-1.0.9/Documentation =
php-memcached-1.0.9gzip/Documentation=0A=
a20=0A=
// $options["compress"] controls gzip compression; set to 0 to completely=0A=
// disable, set to above zero to set the compression threshold: if the=0A=
// data is smaller than this amount (in bytes), don't compress, if it's=0A=
// larger, compress.=0A=
.=0A=
diff -f php-memcached-1.0.9/MemCachedClient.inc.php =
php-memcached-1.0.9gzip/MemCachedClient.inc.php=0A=
a92=0A=
    /**=0A=
     * size of val to force compression; 0 turns off=0A=
     * @ var int=0A=
     */=0A=
    var $compress;  =0A=
.=0A=
a118=0A=
			$this->compress =3D $options["compress"];=0A=
.=0A=
d786=0A=
a839=0A=
		if (($this->compress > 0) && (strlen($val) > $this->compress)) {=0A=
			$this->_debug("_set(): compressing data. size in:".strlen($val));=0A=
			$val=3Dgzcompress($val);=0A=
			$this->_debug("_set(): done compressing data. size =
out:".strlen($val));=0A=
			$flags |=3D 2;=0A=
		}=0A=
=0A=
.=0A=
a1036=0A=
						if(@$flags_array[$sk] & 2)=0A=
							$val[$sk] =3D gzuncompress($val[$sk]);=0A=
.=0A=

------=_NextPart_000_005E_01C361C1.5FF72280--