Compression: client or server?

[Justin Matlock]

This is a multi-part message in MIME format.

------=_NextPart_000_004D_01C361BF.B815CC70
Content-Type: text/plain;
	charset="iso-8859-1"
Content-Transfer-Encoding: 7bit

You guys read my mind.  :)

I've been doing this for a few days now...  in tests, it takes 400k text
(the linux words file), and takes it down to 100k.  The extra time it took
to compress (a few milliseconds) was insignificant compared to the speed
increase from the size reduction.  As far as CPU load goes -- I didn't
notice a major increase in CPU, but it's somewhat hard to measure with PHP.
My client machine is a Linux box on a AMD Athlon 700, 256mb RAM, so your
results may vary.

Attached patch to make the PHP API use gzip compression; it also updates the
Documentation file about the new $options['compress'] configuration
variable, which allows complete disabling of compression, or setting a
minimum threshold to activate compression (I recommend 10240 from testing).

Justin


----- Original Message ----- 
From: "Brad Fitzpatrick" <brad@danga.com>
To: <memcached@lists.danga.com>
Sent: Wednesday, August 13, 2003 3:53 PM
Subject: Re: Compression: client or server?


> I agree.
>
> The issue then is stats, though.  It'd be interesting to know how much
> physical vs. logical data is in memcache.  Based on that, you could adjust
> the compression threshold on the clients.  I imagine adding an option to
> the Perl MemCachedClient API that says, "Compress after this size".
>
> At lunch just now Evan and I realized another point to add it to the
> server, though:  what if somebody had more memcache servers than web
> nodes?  But we don't buy this argument, because in our experience you
> can't just _buy_ a memcache machine.  We were unable to find a system or
> parts vendor who could get us a machine with a crap processor and tons of
> RAM.
>
> So, yeah:  compression definitely client-side, but the server should do
> stats by the client sending more data about how much it shrank it to.  See
> any problem with that?
>
>
> On Wed, 13 Aug 2003, Lisa Marie Seelye wrote:
>
> > On Wed, 2003-08-13 at 14:44, Brad Fitzpatrick wrote:
> > > Thoughts?
> >
> > What CPU resources are required to compress the data? How much of a lag
> > will it introduce to the System?
> >
> > The best way to go about storing compressed data is for the client to
> > _send the data in a compressed format_ and then know well enough to
> > decompress it when they get it back.
> >
> > The server should store whatever the client sends them and don't mess
> > with it -- put the burden on the user, not the server.
> >
> >
> > --
> > Regards,
> > -Lisa
> > <Vix ulla tam iniqua pax, quin bello vel aequissimo sit potior>
> >
>

------=_NextPart_000_004D_01C361BF.B815CC70
Content-Type: application/octet-stream;
	name="php-memcached-gzip.patch"
Content-Transfer-Encoding: quoted-printable
Content-Disposition: attachment;
	filename="php-memcached-gzip.patch"

diff -f php-memcached-1.0.9/Documentation =
php-memcached-1.0.9gzip/Documentation=0A=
a20=0A=
// $options["compress"] controls gzip compression; set to 0 to completely=0A=
// disable, set to above zero to set the compression threshold: if the=0A=
// data is smaller than this amount (in bytes), don't compress, if it's=0A=
// larger, compress.=0A=
.=0A=
diff -f php-memcached-1.0.9/MemCachedClient.inc.php =
php-memcached-1.0.9gzip/MemCachedClient.inc.php=0A=
a92=0A=
    /**=0A=
     * size of val to force compression; 0 turns off=0A=
     * @ var int=0A=
     */=0A=
    var $compress;  =0A=
.=0A=
a118=0A=
			$this->compress =3D $options["compress"];=0A=
.=0A=
d786=0A=
a839=0A=
		if (($this->compress > 0) && (strlen($val) > $this->compress)) {=0A=
			$this->_debug("_set(): compressing data. size in:".strlen($val));=0A=
			$val=3Dgzcompress($val);=0A=
			$this->_debug("_set(): done compressing data. size =
out:".strlen($val));=0A=
			$flags |=3D 2;=0A=
		}=0A=
=0A=
.=0A=
a1036=0A=
						if(@$flags_array[$sk] & 2)=0A=
							$val[$sk] =3D gzuncompress($val[$sk],$len);=0A=
.=0A=

------=_NextPart_000_004D_01C361BF.B815CC70--