[PATCH] utf8 flag support on perl lib

Tomash Brechko tomash.brechko at gmail.com
Thu Jan 10 10:09:07 UTC 2008


On Thu, Jan 10, 2008 at 18:21:44 +0900, sugi at nemui.org wrote:
> Hello all,
> 
> Currently, perl library of memcached (Cache::Memcached) will lost
> utf8 flag when treat scalar values. However for reference values,
> the flag will be kept transparently by Storable.
> 
> I wrote a patch and test to keep utf8 flag of scalar values.
> Please consider to merge the patch?

Cache::Memcached::Fast doesn't preserve UTF-8 and tainted flags
either.

Some time ago I implemented the fix for C::M very close to your patch,
but decided not to send it out.  The problem with the proposed
solution is that (man Encode):

  Messing with Perl's Internals

       The following API uses parts of Perl's internals in the current imple-
       mentation.  As such, they are efficient but may change.

       is_utf8(STRING [, CHECK])
         [INTERNAL] Tests whether the UTF-8 flag is turned on in the STRING.
         If CHECK is true, also checks the data in STRING for being well-
         formed UTF-8.  Returns true if successful, false otherwise.

         As of perl 5.8.1, utf8 also has utf8::is_utf8().

       _utf8_on(STRING)
         ...

In other words, the functions you have to use are internal, subject to
change without notice, and appear only in the recent versions of Perl.


Besides, more often than not you use memcached client together with
some other means to get the data if it's missing from the cache.
While you may fix C::M(::F), not every other backend preserves these
flags automatically.  Perhaps a cleaner solution would be to get the
data either from memcached or from another backend, and then to
restore the flags if desired.  It's always safe to enable UTF-8, and
it's up to your script policy to manage tainted flag.


-- 
   Tomash Brechko


More information about the memcached mailing list