[PATCH] speed up tokenize_command

[Paolo Borelli]

Il giorno lun, 16/04/2007 alle 15.59 -0700, Marc ha scritto:
> OK I took a closer look.   Looks like it’s faster for O3 and it’s
> cleaner too!  I’ll check with Steve about rolling it in.  Thanks.
> 

Yeah, I don't think O3 matters much here: the main difference should be
that the old version did

while (letter)
    lenght++

while I use pointer arithmetic (end - start) to get the lenght of a
token.

ciao
	Paolo

>