Re: Extended globbing seems to have become much slower in recent versions of Zsh

[Jesper Nygårds <jesper.nygards@xxxxxxxxx>]

Previously I was using the Apple-provided version of 5.0.5, and the
Homebrew version of 5.2. Now I have instead used different versions of
Homebrew zsh, and I have made sure that the compiler flags were the same
through all versions. I have then run the same tests again, with versions
5.0.5, 5.0.6, 5.1 and 5.2.


% /usr/local/bin/zsh --version
zsh 5.0.5 (x86_64-apple-darwin14.5.0)

% /usr/local/bin/zsh -f
% setopt EXTENDED_GLOB; float SECONDS MYNOW

% MYNOW=$SECONDS; :
**/*~(|*/)(build|target|node|node_modules|.idea|*.min.js|*.iml|TAGS)(|/*)(-.D);
print $(( $SECONDS - $MYNOW ))
21.925625999999998

% MYNOW=$SECONDS; :
(#i)**/*~(|*/)(build|target|node|node_modules|.idea|*.min.js|*.iml|TAGS)(|/*)(-.D);
print $(( $SECONDS - $MYNOW ))
15.809167000000002

% MYNOW=$SECONDS; :
**/*~(|*/)(build|target|node|node_modules|.idea|*.min.js|*.iml|TAGS)(|/*);
print $(( $SECONDS - $MYNOW ))
9.8283599999999893

% MYNOW=$SECONDS; :
(#i)**/*~(|*/)(build|target|node|node_modules|.idea|*.min.js|*.iml|TAGS)(|/*);
print $(( $SECONDS - $MYNOW ))
4.0556160000000006

-------------------------------------------------------

% /usr/local/bin/zsh --version
zsh 5.0.6 (x86_64-apple-darwin14.5.0)

% /usr/local/bin/zsh -f
% setopt EXTENDED_GLOB; float SECONDS MYNOW

% MYNOW=$SECONDS; :
**/*~(|*/)(build|target|node|node_modules|.idea|*.min.js|*.iml|TAGS)(|/*)(-.D);
print $(( $SECONDS - $MYNOW ))
51.819927999999997

% MYNOW=$SECONDS; :
(#i)**/*~(|*/)(build|target|node|node_modules|.idea|*.min.js|*.iml|TAGS)(|/*)(-.D);
print $(( $SECONDS - $MYNOW ))
54.338758999999996

% MYNOW=$SECONDS; :
**/*~(|*/)(build|target|node|node_modules|.idea|*.min.js|*.iml|TAGS)(|/*);
print $(( $SECONDS - $MYNOW ))
35.885299000000003

% MYNOW=$SECONDS; :
(#i)**/*~(|*/)(build|target|node|node_modules|.idea|*.min.js|*.iml|TAGS)(|/*);
print $(( $SECONDS - $MYNOW ))
29.573196999999993

-------------------------------------------------------

% /usr/local/bin/zsh --version
zsh 5.1 (x86_64-apple-darwin14.5.0)

% /usr/local/bin/zsh -f
% setopt EXTENDED_GLOB; float SECONDS MYNOW

% MYNOW=$SECONDS; :
**/*~(|*/)(build|target|node|node_modules|.idea|*.min.js|*.iml|TAGS)(|/*)(-.D);
print $(( $SECONDS - $MYNOW ))
55.068858999999989

% MYNOW=$SECONDS; :
(#i)**/*~(|*/)(build|target|node|node_modules|.idea|*.min.js|*.iml|TAGS)(|/*)(-.D);
print $(( $SECONDS - $MYNOW ))
55.551114999999982

% MYNOW=$SECONDS; :
**/*~(|*/)(build|target|node|node_modules|.idea|*.min.js|*.iml|TAGS)(|/*);
print $(( $SECONDS - $MYNOW ))
31.761401000000035

% MYNOW=$SECONDS; :
(#i)**/*~(|*/)(build|target|node|node_modules|.idea|*.min.js|*.iml|TAGS)(|/*);
print $(( $SECONDS - $MYNOW ))
33.984271000000035

-------------------------------------------------------

% /usr/local/bin/zsh --version
zsh 5.2 (x86_64-apple-darwin14.5.0)

% /usr/local/bin/zsh -f
% setopt EXTENDED_GLOB; float SECONDS MYNOW

% MYNOW=$SECONDS; :
**/*~(|*/)(build|target|node|node_modules|.idea|*.min.js|*.iml|TAGS)(|/*)(-.D);
print $(( $SECONDS - $MYNOW ))
46.693610000000007

% MYNOW=$SECONDS; :
(#i)**/*~(|*/)(build|target|node|node_modules|.idea|*.min.js|*.iml|TAGS)(|/*)(-.D);
print $(( $SECONDS - $MYNOW ))
48.288181999999992

% MYNOW=$SECONDS; :
**/*~(|*/)(build|target|node|node_modules|.idea|*.min.js|*.iml|TAGS)(|/*);
print $(( $SECONDS - $MYNOW ))
31.169990000000013

% MYNOW=$SECONDS; :
(#i)**/*~(|*/)(build|target|node|node_modules|.idea|*.min.js|*.iml|TAGS)(|/*);
print $(( $SECONDS - $MYNOW ))
25.161075000000011




On Wed, Mar 2, 2016 at 1:03 AM, Bart Schaefer <schaefer@xxxxxxxxxxxxxxxx>
wrote:

> On Mar 1,  8:11pm, Jesper Nygards wrote:
> }
> } New runs with your suggestion:
>
> Well, that's interesting.  Seems to indicate that building argv to pass
> to builtins is even more expensive than an array assignment.
>
> Still, I can't find anything on my system that causes as dramatic a
> difference as yours from 5.0.5 -> 5.2+.  The (-.D) qualifiers seem to
> have the largest effect for me, for you its the exclusion pattern.
>
> For the sake of covering all bases, are you sure the same compiler
> optimization flags were applied to the 5.0.5 and 5.2 builds you are
> testing?  With your exclusion pattern and about 13,000 files in **/*,
> there are about 100,000 calls to strlen() in either version, so any
> optimization of that might make a large difference.
>
> I did find a small but observable speedup when compiling without
> MULTIBYTE_SUPPORT, too; there were a lot of multibyte repairs to
> pattern.c after 5.0.5.
>