Re: Extended globbing seems to have become much slower in recent versions of Zsh

[Jesper Nygårds <jesper.nygards@xxxxxxxxx>]

Here's another run with your supplied patterns. I hope I didn't
misunderstand how you wanted me to use the SECONDS variable.

In any case, as you can see below, the globbing more or less takes the same
time for the simple patterns, but as soon as I introduce the exclusions,
the 5.2 version is around six times slower than the 5.0.5 version.



% du -sh .
2.4G .

% print -l **/*(D) | wc -l
272681

------------------------------------------------------

% /usr/local/bin/zsh --version
zsh 5.2 (x86_64-apple-darwin14.5.0)

% /usr/local/bin/zsh -f

% pwd
/opt/intyg

% setopt EXTENDED_GLOB
% float SECONDS MYNOW

% MYNOW=$SECONDS; myfiles=(**/*); print $(( $SECONDS - $MYNOW ))
20.477513999999996

% MYNOW=$SECONDS; myfiles=(**/*(-.D)); print $(( $SECONDS - $MYNOW ))
20.415430000000001

% MYNOW=$SECONDS;
myfiles=(**/*~(|*/)(build|target|node|node_modules|.idea|*.min.js|*.iml|TAGS)(|/*)(-.D));
print $(( $SECONDS - $MYNOW ))
36.53519

% MYNOW=$SECONDS;
myfiles=((#i)**/*~(|*/)(build|target|node|node_modules|.idea|*.min.js|*.iml|TAGS)(|/*)(-.D));
print $(( $SECONDS - $MYNOW ))
38.467647999999997

% MYNOW=$SECONDS;
myfiles=(**/*~(|*/)(build|target|node|node_modules|.idea|*.min.js|*.iml|TAGS)(|/*));
print $(( $SECONDS - $MYNOW ))
23.871832999999981

MYNOW=$SECONDS;
myfiles=((#i)**/*~(|*/)(build|target|node|node_modules|.idea|*.min.js|*.iml|TAGS)(|/*));
print $(( $SECONDS - $MYNOW ))
23.234887999999955

-------------------------------------------------------

% /bin/zsh --version
zsh 5.0.5 (x86_64-apple-darwin14.0)

% /bin/zsh -f

% pwd
/opt/intyg

% setopt EXTENDED_GLOB
% float SECONDS MYNOW

% MYNOW=$SECONDS; myfiles=(**/*); print $(( $SECONDS - $MYNOW ))
15.603122999999997

% MYNOW=$SECONDS; myfiles=(**/*(-.D)); print $(( $SECONDS - $MYNOW ))
17.826644999999999

% MYNOW=$SECONDS;
myfiles=(**/*~(|*/)(build|target|node|node_modules|.idea|*.min.js|*.iml|TAGS)(|/*)(-.D));
print $(( $SECONDS - $MYNOW ))
6.5746579999999994

% MYNOW=$SECONDS;
myfiles=((#i)**/*~(|*/)(build|target|node|node_modules|.idea|*.min.js|*.iml|TAGS)(|/*)(-.D));
print $(( $SECONDS - $MYNOW ))
6.1519139999999766

% MYNOW=$SECONDS;
myfiles=(**/*~(|*/)(build|target|node|node_modules|.idea|*.min.js|*.iml|TAGS)(|/*));
print $(( $SECONDS - $MYNOW ))
3.6434740000000261

% MYNOW=$SECONDS;
myfiles=((#i)**/*~(|*/)(build|target|node|node_modules|.idea|*.min.js|*.iml|TAGS)(|/*));
print $(( $SECONDS - $MYNOW ))
4.0673059999999737


On Mon, Feb 29, 2016 at 8:12 PM, Bart Schaefer <schaefer@xxxxxxxxxxxxxxxx>
wrote:

> On Feb 28,  9:18pm, Jesper Nygards wrote:
> }
> } The results of my experiments seem to indicate that something has changed
> } relatively recently with regards to extended globbing.
>
> There were a number of changes mostly related to correct handling of
> multibyte characters and single bytes in the range 128-255 appearing
> in file and directory names.  This means some strings are passing
> through the "unmetafication" process more often than they were, but
> I can't say for sure this is the source of any slowdown.
>
> It'd be helpful if you re-ran your tests with more variations of the
> pattern:
>
> **/*
> **/*(-.D)
> **/*~(your long list of exclusions)
> (#i)**/*~(your long list of exclusions)
> **/*~(your long list of exclusions)(-.D)
>
> Also, first declare "float SECONDS" and then use "print $SECONDS" instead
> of "date", so we get a more precise measure of where the time is spent.
>