Re: Prompt expansion, multi-job for

[Thomas Köhler <jean-luc@xxxxxxxxxxxxxxxxx>]

On Thu, Jan 06, 2000 at 01:04:55PM +0100,
Oliver Kiddle <opk@xxxxxxxxxxxxx> wrote:
> 
> Andre Pang wrote:
> 
> >     2) Is there any way to emulate make's -jn option in the for command?
> > This would be *really* useful for SMP systems.  Currently, doing something
> > like "for i in ***/***.wav; do l3enc $i; done" doesn't take advantage of
> > multiple CPUs (on a Linux system, at least); hacking the for command to
> > accept a 'make -j'-like parameter (eg: for -j2 i in *; do l3enc $i; done)
> > would allow us lucky ones who have SMP systems to do many things similar to
> > the above without going mad and writing silly Makefiles simply so we can
> > utilise make -j.  If someone's written a function to do this already (Bart?
> > ;) it'd be *very* cool.
> 
> If I'm not mistaken, you should be able to do this by running l3enc as a
> background process, i.e. for i in *.wav; do l3enc $i& done. I haven't
> got an SMP system available to me to test but I'd assume that it would
> allocate different l3enc processes to different processors. Putting the
> process in the background means that zsh will get control back
> straight-away and can run the next l3enc.

Well, bad idea if you have a few hundred wavs :)

> The trouble with this is that it will attempt to run all the l3encs
> together. If you have say 4 processors, it may be most efficient to run
> a maximum of 4 l3encs at once. This would be a bit messy to implement in
> Zsh (compared to something like Ada) but can be done.

Well - let's try something like this:

for i in **/*.wav ; do jspool l3enc "'$i'" ; done
jrun 4

Now, we need jspool and jrun. jspool is simple:
function jspool() {
   print $@ >> ~/.zjspool/jobs
}

Now, jrun:
function jrun() {
   print rm -f ~/.zjspool/jobs >> ~/.zjspool/jobs
   print 1 > ~/.zjspool/current
   i=0
   while [ $i -lt $1 ] ; do
      zjrun $i &!
      i=$[i+1]
   done
}

zjrun is a shell script:

#!/usr/bin/zsh
while : ; do
   sleep 1
   while [ -f ~/.zjspool/lock ] ; do sleep 5 ; done
   print $$ >> ~/.zjspool/lock
   PID=$(head -1 ~/.zjspool/lock)
   if [ $PID -ne $$ ] ; then continue ; fi
   LINE=$(cat ~/.zjspool/current)
   if [ ! -f ~/.zjspool/jobs ] ; then
      rm -f ~/.zjspool/lock
      exit
   fi
   JOB=$(head -$LINE ~/.zjspool/jobs | tail -1)
   print $[LINE + 1] > ~/.zjspool/current
   rm -f ~/.zjspool/lock
   print -n "$1: "
   eval $JOB
done

You get the idea...

Ah - you have 4 jobs running, but you want another important job to run
in between?
echo "XXX" > ~/.zjspool/lock
wait for the 4 running jobs to finish
run your important other stuff 
rm ~/.zjspool/lock

> Oliver Kiddle

CU,
Thomas

-- 
 Thomas Köhler Email:   jean-luc@xxxxxxxxxxxxxxxxx   | LCARS - Linux for
     <><        WWW:  http://home.pages.de/~jeanluc/ | Computers on All
                IRC:             jeanluc             | Real Starships
   PGP public key: http://www.mayn.de/users/jean-luc/PGP-Public.asc