Re: Mixing and multiple redirection of stderr/stdout

[Michael Lehn <michael.lehn@xxxxxxxxxx>]

Am 18.05.2011 um 16:26 schrieb Bart Schaefer:

> On May 18, 11:18am, Michael Lehn wrote:
> } 
> } Am 18.05.2011 um 07:13 schrieb Aaron Davies:
> } 
> } > Relatedly, can these (or any other) redirections guarantee the
> } > same interleaving of stdout and stderr that would be produced on
> } > the console with no redirections? I've often seen (mostly in other
> } > shells, iirc) that a process which would have, e.g., alternating
> } > prints to out and err, is captured to file as one long block of out
> } > followed by one long block of err.
> }
> } So can something like my "mixing stdout&stderr but also redirect
> } stdout, stderr separately to files" work at all? The longer I think
> } about it the more I am confused how it could be realized technically.
> } I guess it can not be done by using "open, close, dup and co"
> } internally. Or am I wrong?
> 
> In fact under the covers zsh is arranging something similar to
> 
>   exec 3>file_mix
>   { script.sh |
>     tee -a /proc/fd/3 |
>     cat > file_out } 2>&1 |
>    tee -a /proc/fd/3 |
>    cat > file.err
> 
> so there is pipe buffering introduced that can affect the order of the
> output in the mixed file even though the file is opened only once.

Thanks for clearing this up!  All this redirecting remains uncertain if
one does not have a glue about its internal realization.  

> 
> Even with >file_mix 2>&1 though, the order of output in the file can
> be different from the order of output to the terminal because the
> default OS buffering strategy for file descriptors may differ based
> on the kind of device being written.  The introduction of extra pipes
> just exaggerates this effect.

Yes, that explains the effect!

> 
> On the other hand zsh's internal tee/cat analogs are optimized for
> efficiency rather than for preservation of ordering.  They could be
> redone using select() and non-blocking descriptors to create a much
> closer approximation of the interleaved write.
> 

Right.  So the simplest solution would be writing a small C-programm
doing this.  I understand that a general purpose tool like a shell has
different priorities.

Thanks for your advice!

Michael