Zsh Mailing List Archive
Messages sorted by: Reverse Date, Date, Thread, Author

Re: piping stderr

The behavior you're seeing is due to how the Unix stdio library buffers output. The stderr stream is always line buffered by default. When the file descriptor is open on a tty both stdout is line buffered. When directed to a pipe, fifo, or file stdout is fully buffered. Which means that when you redirect the output to a file or a coprocess (which sends the output through a pipe) the stderr output will typically be seen first because it is flushed as soon as an entire line is written. But even that isn't guaranteed since if the program writes a large amount of data to stdout some or all of the data could be flushed before the first line is written to stderr. The only reliable way to solve this is to redirect the output to a pty (psuedo-tty). One way is to use the script(1) command to run your script and capture its output. 

On 09-05-15 04:47, Atom Smasher wrote:

On Thu, 14 May 2009, Bart Schaefer wrote:


On Thu, May 14, 2009 at 3:49 AM, Atom Smasher <atom@xxxxxxxxxxx> wrote:

i'd like to pipe all stderr from a script into a command to do
logging...
but this isn't synchronous:
exec 2> >( multilog /path/ )

i need it to be synchronous so that multiple scripts running at the
same time (and also logging debug messages) can all log together and
not clobber each others output.


I'm not sure what you think "synchronous" implies here. Within a
particular "instance" of zsh, such as a single running script,
commands can be synchronous in the sense that zsh waits for the
command to finish before going on to the next one. However, multiple
scripts "running at the same time" already implies that the zsh
instances are asynchronous with respect to one another, so no command
started by a particular zsh instance will be synchronous with respect
to other instances (unless you employ some sort of external locking
scheme, but then you're just preventing the scripts from "running at
the same time").

==============

among different scripts (different invocations of the same script)
things are wildly asynchronous. but within each script, which is logging
debug output into multilog, i want to make sure that the stderr and
debug messages appear in the right order. this script shows how that
doesn't (always) happen:

#!/bin/zsh

exec 2> >(rot13)

date
dattxxx
date
dattxxx
date
dattxxx
date
dattxxx

the "dattxxx" commands are there to produce "command not found" errors
on stderr. with zsh 4.2.1 the redirect into a coproc makes it just hang.



That said ... have you tried your example above? I'd be mildly
surprised if multilog doesn't already do something to assure that
multiple loggers don't clobber one another's output. They might
intermingle their output, in which case you need to tag each line of
it with a PID or some such, but Dan Bernstein is generally pretty
compulsive about preventing data loss.

============

if there's one thing multilog does, it makes sure that multiple inputs
don't clobber each other. i am tagging each line with a PID so i can
grep out a particular process and see what's going on.

the problem i'm seeing with the script above, and variations on it, is
that sometimes the output is in the right order, but sometimes all of
stdout appears, followed by all of stderr.
Messages sorted by: Reverse Date, Date, Thread, Author