Re: Using file lines as "input files"

[Dominik Vogt <dominik.vogt@xxxxxx>]

On Sat, Jul 09, 2022 at 04:21:37AM +0200, Mikael Magnusson wrote:
> I realized I misinterpreted the question originally, and the following
> doesn't seem to work 100% but it was a fun idea:
> % mkfifo apipe
> % foo[265000]='' # number of lines in the file

> % cksum apipe$^foo # pass "apipe" to cksum 265000 times

For some mysterious reason that doesn't work with the shwordsplit
option active:

  $ foo[9]=''
  $ setopt shwordsplit
  $ echo x^foo
  x x
  $ unsetopt shwordsplit
  $ echo x^foo
  x x x x x x x x x

> (in another terminal or job control etc)
> % while read; do echo $REPLY > apipe; done < infile
>
> When I tried the above on some test data, I got about 10 broken pipes.
> Also several lines sometimes get passed through the pipe without an
> intervening EOF, I'll admit I don't know the finer points of pipe/fifo
> behavior when you open and close them rapidly.

Hm, a fifo created with mkfifo is automatically blocking.  So,
when either end is opened while the other is not present, it
blocks until the other end is opened.

The reader gets an EOF when there's no more data and no writer has
the fifo open.  Otherwise it waits for more data.

1) Multiple lines processed by a single reader:

 * writer opens the fifo and blocks for a reader
 * reader opens the fifo and blocks for data
 * writer writes its data and closes the fifo
 * the next writes opens the fifo
 * the reader processes the first writer's data but gets no EOF
   because the new writer has the fifo open
 * the new writer writes another line to the fifo and terminates
 * the reader reads the next line, gets an EOF because no writer
   is open and terminates itself.

2) SIGPIPE may be generated in this case:

 * writer opens the fifo and blocks for a reader
 * reader opens the fifo and blocks for data
 * the writer unblocks, writes its data and closes.
 * the reader unblocks consumes the data and gets an EOF
 * the next writer opens the pipe without blocking because the
   reader has not yet closed the fifo
 * the reader closes the fifo
 * the writer tries to write data but gets SIGPIPE because there
   is no reader

Unfortunately fifos have no notion of an EOF as part of the data
stream.

> That said, this also seems to take around 4-5 seconds to run.

A pity that pipes are so uncomfortable to handle in Unix.  I like
that approach.

Ciao

Dominik ^_^  ^_^

--

Dominik Vogt