TTY management

[Bart Schaefer <schaefer@xxxxxxxxxxxxxxxx>]

There are now at least three threads about terminal handling when the
shell has connected the output(s) of a process to something other than
a terminal device.  There hasn't been any actual wrong information in
the responses, but that the question keeps getting repeated anyway
indicates to me that it's not being explained fully/clearly enough.

First let's talk about "vim | cat".  Programs that expect to use
capabilities of an interactive terminal nearly always make their own
determination of whether they can do so, but how this is done varies a
lot.  Vim in particular works very hard to find a terminal - it will
look at both stdin and stdout, and if either of those is a terminal it
will try to interact with that (assuming the descriptor is open for
both reading and writing).  This is why there are different results
for "cat | vim > ...".  Other programs work even harder and will try
stderr as well.  Still others have a command-line option or other
control to force them to behave as if connected to a terminal even
when they aren't.  Some others always spew terminal control sequences
without bothering to check.  In no case is this under the control of
the shell; if you have a program that doesn't behave as you want,
there's nothing the shell can do about it.

Regarding isatty() -- as has been explained, this is a system call.
It examines an ioctl that is restricted to terminal devices.  The
shell (or any other process) can't force an arbitrary file descriptor
to claim it is a terminal device, that violates the "special files"
model under which terminal devices exist.  Terminal devices operate
using (what to the OS appear to be) hardware-level drivers to mediate
the bidirectional data flow.  Pseudo-terminals split the master and
slave into two descriptors to be able to emulate this, but therefore
have to be managed differently; no single descriptor can simply be
inherited as stdin/out/err by a forked child as is normally done.  The
"script" command exists specifically to do this for you.

There are actually security-related reasons for this -- think 1980/90s
technology -- you don't want a man-in-the-middle attack capturing a
login exchange from a physical terminal in a computer lab, for
example.  Virtual terminal devices came along later and have to
address this differently.

The upshot of this is that, as PWS said, the core shell is not the
place to handle this.  If someone wants to find open-source of the
"script" command and turn it into a module, though, that might be
worth considering.