Re: BUG: Zsh loses history entries since 2015

[Bart Schaefer <schaefer@xxxxxxxxxxxxxxxx>]

On Tue, Mar 25, 2025 at 3:15 AM Michael Stapelberg
<stapelberg+zsh@xxxxxxxxxx> wrote:
>
> Fundamentally, I think the savehistfile() function, when called because
> Zsh is exiting, is not handling an interrupted readhistfile() call correctly.

Outside of your test build, is this something that happens for example
when shutting down a window manager or desktop, so that (possibly
multiple) shells are being "asked" to quit?

Presumably you do keep HIST_SAVE_BY_COPY set?

> I would have attached a patch with a suggested fix, but I am not entirely sure
> which way we should go: Would we want to gracefully handle an interrupted
> readhistfile()? Should readhistfile() even be interruptible at all?

Some users (you appear possibly to be one of them) maintain very large
history files, so it would be impractical for it to try to block
interrupts.  The code from workers/34817 is an attempt to handle more
gracefully something that was already possible, though I no longer
recall what the previous side-effects of interrupt were other than the
parse error warning mentioned in the email thread.  The other side of
this is that readhistfile() could be occurring at various different
points -- particularly shell startup as well as shutdown, but also due
to an "fc" command, which you wouldn't want to be interrupt-proof I
suspect.

I can think of a few possible approaches, none of which seem ideal,
and I have not looked into implementation issues, for example that
these should likely occur only during startup or shutodwn and "fc"
would need another approach:

1) If reading history is interrupted, disable saving history entirely,
perhaps by unset/reset of HISTFILE and/or SAVEHIST.  This would
preserve file contents but lose current shell history.

2) Change behavior to append current shell history to the file as if
APPEND_HISTORY, without attempting to enforce SAVEHIST size or
duplicate expiration.  This is probably the option most likely to
avoid losing anything, but could result in very large files.  Also, if
you're getting interrupts during readhistfile(), what about interrupts
during savehistfile()?

3) Stop reading, but go on saving internal history per user
configuration.  Keeps all the current shell history but would lose
part of the existing history file.  Is that what happens now?