Re: The HIST_EXPIRE_DUPS_FIRST might corrupt and wipe partially history file if many shells exit at the same time

[Bart Schaefer <schaefer@xxxxxxxxxxxxxxxx>]

On Sun, Mar 19, 2023 at 4:56 AM Piotr Karbowski
<piotr.karbowski@xxxxxxxxxxxxx> wrote:
>
> They way I can reproduce it sometimes is if multiple zsh shells exit at
> the very same time. It happens when I terminate tmux session that have
> 10+ zsh instances,or when I just reboot my system while I have dozens of
> urxvt instances open.
>
>      +SAVEHIST='96000'
>      +setopt hist_expire_dups_first
>
> I have disabled this feature since due those corruptions. Would love to
> get back to it though, perhaps adding some locking mechanism would help
> here?

There is a locking mechanism.  A couple of things may be happening:

1) Your home directory (or ZDOTDIR) is on a remote filesystem with
asynchronous mounting, so the locking mechanism doesn't always work.
You haven't described your system configuration in any detail, but
from what you have said this one seems unlikely.
2) The locking mechanism doesn't work for some other reason.  Make
sure the HIST_SAVE_BY_COPY option is set (it's the default but check
anyway).
3) When exiting a session (or particularly when rebooting), there may
be a limited amount of time for the processes to clean up and exit
before they are killed.  With the combination of options you have,
each shell must re-read the history file to merge it with its local
history before writing it back.  With multiple shells and tens of
thousands of lines of history, this might take longer than all the
shells are allowed to stay alive.

Try putting something in your .zlogout file that writes a timestamp to
a file (different for each shell, e.g., use $$ in the filename).
Since .zlogout runs after history is saved, if you find any of those
files to be missing, that's a clue that #3 is happening.  You could
also look to see if a ".zsh_history.new" file is left behind.