Re: [4.0.2 bug] commands not written to history

["Bart Schaefer" <schaefer@xxxxxxxxxxxxxxxxxxxxxxx>]

On Jul 12,  2:43pm, Vincent Lefevre wrote:
}
} [...] the history contained about 100 lines (it was just after I used
} a new history file), and $HISTSIZE is 2000 here.

Was there more than one zsh process involved?

} greux:~> diff $HISTFILE temphistfile
} 112c112
} < : 994936533:0;mail
} ---
} > : 994937129:867;mail

This difference could be explained by HIST_IGNORE_DUPS (which you said
in 3981 that you do have set) if you ran the command "mail" twice in
a row.  The in-memory history gets the most recent start-time, but the
file is not re-written unless the command text differs.  I suppose that
could be considered a bug when EXTENDED_HISTORY is set, but there's no
good fix except to save a duplicate in the file even though there is
no duplicate in memory.  And I hadn't realized that INC_APPEND_HISTORY
causes the elapsed-time to be lost from the entries, but that makes
sense because they're written out before the command has finished.

} 116,120c116,119
} < : 994939941:0;m $HISTFILE
} < : 994939999:0;mail
} < : 994940004:0;m $HISTFILE
} < : 994940081:0;mail
} < : 994940743:0;rem
} ---
} > : 994939966:13;m $HISTFILE
} > : 994939999:2;mail
} > : 994940004:8;m $HISTFILE
} > : 994940743:308;rem
} 122d120
} < : 994941319:0;diff $HISTFILE temphistfile
} 
} Is it normal?
} 
} It seems that a "mail" command has disappeared from the history file.

There are more commands in $HISTFILE than in temphistfile, which means
that the command was written to $HISTFILE but is not present in memory.
That's extremely odd unless $HISTFILE was written by two or more shells
(each using INC_APPEND_HISTORY) or unless you have one of the IGNORE_ALL
options set (which you've previously said that you don't).  Hence my very
first question.

I'm curious about lines 113-115.  They have zero elapsed times in both
$HISTFILE and temphisfile (otherwise they'd have showed up as a diff).
So either those were very short commands (executed in less than 1 sec.)
or there are two shells involved and you did the diff in the one that was
started later (so it read part of its history from the first shell).

It's a little bit odd that the start times for the first "m $HISTORY"
commands are different as recorded in $HISTFILE and temphistfile.  Could
be another ignore-dups thing.

I just noticed looking at the history code that using INC_APPEND_HISTORY
acts like SHARE_HISTORY each time $HISTFILE exceeds $[SAVEHIST*1.2]
entries.  That is, the history file is re-read and then written in order
to truncate it back down to $SAVEHIST entries, so any interleaved
entries from multiple simultaneous shells will become interleaved in the
shell that performed the truncation, but remain independent in all the
other shells.  There's no way to predict how often this will happen, or
to which shell, though it will tend to happen in the shell where you're
working most actively.

That also means that if you have HIST_IGNORE_DUPS (ignore duplicates only
if they occur adjacent to one another), the interleaving could cause two
entries that were not initially ignored to become adjacent and thus cause
one of them to be ignored when the file is re-read.  I think.

But neither of those things should have been going on in the example
excerpted above (the file should never yet have been re-read/written).

Anyway, one other thing to remember is that INC_APPEND_HISTORY writes to
$HISTFILE as soon as you press return (or any `accept' binding) for a 
command; it doesn't wait for the command to finish.

} > } It seems to be due to the fact that the time stamps are not in the
} > } increasing order.
} 
} > Can you show an example of this, please?
} 
} This can be done by manually copying commands to the history file
} [... or] because host lepois wasn't synchronized

Aha.  Well, that *shouldn't* make a difference.  The history ring doesn't
actually pay any attention to the timestamps.

-- 
Bart Schaefer                                 Brass Lantern Enterprises
http://www.well.com/user/barts              http://www.brasslantern.com

Zsh: http://www.zsh.org | PHPerl Project: http://phperl.sourceforge.net