Infinite Bash History (& More)

…or indefinite Bash history, I suppose, because infinite history will take an eternity to build up, and it won’t fit on your hard drive. Semantics aside, here’s how to set things up such that

• old commands won’t ever be purged from the history,
• histories from different sessions are merged (and histories from not-properly-exited sessions aren’t lost),
• duplicate commands in direct succession are historized only once (and existing duplicates are skipped when looking back through the history),
• pressing the up/down arrow after typing part of a command performs a history search for entries with that prefix, and
• history entries are timestamped.

This configuration¹ has been working well for me over the past couple of years – feel free to take the bits you like and leave the bits you don’t, or even to build² on top of it.

To infinity and beyond!

Depending on your operating system, Bash history is limited to something like 500 or 1000 commands by default. Back when computers were orders of magnitude slower and disk space was scarce, that was a sensible trade-off between recall and bloat – today, there’s really no reason (embedded systems aside) to impose any limits here.

The following snippet removes the default limit, yielding an essentially infinite history.

# Read this number of lines into history buffer on startup.
export HISTSIZE=1000000

# `HISTFILESIZE` is usually set *after* bash reads the history file (which is
# done after reading any configs like `.bashrc`). If it is unset at this point,
# it is set to the same value as `HISTSIZE`. Therefore we must set it to `NIL`,
# in which case it isn't "unset", but doesn't have a value either, enabling us
# to keep an unlimited history.
export HISTFILESIZE=""

You may need to place this snippet both in ~/.bash_profile (executed for login shells) and ~/.bashrc (for interactive non-login shells) – both kinds of shells write to ~/.bash_history, so setting these environment variables in only one³ of them will still lead to history truncation whenever an instance of the other kind of shell runs. (Note that when mentioning .bashrc below, I’m really referring to both of them.)

Data loss? No thank you!

Infinite history is nice, but it doesn’t prevent accidental data loss: By default, Bash reads the history file at the beginning of the session, appends any executed commands to its internal buffer during the session, then writes the buffer contents to the file when then session is closed. Two drawbacks:

1. When running two or more concurrent sessions, only the commands from the one that terminates last will be preserved (i.e., your run-of-the-mill race condition).
2. If a session exits improperly for whatever⁴ reason, the commands executed during that session are toast as they haven’t been written to ~/.bash_history.

Here’s a sufficiently-commented fix (plus some nice-to-have bits), ready for pasting into your .bashrc:

# Append to the history file instead of overwriting it, thus allowing *merging*
# of session histories.
shopt -s histappend

# Append to history file *immediately* (and not only when exiting the shell).
PROMPT_COMMAND="$PROMPT_COMMAND; history -a"

# Alternate version: Also *load* from history immediately to pick up commands
# recently executed in other, concurrent sessions - but that sort of mixes the
# history of different sessions *within* those still-active sessions, which
# confuses me more than it helps. Your mileage may vary!
#PROMPT_COMMAND="$PROMPT_COMMAND; history -a; history -n"

# Don't put duplicate lines in the history, ignore same successive entries, and
# ignore lines that start with a space.
export HISTCONTROL=ignoreboth

# Combine multiline commands in history (command-based history instead of
# line-based history). Enabled by default on many systems, but essential, so
# worth pointing out.
shopt -s cmdhist

Note that the shopt command can be used to enable or disable a bunch of useful Bash features – take a look at the documentation, you’ll likely find something relevant to how you like to work.

Improved history navigation

If you’re anything like me, you often find yourself looking for parts of previously-executed commands to reuse. Instead of manually searching through the history or using the built-in ctrl R reverse-i-search⁵, wouldn’t it be neat if you could simply start typing and then press the ↑ arrow to page through previous commands that begin with the text you’ve just entered?

To give an example, I sometimes download YouTube videos in a specific format, but usually can’t remember the relevant yt-dlp invocation. So I type yt-dlp -f, then press ↑ a couple of times until I find the most recent command using the intended format, say yt-dlp -f 137+140 https://www.youtube.com/watch?v=OHwOrNYlDtQ, which I can then modify. Another common use case for me is recalling past FFmpeg incantations.

The following snippet sets that up for you – again, place it somewhere in your .bashrc.

# Instead of just walking the history, perform a *prefix search* on up and down
# arrow press.
bind '"\e[B": history-search-forward'
bind '"\e[A": history-search-backward'

Adding timestamps

Properly keeping track of what you’ve executed in the past is cool and all, but when you’ve executed a certain command can come in rather handy at times. Which is why Bash supports adding timestamps to history entries! Adding the following line to your .bashrc enables this feature and determines how timestamps are displayed.

# Store timestamps in `~/.bash_history` and render them human-readably in the
# output of the `history` command.
export HISTTIMEFORMAT="%F %T  "

Now, after starting a new session or running source ~/.bashrc, each command added to ~/.bash_history will be preceded by its UNIX timestamp (i.e., the number of seconds since January 1, 1970 in UTC) as a comment:

...
#1650261639
exadserve
#1650268302
history | tail -n 100
#1650268308
history

In the output of the history command, these timestamps are displayed inline, but converted to a human-readable form according to the HISTTIMEFORMAT environment variable:

...
64963  2022-04-18 08:00:39  exadserve
64964  2022-04-18 09:51:42  history | tail -n 100
64965  2022-04-18 09:51:48  history

You can find my complete .bashrc on GitHub – there’s more to it than just history management; it contains a collection of handy aliases and functions.