Installing enhanced bash autocompletion on macOS (aka restoring command line mojo)

Background – Why bother?

Back around 1996, the Web wasn’t a thing, Google didn’t exist and the Speech Research group at Swansea University, where I was, were mostly working on small, shared Unix systems using Dumb Terminals.

Your new shiny, circa mid-1990’s

For us, word processing meant LaTeX, graphics was Xfig (if we could get on the workstation), or Gnuplot, and we used vi for editing anything and everything (and were grateful we had enough CPU not to have to use the evil ex on regular basis again). It were grim, except of course it wasn’t 🙂.

When we were working, a big chunk of our time was spent interacting with our Unix operating systems, for which we used the C shell.

C shell had been chosen by most Unix OS’s as that time as the default shell because it was allegedly more “user friendly” than the Bourne shell. It wasn’t punched cards, but I would have hated to have to use the alternative Bourne shell, as I don’t think most people even back then would have described C shell as “user-friendly”.

For example, in the C shell if you typed:

  • Part of a command in the terminal and hit Tab hoping for autocomplete, tough, you would just get a Tab character.
  • The wrong command line, and then tried using the cursor keys to correct the broken part, no joy, it usually messed up the screen.
  • The wrong command line and hit enter, never mind, scroll back in your history using the cursor to find and then fix it, except, nope, out of luck on that scroll back one as well

It’s true, if you were really persistent you could edit your previous command lines, but you needed to know all sorts of arcana about using ‘!’s. No problem, you might say; just open a browser, find the page using Google, copy the instruction … except, crap, out of luck on that one, no Google and indeed no browser for us at the time.

Eventually, we heard about the Tcsh shell (escapes me how, might have been newsgroups, contact in the computer society or possibly even a divine entity deciding to take mercy on us :-/). This was a shell that was “interactive”. It held the promise of autocompletion, being able to use the cursor keys to edit and scroll through your command line history … we were desperate, we had to have it.

So after a quick install of several days spent finding somewhere to download it from, figuring out how to build it and then persuading our local system admin to install it, changing shells etc, usability did indeed take a massive step forward.

Obviously, the command line editing deficiencies got fixed (except on HP-UX, which we could never quite make work), but rather unexpectedly, and for my money at least as importantly, the autocompletion turned out to be a great way for exploring and learning about the available tools on your system.

Type any letter, hit the Tab autocomplete key, and the Tcsh shell would show you all of your options. Without it, and the poking around it enabled, how else would I have discovered such gems as Fortune and Xeyes, or more importantly, but perhaps less prosaically, dramatically reduced the quantity of my typo’s and increased the reuse of my previous command invocations.

Fast forward to present day.

No matter how useful some of them are, the trend seems to have been for modern command line tools to have largely abandoned the traditional Unix philosophy. Instead of small, tightly focused, modular tools that interoperate together well. We have commands that have grown a plethora of sub-commands, with each of these having its own set of occasionally inconsistent, and invariably hard to remember, sub-options and arguments.

Take the massively popular Git Linux kernel patching and distributed version control system. This is what the somewhat overwhelming list of Git of sub-commands looks like for me:

Git and its sub-commands … what happened when a kernel developer created a command line interface for his tool.

 

By default, on macOS and on most Linux distributions today you get the Bourne again shell, or Bash shell as it is more commonly known. On macOS, this has basic autocompletion enabled out of the box, but its utility is extremely limited when dealing with these new monolithic tools.

For example with Git hitting the Tab autocomplete key after typing “gi” gets me “git” … yay, way to go, big win, not. Beyond adding the “t”, it is unable to offer any assistance, or to help me explore, all of the rest of the stuff from the monstrous list that I might need to type to make a notoriously difficult tool do something useful.

Git is far from the only tool with this problem, and to be fair it’s not quite back to the bad old days of the C shell (we have Google and browser tabs now).

Whatever else you might think of it, the move from the old Unix command pattern that was:

command options arguments

to the use of:

command options sub-command sub-options sub-arguments

ends up chewing up screen estate with open man pages and providing a far sterner test of typing skills. Generally I think it is also reasonable to say that it can end up diverting mental horsepower from whatever it is that you are actually trying to achieve in the first place.

So my point is this. The default Bash shell configuration on macOS is not much use with many modern command line tools (I don’t know why Apple doesn’t address the problem) and I think it is worthwhile fixing this.  It’s worth doing because it saves typing, helps learning about the system you are working with and generally makes for a happier, more productive life (maybe).

Adding enhanced Bash autocompletions

Pre-requisite- basic familiarity with macOS’s Terminal command line.

It’s probably obvious, but just in case, most of this is unlikely to make much, or any sense, without a good basic understanding of working with, and using the Unix/macOS command line. There are many introductions to this topic such as this, this and this if you need a place to start.

Pre-requisite – need Homebrew

The simplest way I have found to get this fixed is to use the rather excellent Homebrew package manager for macOS to install the missing dependencies and configuration files.

They have simple instructions on their website for how to install Homebrew. However, and it’s been a while since I did this, but from memory, you may run into problems if you do not have Apple’s Xcode developer tools installed and have not run them long enough to have clicked through Apple’s EULA.

The download for Xcode is in the App Store (FWIW, downloading and installing this is likely to be the longest part of the entire process).

There’s more on Homebrew and fixing the Xcode install problem over here, whilst there’s a nice short overview of working with it on an ongoing basis in this blog post.

Once Homebrew is installed you should check it is working in your terminal by running

brew search

Which should reward you with a large list of software that you can install (if you want to test further you could try brew install fortune, or maybe brew install cowsay).

Enhanced bash autocompletion overview

Enhanced Bash autocompletion really comes as two parts. The first is an extension to the default Bash shell configuration and the second are the sets of per program configuration files.

At a high level the configuration files tell the Bash extension what a program’s subcommands and their options are, and how to query the system for relevant dynamic information. For example the process handling kill command takes static arguments such as the names of signals, HUP, STOP, CONT etc, as well as the dynamic numeric process identifiers PID, PPID, PGID etc for what is actually running on the system (that you might subsequently then want to send the signals to).

The actual delivery of the configuration files can be a bit messy, mainly because as usual everyone has a standard, just not the same as everyone else’s. Some of the configuration files are independently generated and distributed, whilst others are bundled, or can be generated, from the programs to which they apply.

For completeness, we’ll cover both cases.

Installing and enabling the autocompletion extension

Bash autocompletion requires the installation of the Bash autocompletion extension.

To do this, execute

brew install bash-completion

After that completes the extension gets picked up by any Bash shell that is being started. To do this to you will need a snippet similar to the following, present in your ~/.bash_profile (brew prints this out at the end of the install, but it is easy to miss):

if [ -f $(brew --prefix)/etc/bash_completion ]; then
. $(brew --prefix)/etc/bash_completion
fi

NB: It is easy to fix, but be careful when editing this file as it can break your Terminal sessions.

This will then enable Bash autocompletion for any executable that satisfies all of the following:

  1. Has a suitable autocompletion installed either via the default $(brew --prefix)/etc/bash_completion or for any additional configurations that it finds in the directory $(brew --prefix)/etc/bash_completion.d
  2. Homebrew finds the program the configuration refers to has been installed under the brew installation location brew --prefix i.e. it doesn’t work for most of the system default programs such as those installed in /bin, /sbin, /usr/bin, /usr/sbin etc.
  3. The Bash shell’s autocompletion process has been instantiated since the configuration was installed (You can manually trigger this by either opening another terminal or by re-loading the Bash profile, i.e. . \~/.bash_profile

All being well, you should be able to test by typing kill - and hitting the Tab key a couple of times and you should see the (static) signal name completions:

What you should see for kill if enhanced autocompletion is working

You could then pick one of the those, hit Tab again and get the relevant dynamic numeric identifiers that you could send the signal to if you wanted to go killing your system’s processes (probably not a wise idea unless you know what you’re up to).

Adding additional autocompletions via Homebrew

The easiest way to add extra autocompletions to your new setup is with Homebrew again, as they’ve got a number of additional ones available for install.

You can find them simply enough with brew search completion

Output from brew search completion
List of installable completions from Homebrew’s brew search command

I’m doing quite a bit of work with Docker at the moment, and installed Docker with Homebrew, so

brew install docker-completion

has been a bit of a boon for me, as can be seen by the tick indicating that I have it installed already along with a few others.

Adding built in autocompletions from programs

Some programs come with the ability to generate or supply their own autocompletions.

The two that I have met most recently are the kubectl and minikube programs that I have been using for interacting with Kubernetes and managing my local Kubernetes test cluster.

Since I installed minikube with Homebrew, I got minikube’s completions working with:

minikube completion bash > $(brew --prefix)/etc/bash completion.d/minikube

Similarly, I’ve ended up with kubectl in /usr/local/bin, so this is working great for me:

kubectl completion bash > $(brew --prefix)/etc/bash_completion.d/kubectl

Manually adding other autocompletions

Sometimes Homebrew will not have the autocompletion that you want and it may not be convenient to install the binaries under Homebrew’s brew --prefix directory.

A case in point is with Google’s gcloud tool and its completions. Its completions are included in the top level SDK directory that Google provides and you place in whatever location you like. I’ve ended up having its completions installed at:

/usr/local/google-cloud-sdk/completion.bash.inc

It’s probably not sensible to move the autocompletions from there, because if I did, then they will be likely to go out of date when I pick up updates from Google.

The solution is to independently add these types of things to your ~/.bash_profile. So, for instance, my bash profile now contains the following:

GOOGLE_CLOUD_SDK_DIR=/usr/local/google-cloud-sdk

if  [ -f  $GOOGLE_CLOUD_SDK_DIR/completion.bash.inc ]; then source $GOOGLE_CLOUD_SDK_DIR/completion.bash.inc; fi

Not working

If after it is installed it is not working then the following is usually the problem:

  1. It’s not been re-instantiated since it was installed, try starting a new terminal.
  2. The executable that isn’t under brew --prefix. Figuring out where on the PATH something is coming from can be done by running which executable_name .

More info.

The autocompletions are largely written as bash scripts and they look to be eminently hackable if you fancy tweaking or writing your own.

There’s a slightly old, but useful article on this over at Linux Journal here .

Once you’ve done there, you could then have a delve into ($brew --prefix)/etc/bash_completion.d directory or head over to bash completions on Github to see what others have done.

Of course, Bash is not the only option, if you’re adventurous then there are also other options such as the Z shell, the Friendly shell as well … hmm, brew install zsh