A proposal for shell libraries

I recently wrote a new shell library called portable-color. Its job is to colorize shell output, but be much more respectful of the environment it's being executed in than just jamming ANSI color codes (now ECMA-48; see "SGR") into stdout.

I'll talk more about portable-color itself on another day. Today, what I wanted to do was figure out how to actually install a shell library on my system.

You see, there's not a standard for this—at least not one I can find. Executing shell scripts in the context of other scripts (more commonly called "sourcing", after the bashism) seems to be a second-class citizen when it comes to the Filesystem Hierarchy Standard or even the venerable hier(7).

I asked the question in the fediverse and got a lot of interesting answers, most promisingly one about libexec. And we could totally use libexec, particularly if we realize that a shell library is less a library and more an execution that changes our environment.

Stack this all with the fact that I prefer to keep my user scripts in the XDG standard .local/bin—I wanted to keep my libraries for user scripts in there too.

So, in order to use portable-color with my new script, I set about making ".local/libexec/portable-color" and adding "portable-color.sh" there. So far, so good, so well-organized.

When I set out to actually load it, though, things got a little more complicated. I thought "let's make it an environment variable", which meant I'd be inventing one. I also wanted my scripts to run even if the environment variable wasn't set, so there should be fallback behavior.

All this and I wanted to keep the "load portable-color" invocation as simple as possible.

I did some digging to make sure I wasn't gonna be tripping over bashisms when using parameter expansion, and wrote this:

source ${SHELL_LIBEXEC_DIR:-.}/portable-color/portable-color.sh || exit 1

This worked. But I didn't really like it.

First, it meant that every time I wanted to load my library, I needed to get that expansion right. Second, I was restricted to just one directory—searching more would have meant a much more complicated upfront invocation.

So I went digging again. My first discovery was that, in fact, "source" was a bashism! Always learning. I also had some weird idea stuck in my head that the dot built-in was different somehow. Turns out it wasn't. Bash just added "source" to make things easier to read.

But while reading about dot, I noticed two things. One was that it respected PATH, which I did already know. The other interesting detail was it didn't require the execute bit to be set on the script that it found.

So, I drastically simplified. Here's my simple recommendation for how to deal with shell libraries.

1. Install your tools in .local/bin. Don't give them an extension.
2. Install your libraries in .local/bin as well, give them the ".sh" extension, don't set them executable. (If you really want to, you can keep things organized in another directory—say, .local/lib—as long as you add it to your PATH.)
3. Always load libraries with dot.

Loading libraries now looks like this:

. portable-color.sh || exit 1

Much nicer. It also lets me develop my libraries right next to my scripts (although I have to be careful, since it seems shell libraries on my PATH take precedence over one in the current directory.)

Not setting the executable bit means Zsh, at least, won't tab-complete my libraries, even though they're on my PATH.

Finally, as for the no-extension vs. ".sh" extension thing, well, that's just preference, but I think it's a good preference. Interactive commands almost never have extensions. Libraries aren't interactive commands, and don't need a shebang. They're meant to run in the shell that executes them.