from Hacker News

Techniques I use to create a great user experience for shell scripts

by hundredwatt on 9/11/24, 4:29 PM with 275 comments

by haileys on 9/13/24, 11:14 PM
Don't output ANSI colour codes directly - your output could redirect to a file, or perhaps the user simply prefers no colour. Use tput instead, and add a little snippet like this to the top of your script:
```
    command -v tput &>/dev/null && [ -t 1 ] && [ -z "${NO_COLOR:-}" ] || tput() { true; }
```
This checks that the tput command exists (using the bash 'command' builtin rather than which(1) - surprisingly, which can't always be relied upon to be installed even on modern GNU/Linux systems), that stdout is a tty, and that the NO_COLOR env var is not set. If any of these conditions are false, a no-op tput function is defined.
This little snippet of setup lets you sprinkle tput invocations through your script knowing that it's going to do the right thing in any situation.
by Arch-TK on 9/14/24, 11:31 AM
This reads like what I've named as "consultantware" which is a type of software developed by security consultants who are eager to write helpful utilities but have no idea about the standards for how command line software behaves on Linux.
It ticks so many boxes:
* Printing non-output information to stdout (usage information is not normal program output, use stderr instead)
* Using copious amounts of colours everywhere to draw attention to error messages.
* ... Because you've flooded my screen with even larger amount of irrelevant noise which I don't care about (what is being ran).
* Coming up with a completely custom and never before seen way of describing the necessary options and arguments for a program.
* Trying to auto-detect the operating system instead of just documenting the non-standard dependencies and providing a way to override them (inevitably extremely fragile and makes the end-user experience worse). If you are going to implement automatic fallbacks, at least provide a warning to the end user.
* ... All because you've tried to implement a "helpful" (but unnecessary) feature of a timeout which the person using your script could have handled themselves instead.
* pipefail when nothing is being piped (pipefail is not a "fix" it is an option, whether it is appropriate is dependant on the pipeline, it's not something you should be blanket applying to your codebase)
* Spamming output in the current directory without me specifying where you should put it or expecting it to even happen.
* Using set -e without understanding how it works (and where it doesn't work).
by sgarland on 9/13/24, 11:20 PM
Nowhere in this list did I see “use shellcheck.”
On the scale of care, “the script can blow up in surprising ways” severely outweighs “error messages are in red.” Also, as someone else pointed out, what if I’m redirecting to a file?
by gorgoiler on 9/14/24, 4:36 AM
It is impossible to write a safe shell script that does automatic error checking while using the features the language claims are available to you.
Here’s a script that uses real language things like a function and error checking, but which also prints “oh no”:
```
  set -e

  f() {
    false
    echo oh
  }

  if f
  then
    echo no
  fi
```
set -e is off when your function is called as a predicate. That’s such a letdown from expected- to actual-behavior that I threw it in the bin as a programming language. The only remedy is for each function to be its own script. Great!
In terms of sh enlightenment, one of the steps before getting to the above is realizing that every time you use “;” you are using a technique to jam a multi-line expression onto a single line. It starts to feel incongruous to mix single line and multi line syntax:
```
  # weird
  if foo; then
    bar
  fi

  # ahah
  if foo
  then
    bar
  fi
```
Writing long scripts without semicolons felt refreshing, like I was using the syntax in the way that nature intended.
Shell scripting has its place. Command invocation with sh along with C functions is the de-facto API in Linux. Shell scripts need to fail fast and hard though and leave it up to the caller (either a different language, or another shell script) to figure out how to handle errors.
by Yasuraka on 9/14/24, 4:53 AM
Here's a script that left an impression on me the first time I saw it:
https://github.com/containerd/nerdctl/blob/main/extras/rootl...
I have since copied this pattern for many scripts: logging functions, grouping all global vars and constants at the top and creating subcommands using shift.
by koolba on 9/13/24, 11:32 PM
```
    if [ "$(uname -s)" == "Linux” ]; then 
       stuff-goes-here
    else # Assume MacOS 
```
While probably true for most folks, that’s hardly what I’d call great for everybody not on Linux or a Mac.
by archargelod on 9/14/24, 1:06 PM
One of my favorite techniques for shell scripts, not mentioned in the article:
For rarely run scripts, consider checking if required flags are missing and query for user input, for example:
```
  [[ -z "$filename" ]] && printf "Enter filename to edit: " && read filename
```
Power users already know to always do `-h / --help` first, but this way even people that are less familiar with command line can use your tool.
if that's a script that's run very rarely or once, entering the fields sequentially could also save time, compared to common `try to remember flags -> error -> check help -> success` flow.
by xyzzy4747 on 9/13/24, 11:25 PM
Not trying to offend anyone here but I think shell scripts are the wrong solution for anything over ~50 lines of code.
Use a better programming language. Go, Typescript, Rust, Python, and even Perl come to mind.
by 0xbadcafebee on 9/14/24, 4:30 PM
If you want a great script user experience, I highly recommend avoiding the use of pipefail. It causes your script to die unexpectedly with no output. You can add traps and error handlers and try to dig out of PIPESTATUS the offending failed intermediate pipe just to tell the user why the program is exiting unexpectedly, but you can't resume code execution from where the exception happened. You're also now writing a complicated ass program that should probably be in a more complete language.
Instead, just check $? and whether a pipe's output has returned anything at all ([ -z "$FOO" ]) or if it looks similar to what you expect. This is good enough for 99% of scripts and allows you to fail gracefully or even just keep going despite the error (which is good enough for 99.99% of cases). You can also still check intermediate pipe return status from PIPESTATUS and handle those errors gracefully too.
by TeeMassive on 9/14/24, 1:47 AM
I'd add, in each my Bash scripts I add this line to get the script's current directory:
SCRIPT_DIR=$( cd -- "$( dirname -- "${BASH_SOURCE[0]}" )" &> /dev/null && pwd )
This is based on this SA's answer: https://stackoverflow.com/questions/59895/how-do-i-get-the-d...
I never got why Bash doesn't have a reliable "this file's path" feature and why people always take the current working directory for granted!
by jiggawatts on 9/13/24, 10:52 PM
Every time I see a “good” bash script it reminds me of how incredibly primitive every shell is other than PowerShell.
Validating parameters - a built in declarative feature! E.g.: ValidateNotNullOrEmpty.
Showing progress — also built in, and doesn’t pollute the output stream so you can process returned text AND see progress at the same time. (Write-Progress)
Error handling — Try { } Catch { } Finally { } works just like with proper programming languages.
Platform specific — PowerShell doesn’t rely on a huge collection of non-standard CLI tools for essential functionality. It has built-in portable commands for sorting, filtering, format conversions, and many more. Works the same on Linux and Windows.
Etc…
PS: Another super power that bash users aren’t even aware they’re missing out on is that PowerShell can be embedded into a process as a library (not an external process!!) and used to build an entire GUI that just wraps the CLI commands. This works because the inputs and outputs are strongly typed objects so you can bind UI controls to them trivially. It can also define custom virtual file systems with arbitrary capabilities so you can bind tree navigation controls to your services or whatever. You can “cd” into IIS, Exchange, and SQL and navigate them like they’re a drive. Try that with bash!
by rednafi on 9/14/24, 11:12 AM
I ask LLMs to modify the shell script to strictly follow Google’s Bash scripting guidelines[^1]. It adds niceties like `set -euo pipefail`, uses `[[…]]` instead of `[…]` in conditionals, and fences all but numeric variables with curly braces. Works great.
[^1]: https://google.github.io/styleguide/shellguide.html
by _def on 9/14/24, 9:50 AM
> This matches the output format of Bash's builtin set -x tracing, but gives the script author more granular control of what is printed.
I get and love the idea but I'd consider this implementation an anti-pattern. If the output mimics set -x but isn't doing what that is doing, it can mislead users of the script.
by ndsipa_pomu on 9/14/24, 10:10 AM
I can highly recommend using bash3boilerplate (https://github.com/kvz/bash3boilerplate) if you're writing BASH scripts and don't care about them running on systems that don't use BASH.
It provides logging facilities with colour usage for the terminal (not for redirecting out to a file) and also decent command line parsing. It uses a great idea to specify the calling parameters in the help/usage information, so it's quick and easy to use and ensures that you have meaningful information about what parameters the script accepts.
Also, please don't write shell scripts without running them through ShellCheck. The shell has so many footguns that can be avoided by correctly following its recommendations.
by emmelaich on 9/14/24, 8:56 AM
Tiny nitpick - usage errors are conventionally 'exit 2' not 'exit 1'
by bfung on 9/13/24, 10:21 PM
Only one that’s shell specific is 4. The rest can be applied any code written. Good work!
by denvaar on 9/13/24, 10:08 PM
I'd add that if you're going to use color, then you should do the appropriate checks for determining if STDOUT isatty
by anthk on 9/14/24, 9:19 AM
A tip:
```
        sh -x $SCRIPT
```
shows a debugging trace on the script in a verbose way, it's unvaluable on errors.
You can use it as a shebang too:
```
         #!/bin/sh -x
```

by jojo_ on 9/14/24, 3:42 AM

Few months ago, I wrote a bash script for an open-source project.

I created a small awk util that I used throughout the script to style the output. I found it very convenient. I wonder if something similar already exists.

Some screenshots in the PR: https://github.com/ricomariani/CG-SQL-author/pull/18

Let me know guys if you like it. Any comments appreciated.

    function theme() {
        ! $IS_TTY && cat || awk '

    /^([[:space:]]*)SUCCESS:/   { sub("SUCCESS:", " \033[1;32m&"); print; printf "\033[0m"; next }
    /^([[:space:]]*)ERROR:/     { sub("ERROR:", " \033[1;31m&"); print; printf "\033[0m"; next }

    /^        / { print; next }
    /^    /     { print "\033[1m" $0 "\033[0m"; next }
    /^./        { print "\033[4m" $0 "\033[0m"; next }
                { print }

    END { printf "\033[0;0m" }'
    }

Go to source: https://github.com/ricomariani/CG-SQL-author/blob/main/playg...

Example usage:

    exit_with_help_message() {
        local exit_code=$1

        cat <<EOF | theme
    CQL Playground

    Sub-commands:
        help
            Show this help message
        hello
            Onboarding checklist — Get ready to use the playground
        build-cql-compiler
            Rebuild the CQL compiler

Go to source: https://github.com/ricomariani/CG-SQL-author/blob/main/playg...

        cat <<EOF | theme
    CQL Playground — Onboarding checklist

    Required Dependencies
        The CQL compiler
            $($cql_compiler_ready && \
                echo "SUCCESS: The CQL compiler is ready ($CQL)" || \
                echo "ERROR: The CQL compiler was not found. Build it with: $CLI_NAME build-cql-compiler"
            )

Go to source: https://github.com/ricomariani/CG-SQL-author/blob/main/playg...

by fragmede on 9/13/24, 10:31 PM
Definitely don't check that a variable is non-empty before running
```
    rm -rf ${VAR}/*
```
That's typically a great experience for shell scripts!
by dvrp on 9/14/24, 1:03 AM
These are all about passive experiences (which are great don't get me wrong!), but I think you can do better. It's the same phenomenon DHH talked about in the Rails doctrine when he said to "Optimize for programmer happiness".
The python excerpt is my favorite example:
```
$ irb
irb(main):001:0> exit
$ irb
irb(main):001:0> quit
$ python
>>> exit
Use exit() or Ctrl-D (i.e. EOF) to exit
```
<quote> Ruby accepts both exit and quit to accommodate the programmer’s obvious desire to quit its interactive console. Python, on the other hand, pedantically instructs the programmer how to properly do what’s requested, even though it obviously knows what is meant (since it’s displaying the error message). That’s a pretty clear-cut, albeit small, example of [Principle of Least Surprise]. </quote>
by pmarreck on 9/14/24, 3:04 AM
Regarding point 1, you should `exit 2` on bad usage, not 1, because it is widely considered that error code 2 is a USAGE error.
by latexr on 9/14/24, 11:38 AM
> if [ -z "$1" ]
I also recommend you catch if the argument is `-h` or `--help`. A careful user won’t just run a script with no arguments in the hopes it does nothing but print the help.¹
```
  if [[ "${1}" =~ ^(-h|--help)$ ]]
```
Strictly speaking, your first command should indeed `exit 1`, but that request for help should `exit 0`.
¹ For that reason, I never make a script which runs without an argument. Except if it only prints information without doing anything destructive or that the user might want to undo. Everything else must be called with an argument, even if a dummy one, to ensure intentionality.
by calmbonsai on 9/14/24, 10:13 PM
Maybe in the late ‘90s it may have been appropriate to use shell for this (I used Perl for this back then) sort of TUI, but now it’s wrong-headed to use shell for anything aside from bootstrapping into an appropriately dedicated set of TUI libraries such as Python, Ruby, or hell just…anything with proper functions, deps checks, and error-handling.
by baby on 9/14/24, 3:53 PM
Let's normalize using python instead of bash
by mkmk on 9/11/24, 4:38 PM
I don’t remember where I got it, but I have a simple implementation of a command-line spinner that I use keyboard shortcuts to add to most scripts. Has been a huge quality of life improvement but I wish I could just as seamlessly drop in a progress bar (of course, knowing how far along you are is more complex than knowing you’re still chugging along).
by account42 on 9/16/24, 3:17 PM
> Strategic Error Handling with "set -e" and "set +e"
I think appending an explicit || true for commands that are ok to fail makes more sense. Having state you need to keep track of just makes things less readable.
by watmough on 9/13/24, 10:54 PM
Good stuff.
One rule I like, is to ensure that, as well as validation, all validated information is dumped in a convenient format prior to running the rest of the script.
This is super helpful, assuming that some downstream process will need pathnames, or some other detail of the process just executed.
by markus_zhang on 9/13/24, 11:11 PM
I was so frustrated by having to enter a lot of information for every new git project (I use a new VM for each project) so I wrote a shell script that automates everything for me.
I'll probably also combine a few git commands for every commit and push.
by teo_zero on 9/14/24, 6:44 AM
```
  if [ -x "$(command -v gtimeout)" ]; then
```
Interesting way to check if a command is installed. How is it better than the simpler and more common "if command...; then"?
by artursapek on 9/14/24, 3:40 PM
This post and comment section are a perfect encapsulation of why I'll just write a Rust or Go program, not bash, if I want to create a CLI tool that I actually care about.
by pmarreck on 9/14/24, 3:08 AM
https://github.com/charmbracelet/glow is pretty nice for stylized TUI output
by teo_zero on 9/14/24, 6:11 AM
In the 4th section, is there a reason why set +e is inside the loop while set -e is outside, or is it just an error?
by Rzor on 9/13/24, 10:27 PM
Nicely done. I love everything about this.
by Myrmornis on 9/14/24, 3:58 AM
In the first example, the error messages should be going to stderr.
by worik on 9/13/24, 11:48 PM
I liked the commenting style
by thangalin on 9/14/24, 8:00 AM
The first four parts of my Typesetting Markdown blog describes improving the user-friendliness of bash scripts. In particular, you can use bash to define a reusable script that allows isolating software dependencies, command-line arguments, and parsing.
https://dave.autonoma.ca/blog/2019/05/22/typesetting-markdow...
In effect, create a list of dependencies and arguments:
```
    #!/usr/bin/env bash
    source $HOME/bin/build-template

    DEPENDENCIES=(
      "gradle,https://gradle.org"
      "warp-packer,https://github.com/Reisz/warp/releases"
      "linux-x64.warp-packer,https://github.com/dgiagio/warp/releases"
      "osslsigncode,https://www.winehq.org"
    )

    ARGUMENTS+=(
      "a,arch,Target operating system architecture (amd64)"
      "o,os,Target operating system (linux, windows, macos)"
      "u,update,Java update version number (${ARG_JAVA_UPDATE})"
      "v,version,Full Java version (${ARG_JAVA_VERSION})"
    )
```
The build-template can then be reused to enhance other shell scripts. Note how by defining the command-line arguments as data you can provide a general solution to printing usage information:
https://gitlab.com/DaveJarvis/KeenWrite/-/blob/main/scripts/...
Further, the same command-line arguments list can be used to parse the options:
https://gitlab.com/DaveJarvis/KeenWrite/-/blob/main/scripts/...
If you want further generalization, it's possible to have the template parse the command-line arguments automatically for any particular script. Tweak the arguments list slightly by prefixing the name of the variable to assign to the option value provided on the CLI:
```
    ARGUMENTS+=(
      "ARG_JAVA_ARCH,a,arch,Target operating system architecture (amd64)"
      "ARG_JAVA_OS,o,os,Target operating system (linux, windows, macos)"
      "ARG_JAVA_UPDATE,u,update,Java update version number (${ARG_JAVA_UPDATE})"
      "ARG_JAVA_VERSION,v,version,Full Java version (${ARG_JAVA_VERSION})"
    )
```
If the command-line options require running different code, it is possible to accommodate that as well, in a reusable solution.
by gjvc on 9/14/24, 6:37 AM
literally nothing here of interest