Explanation for newbies:
-
Shell is the programming language that you use when you open a terminal on linux or mac os. Well, actually “shell” is a family of languages with many different implementations (bash, dash, ash, zsh, ksh, fish, …)
-
Writing programs in shell (called “shell scripts”) is a harrowing experience because the language is optimized for interactive use at a terminal, not writing extensive applications
-
The two lines in the meme change the shell’s behavior to be slightly less headache-inducing for the programmer:
set -euo pipefailis the short form of the following three commands:set -e: exit on the first command that fails, rather than plowing through ignoring all errorsset -u: treat references to undefined variables as errorsset -o pipefail: If a command piped into another command fails, treat that as an error
export LC_ALL=Ctells other programs to not do weird things depending on locale. For example, it forcesseqto output numbers with a period as the decimal separator, even on systems where coma is the default decimal separator (russian, dutch, etc.).
-
The title text references “posix”, which is a document that standardizes, among other things, what features a shell must have. Posix does not require a shell to implement
pipefail, so if you want your script to run on as many different platforms as possible, then you cannot use that feature.
set -euo pipefailis, in my opinion, an antipattern. This page does a really good job of explaining why. pipefail is occasionally useful, but should be toggled on and off as needed, not left on. IMO, people should just write shell the way they write go, handling every command that could fail individually. it’s easy if you write adiefunction like this:die () { message="$1"; shift return_code="${1:-1}" printf '%s\n' "$message" 1>&2 exit "$return_code" } # we should exit if, say, cd fails cd /tmp || die "Failed to cd /tmp while attempting to scrozzle foo $foo" # downloading something? handle the error. Don't like ternary syntax? use if if ! wget https://someheinousbullshit.com/"$foo"; then die "failed to get unscrozzled foo $foo" fiIt only takes a little bit of extra effort to handle the errors individually, and you get much more reliable shell scripts. To replace -u, just use shellcheck with your editor when writing scripts. I’d also highly recommend https://mywiki.wooledge.org as a resource for all things POSIX shell or Bash.
After tens of thousands of bash lines written, I have to disagree. The article seems to argue against use of -e due to unpredictable behavior; while that might be true, I’ve found having it in my scripts is more helpful than not.
Bash is clunky. -euo pipefail is not a silver bullet but it does improve the reliability of most scripts. Expecting the writer to check the result of each command is both unrealistic and creates a lot of noise.
When using this error handling pattern, most lines aren’t even for handling them, they’re just there to bubble it up to the caller. That is a distraction when reading a piece of code, and a nuisense when writing it.
For the few times that I actually want to handle the error (not just pass it up), I’ll do the “or” check. But if the script should just fail, -e will do just fine.
This is why I made the reference to Go. I honestly hate Go, I think exceptions are great and very ergonomic and I wish that language had not become so popular. However, a whole shitload of people apparently disagree, hence the popularity of Go and the acceptance of its (imo) terrible error handling. If developers don’t have a problem with it in Go, I don’t see why they’d have a problem with it in Bash. The error handling is identical to what I posted and the syntax is shockingly similar. You must unpack the return of a func in Go if you’re going to assign, but you’re totally free to just assign an err to
_in Go and be on your way, just like you can ignore errors in Bash. The objectively correct way to write Go is to handle everyerrthat gets returned to you, either by doing something, or passing it up the stack (and possibly wrapping it). It’s a bunch of bubbling up. My scripts end up being that way too. It’s messy, but I’ve found it to be an incredibly reliable strategy. Plus, it’s really easy for me to grep for a log message and get the exact line where I encountered an issue.This is all just my opinion. I think this is one of those things where the best option is to just agree to disagree. I will admit that it irritates me to see blanket statements saying “your script is bad if you don’t set -euo pipefail”, but I’d be totally fine if more people made a measured recommendation like you did. I likely will never use set -e, but if it gets the bills paid for people then that’s fine. I just think people need to be warned of the footguns.
EDIT: my autocorrect really wanted to fuck up this comment for some reason. Apologies if I have a dumb number of typos.
I’ve been meaning to learn how to avoid using pipefail, thanks for the info!
Putting
or die “blah blah”after every line in your script seems much less elegant than op’s solutionThe issue with
set -eis that it’s hideously broken and inconsistent. Let me copy the examples from the wiki I linked.Or, “so you think set -e is OK, huh?”
Exercise 1: why doesn’t this example print anything?
#!/usr/bin/env bash set -e i=0 let i++ echo "i is $i"Exercise 2: why does this one sometimes appear to work? In which versions of bash does it work, and in which versions does it fail?
#!/usr/bin/env bash set -e i=0 ((i++)) echo "i is $i"Exercise 3: why aren’t these two scripts identical?
#!/usr/bin/env bash set -e test -d nosuchdir && echo no dir echo survived#!/usr/bin/env bash set -e f() { test -d nosuchdir && echo no dir; } f echo survivedExercise 4: why aren’t these two scripts identical?
set -e f() { test -d nosuchdir && echo no dir; } f echo survivedset -e f() { if test -d nosuchdir; then echo no dir; fi; } f echo survivedExercise 5: under what conditions will this fail?
set -e read -r foo < configfileAnd now, back to your regularly scheduled comment reply.
set -ewould absolutely be more elegant if it worked in a way that was easy to understand. I would be shouting its praises from my rooftop if it could make Bash into less of a pile of flaming plop. Unfortunately ,set -eis, by necessity, a labyrinthian mess of fucked up hacks.Let me leave you with a allegory about
set -ecopied directly from that same wiki page. It’s too long for me to post it in this comment, so I’ll respond to myself.Woah, that
((i++))triggered a memory I forgot about. I spent hours trying to figure out what fucked up my$?one day.When I finally figured it out: “You’ve got to be kidding me.”
When i fixed with
((++i)): “SERIOUSLY! WTAF Bash!”From https://mywiki.wooledge.org/BashFAQ/105
This is great and thanks for taking the time to enlighten us 😄
No worries! Bash was my first language, and I still unaccountably love it after 15 years. I hate it and say mean things about it, but I’m usually pleased when I get to write some serious Bash.
Exercise 6:
That one was fun to learn.
Even with all the jank and unreliability, I think
set -edoes still have some value as a last resort for preventing unfortunate accidents. As long as you don’t use it for implicit control flow, it usually (exercise 6 notwithstanding) does what it needs to do and fails early when some command unexpectedly returns an error.I personally don’t believe there’s a case for it in the scripts I write, but I’ve spent years building the
|| diehabit to the point where I don’t even think about it as I’m writing. I’ll probably edit my post to be a little less absolute, now that I’m awake and have some caffeine in me.One other benefit I forgot to mention to explicit error handling is that you get to actually log a useful error message. Being able to
rg 'failed to scrozzle foo.* because service y was not available'and immediately find the exact line in the script that failed is so nice. It’s not quite a stack trace with line numbers, but it’s much nicer than what you have with bash by default or with set -e.Yup, and
set -ecan be used as a try/catch in a pinch (but your way is cleaner)I was tempted for years to use it as an occasional try/catch, but learning Go made me realize that exceptions are amazing and I miss them, but that it is possible (but occasionally hideously tedious) to write software without them. Like, I feel like anyone who has written Go competently (i.e. they handle every returned
erron an individual or aggregated basis) should be able to write relatively error-handled shell. There are still the billion other footguns built directly into bash that will destroy hopes and dreams, but handling errors isn’t too bad if you just have a littlediefunction and the determination to use it.That’s well put. I might put that at the start of all of my future comments about
bashin the future.Yep. Bash was my first programming language so I have absolutely stepped on every single one of those goddamn pedblasters. I love it, but I also hate it, and I am still drawn to using it.