So I built glaze; a small command-line tool that lets me describe a tmux workspace once and recreate it on demand. Type glaze up and the sessions, windows and panes I described spring back into existence exactly how I left them.

This has been a slow-burning labour of love for the better part of two years and it’s finally in a state where I feel comfortable letting other people look at it. So let’s talk about why it exists, what else is out there and how this one is different.

It’s just a config file, right?

That was the idea, at least. I just wanted to stop rebuilding the same layouts over and over. But, as is tradition around here, I didn’t want to make it too easy for myself.

There was a second, more selfish motivation. As a platform engineer, there isn’t a day that goes by where I don’t work with Terraform. I’ve always been quietly fascinated by how it parses and validates its configuration. That whole experience of getting a precise, friendly error pointing at the exact line you fat-fingered, rather than a stack trace and a 🖕. I wanted to understand how that machinery actually worked.

I’m a heavy tmux user and I wanted to learn HCL parsing from the inside. These two things lined up a little too perfectly. So glaze profiles aren’t YAML; they’re HCL. The same configuration language Terraform uses and parsed with the same underlying library.

Here’s a basic profile:

session {
  name = "daemon-run"

  window {
    name   = "ice-breaker"
    layout = "main-vertical"

    pane {
      commands = ["nvim ./payloads"]
    }

    pane {
      commands = ["watch -n1 netwatch --target arasaka-mainframe"]
    }
  }
}

Drop that in a file called .glaze, run glaze up next to it and you’re jacked in.

I’m not the first to do this; not even the 3rd… or the 4th.

This is a thoroughly-solved problem and I’d be doing you a disservice if I pretended otherwise. There’s a whole shelf of mature, battle-tested tools that do effectively the same thing:

• tmuxinator is the one most are familiar with. Written in Ruby with YAML profiles. Probably what most people reach for.
• teamocil is also written Ruby; also uses YAML.
• smug is written in Go and uses YAML and is the closest in spirit to glazier if we’re being honest.
• tmuxp is written in good’ole reliable Python and it’ll happily eat YAML or JSON.

If you already use and trust one of these, I’ll be straight with you: there isn’t a compelling reason to switch. Keep using what works. I’m not here to convince anyone to rip out a tool they’re happy with.

But if you’re still reading, here’s what I like about mine.

So what’s actually different?

The profile validates itself!

This is the part I set out to build, so it’s the part I’m fondest of. Because glazier is built on HCL, it inherits Terraform-style diagnostics for free. Mistype a layout, point a starting directory at somewhere that doesn’t exist or forget a required block and you don’t get a vague “something went wrong fuck you”. You get told exactly what and where you messed up:

Error: Invalid layout specified

  on .glaze line 4, in session.window:
   4:     layout = "main-plumbus"

The layout value of "main-plumbus" is not a supported preset
(even-horizontal, even-vertical, main-horizontal, main-vertical,
tiled) nor a valid tmux layout string.

There’s even a glaze format command that rewrites your profile into a canonical style and, with --validate, reports any of these diagnostics without touching tmux at all. Both of these scratch exactly the itch that started the whole project.

Variables, templates and string functions! Oh, my!

A static layout is useful. A templated one is better. Profiles can reference variables and you can feed those in from --var flags or GLAZE_ENV_* environment variables:

session {
  name               = "ops-${region}"
  starting_directory = path.pwd

  window {
    name = upper(region)

    pane {
      commands = ["k9s --context ${region}"]
    }
  }
}

$ glaze up --var region=ap-southeast-2

Look familiar? If you work with Terraform it does!

There’s a handful of built-in string functions too. upper, lower, replace, trimspace, join and friends which act as thin wrappers over the same go-cty standard library Terraform uses. Plus, a couple of freebies like path.pwd and path.base so a profile can adapt to wherever it’s run from.

It doesn’t lie to you about timing.

This is the bit of engineering I’m quietly proudest of, even though nobody will ever see it. When you fire a sequence of commands into a tmux pane, the naive approach is to blast them in with a sleep between each one and hope the previous command finished. That’s flaky and it’s how a few other tools handle it.

Glazier instead serialises commands through tmux’s own wait-for signalling, so each command genuinely waits for the previous one to finish before the next is sent. There are no fixed sleeps and no races. The one exception is the final command in a list, which is sent fire-and-forget. If your last command is a long-running dev server or tail -f, waiting on it would hang up forever.

I learned that last part the hard way.

It can mostly save a session!

Run glaze save inside a live session and it’ll capture the structure back out into a profile: windows, panes, names, layouts, focus and starting directories. The “mostly” is doing some load-bearing work in that sentence and it’s a deliberate choice, not a missing feature.

save will not export your pane commands, environment variables, hooks or tmux options. Why? Because each of those is a footgun:

• Commands would re-execute on the next up. A forgotten rm -rf captured from some pane could ruin your whole day on replay.
• Environment variables can only be read as the entire session environment. This means inherited secrets, tokens and keys getting written into a file you might commit. That’s a big fat no from me, dawg.
• Options read back as effective state, hopelessly tangling up with your tmux.conf and manual tweaks.

So, a saved profile is a scaffold. It gets the geometry right and you add back the commands and config you actually want by hand.

A real profile.

Enough talk. Here’s the profile I actually use when working this website:

session {
  name = replace(path.base, ".", "-")

  envs = {
    HUGO_GITHUB_TOKEN        = "${github}"
    HUGO_ACTIVITY_GRAPH_DEMO = "${demo}"
  }

  window {
    name  = "editor"

    pane {
      commands = ["nvim"]
    }
  }

  window {
    name  = "terminal"
    focus = true

    pane {}
  }

  window {
    name = "server"

    pane {
      commands = ["hugo server --disableFastRender"]
    }
  }

  window {
    name = "git"

    pane {
      commands = ["lazygit"]
    }
  }
}

Running glaze up --var github=*** --var demo=true gives me the following windows:

• My editor of choice; NeoVim.
• A dedicated terminal session.
• The Hugo server along with some HUGO_ specific environment variables; values sourced from the CLI.
• Finally, lazygit to manage and commit my changes.

One thing I should point out is where I define the session name with replace(path.base, ".", "-"). This is a workaround for a small known tmux quirk where it won’t allow . or : within a session name. Since the repository is wilhelm.codes, tmux will automatically assign it as wilhelm_codes. And since Glazier refers to the session by the name we explicitly define, it’ll lose context the moment the session is created with the following useful error:

could not create new session `wilhelm.codes`: session `wilhelm.codes` was created but could not be found afterwards

Definitely a small edge case I should address with the next patch.

Anyways, because the session is just a named thing, the rest of the lifecycle is tidy too. List what’s running with glaze ls:

$ glaze ls
NAME      WINDOWS  PATH
glazier*  2        /home/wilhelm/Development/wilhelm.codes
scratch   1        /tmp

The asterisk marks the session I’m currently attached to. When I’m done, I tear it down by profile:

$ glaze down

down is idempotent and that’s on purpose. Bringing down a session that isn’t running is a no-op, not an error, so it’s safe to drop in scripts without defensive checks.

Some over-engineered bits …

To be frank a tool that shells out to tmux did not strictly need a fuzzed HCL parser, build provenance attestations on its release binaries or a CI pipeline that runs the test suite against multiple Go versions and operating systems.

But, that was never really the point. The point was learning how Terraform’s parser ticks, working out how to drive tmux reliably without sleeps and over-engineering the ever-loving-shit out of an already-solved problem because it was fun. Every constraint I imposed upon myself taught me something new. This is ultimately the only metric I actually care about for a hobby project like this.

Where to get it?

It’s up on GitHub with a shiny MIT license. If you’ve got Go installed:

$ go install github.com/wilhelm-murdoch/glazier/cmd/glaze@latest

Or, grab a prebuilt binary from the releases page. I currently build for Linux and macOS on both amd64 and arm64. Each one ships with a checksum and a signed provenance attestation, because of course it does.

In closing …

I would confidently say Glazier has moved on from “experimental” to “stable”. It works and I use it every single day. But, there are rough edges and there may be any number of unencountered failure modes. The down and ls commands only landed recently and I’m not sure if the latter should remain. I’ve also got a running list of ideas I haven’t talked myself out of yet. Like, something similar to Terraform’s *.tfvars files or defining typed variables.

If you’re already happy with Tmuxinator or Smug, then stick with them. But if the idea of a declarative, self-validating, slightly-too-clever tmux profile appeals to you, or you just want to read some Go that wraps tmux in ways it was probably never meant to be wrapped, I’d love for you to take it for a spin.

I sincerely hope you find glazier as useful as I had fun building it.

So let me finally show you my bits.

What even is a bit?

A bit is the smallest unit of “I made a thing” I’m willing to commit to. It could be a quote that stuck with me, a link worth keeping, the odd YouTube video, or a one-line thought too small to earn its own long-form entry. Each one is a little markdown file in /bits and the only thing it really has to declare is what kind of bit it is:

---
type: quote
date: 2026-06-17
source: "Edsger Dijkstra"
---
"Simplicity is a great virtue but it requires hard work to achieve it."

type is the whole trick that everything else hangs off.

One shape per thought.

The thing I didn’t want was for a pithy quote to render like a bare link or to render like a video. They’re different shapes of thought, so they ought to look different.

I’d already learned this lesson building the changelog, which leans on one tiny partial per event type. So I stole from myself. There’s a single lookup table mapping each type to an icon, an accent colour and a label:

{{ $meta := dict
  "quote" (dict "icon" "quote-right"    "colour" "rose"    "label" "quote")
  "link"  (dict "icon" "arrow-up-right" "colour" "blue"    "label" "link")
  "video" (dict "icon" "youtube"        "colour" "red"     "label" "video")
  "blurb" (dict "icon" "pen-nib"        "colour" "emerald" "label" "blurb")
}}

It then matches a micro-partial per type that decides how the content actually renders. Adding a brand new kind of bit is now “add one row, add one tiny template”, which is about as many steps as I’m willing to tolerate.

Hugo, why are you like this?

My bits started life with two perfectly reasonable front-matter fields: type and url. Hugo took one look and quietly robbed me of both.

It turns out url is a reserved key which Hugo uses to treat as the page’s permalink. So, the moment I put a https://... in there, the build fell over with a stern little message about unsupported protocols. type doesn’t even have the decency to error. It silently moves out of .Params.type and into .Type, which also hijacks the layout lookup. So my .Params.type was resolving to nothing and every single bit was quietly rendering as the fallback.

Turns out hte fix for this was simply to rename url to link and read .Type instead of .Params.type. Half an hour of my life dedicated to two words I didn’t know were already spoken for.

Coding is my passion.

Bits flapping in the wind.

Rather than dumping bits into the same fold-out list as the GitHub noise, I pulled them out as their own first-class timeline rows. The lower-signal changelog chatter stays tucked behind the “N changelog events in between” disclosure. So, the home page now reads in three tiers:

• the loud headline articles
• the medium-volume bits
• the quiet changelog hum

The same interstitial machinery I built last time, but bits just get to stand a little taller than a commit.

The tag that went nowhere.

Bits are deliberately render: never in Hugo. They live inline on the bits page and never get their own URL. Except “no page” also quietly means “no place in the taxonomy”, which means a tag only ever used by a bit has no /tags/... page to point at. If I link it anyway I’ve shipped a 404.

So, the tags only become links if there’s actually somewhere to go:

{{ with site.GetPage (printf "/tags/%s" (urlize .)) }}
  <a href="{{ .RelPermalink }}">#{{ .Data.Term }}</a>
{{ else }}
  <span>#{{ . }}</span>
{{ end }}

A tag with a real page becomes a link and a bit-only tag stays as plain text. No broken links and the nice part is it heals itself the moment a proper article picks up the same tag.

In closing …

The front page has quietly turned into a proper little stream. Long-form when I’ve actually got something to say, bits when I don’t and a year of squircles up top keeping a tally of it all.

Which is the entire point. The less effort it takes to post a half-formed thought, the more likely I am to actually post it. And the more I post, the more those gaps fill in.

And, as always, you can check everything out for yourself in the repo.

I want to fill those gaps in and not with more full-fat rows, but with a small line between each pair of articles. Something like “1 bit and 41 changelog events in between”; a kind of interstitial event timeline for all the work that never quite became a full blog post.

Why can’t I hold all these events?

The trick to merging things that don’t look alike… You’re never going to believe this… is to make them look alike. I already pull my GitHub activity for the changelog, or, at least, the last 100 or so events. So, I reused that, threw the bits in alongside it, and normalised the whole lot into one boring, uniform shape. This data structure doesn’t hold all the data for each event, but just enough to cover the highlights.

{{ $events = $events | append (dict
  "kind"  "commit"
  "date"  (time.AsTime .commit.author.date)
  "title" (index (split .commit.message "\n") 0)
  "url"   .html_url
  "icon"  "code-commit"
  "colour" "green"
  "data"  .)
}}

Every piece of content on this site has at the very last these attributes. So created a unifying model is pretty straight-forward. A kind, a date, a title, a link, and enough leftover metadata to do something creative with later if I feel like it. I made a point of keeping the original object tucked away in data too; a gift to future Wilhelm.

Minding the gaps.

With one big sorted pile of events, the rest is just bookkeeping. Walk the articles newest to oldest, and for each one work out the window between it and the article above it. Then, scoop up every event that falls within the relevant dates:

{{ $upper := $now }}
{{ if gt $i 0 }}{{ $upper = (index $articles (sub $i 1)).Date }}{{ end }}
{{ range $e := $events }}
  {{ if and ($e.date.After $article.Date) (not ($e.date.After $upper)) }}
    {{ $group = $group | append $e }}
  {{ end }}
{{ end }}

The newest article’s “window” runs all the way up to now, so whatever I’ve been doing since the last post shows up at the very top. If there’s an empty gap the event list doesn’t render.

Listing the interstitial events.

A summary is nice, but I wanted to be able to actually see the events without leaving the page. Normally that’s where a pile of JS shows up. But, just as with the activity graph, we’re going to rely on pure HTML and CSS. The <details> element does the entire job for free:

<details class="group">
  <summary>... 41 changelog events in between ...</summary>
  <ul>
    <!-- one little row per event, complete with icon and link -->
  </ul>
</details>

Click the summary, the gap unfolds into a tidy little list of every commit, PR and bit. Each linked off to wherever it lives. The default disclosure triangle gets binned and replaced with a chevron that flips on open, courtesy of Tailwind’s group-open: variant. Still no JS and I’ll remain quietly smug about that.

Saving myself a small headache.

Finally, there’s the the responsive question. This site’s overall design isn’t exactly “mobile first” and viewing it on a small screen strips away a lot of detail already. All these thin dividers and fold-out lists look great on a wide screen and turn into a cramped mess on a phone.

I thought about it for a while and weighed up some clever reflowing. In the end I did the only sensible thing and hid the whole timeline below the sm breakpoint. On a phone you get the clean list of articles, but on a desktop you get the full story between them.

“Sometimes the right amount of effort is zero and the discipline is in admitting it.” Me. Wilhelm. I said that.

That being said, I do need to return and focus on making the mobile experience a bit more palatable.

In closing …

What I like about this is that it turns my worst blogging habit - vanishing for months and then resurfacing in a manic flurry - into something the site can actually narrate. The big rows are the headlines while the quiet lines between them are everything else.

It’s also wired up so I can grow it later without unpicking anything. Right now it’s counts and lists, but tomorrow it could filters, more details, or do something I haven’t thought of yet. All off the same pile of events.

For now, though, the gaps have something interesting in them. Which is a much nicer thing to scroll past than a year of silence.

I wanted something that summed up “Has Wilhelm actually been doing anything lately?” at a glance, without the density of 365 tiny day cells. Fifty-two squircles felt about right. Coarse enough to read across the room, fine enough to still give a picture.

What counts as a “week’s worth of activity”?

The short answer is everything. The changelog already pulls my GitHub activity at build time, so I reuse that exact same data. Then, I throw in the things GitHub doesn’t know about; every type of blog post on this site.

Because the changelog fetching already lives in a tidy little partial, sourcing the everything is just a matter of asking for each one and collecting all the publish dates:

{{ $dates := slice }}
{{ range partial "changelog/fetch.html" (dict "url" $commitsUrl "fixture" $fixture) }}
  {{ $dates = $dates | append (time.AsTime .commit.author.date) }}
{{ end }}
{{ range where site.RegularPages "Section" "blog" }}
  {{ $dates = $dates | append .Date }}
{{ end }}

Nothing crazy going on here. I’m just creating a big list of timestamps from wherever I happen to leave a trail.

Sorting a pile of dates into weekly buckets.

Each of these dates now need to land in one of fifty-two buckets. Working out which week a given date belongs to is just some boring epoch arithmetic; how many whole weeks ago was it?

{{ $ago := int (div (sub now.Unix $d.Unix) 604800) }}
{{ if and (ge $ago 0) (lt $ago 52) }}
  {{ $counts.Add (printf "%d" (sub 51 $ago)) 1 }}
{{ end }}

That 604800 is the number of seconds in a week. And yes, I did have to look that up. Anything older than fifty-two weeks just falls off the back and is quietly ignored.

Painting the squircles.

My favourite part of this is there’s no JS involved. The whole thing is a CSS grid of twenty-six columns, and because each squircle is aspect-square, the rows just work and the squircles stretch to fill whatever container I drop the partial into:

<div class="grid grid-cols-26 gap-1.5">
  <!-- 52 of these green little dudes -->
  <div class="aspect-square rounded-full bg-emerald-400"></div>
</div>

Working out the shade was the fiddly bit. My first attempt simply scaled each week against the busiest one, which sounds sensible right up until you remember my data is basically one enormous week and a whole lot of flat nothing. That single monster week hogged the dark end and squashed everything else into the same pale green. So I did what GitHub does and reached for quartiles instead. You rank the weeks that actually saw some activity, chop them into four groups, and let a week’s colour come from where it lands in the pack rather than from some absolute number. Empty weeks stay a faint emerald-100; the rest climb through four steps of green up to emerald-600. The part I like is that it’s relative to me; a busy week is only busy compared to my other weeks.

The little tooltip that pops up when you hover over a squircle is also pure CSS. A group on the wrapper, a group-hover:opacity-100 on the popup, a transition is all I needed. Each squircle now gives a little scale on hover too, just because it’s nice.

The graph’s painful honesty.

Now on to the more embarrassing part. When I first rendered it with real data, I got one lonely green squircle and fifty-one empty ones. Surely, you’ve noticed it on the front page.

For the curious, this is what it would look like if I didn’t have commitment issues.

At the time I assumed I’d done something wrong. But, to my great shame, I hadn’t. Turns out that when you vanish from your own website for close to 18 months and then cram an entire renovation into a single week, the graph renders exactly that. A long, flat, pale stretch of road ending with one pathetic little green emerald.

There’s an extra little indignity baked into the quartiles, too. They need a spread to rank against, and with exactly one active week there’s nothing to compare it to. So my massive renovation-cramming week doesn’t even get to be properly dark green. It turns up as a polite, middling shade.

I effectively built a little a tool whose entire job is to hold up a mirror to my own lack of consistency and commitment. Very cool! 😬👌

Drop it in anywhere.

This whole thing is a single self-contained partial. It was waaaay easier to build out than I had originally thought. And, thanks to Tailwind I didn’t even have to fall back on any JS! I count that as a bonus.

For the moment, it’ll live on the home page, but I can now just place this anywhere in my Hugo site and it’ll work:

{{ partial "activity-graph.html" . }}

I might even build it out a bit more to support different colour schemes or specificy types of targeted site content. So, the more I write, the more those squircles fill in.

Consider yourself warned, me.

Confession Time

While poking around the build, I discovered something a little embarrassing. My Tailwind setup - the source, the config, and the entire node_modules directory - was living inside Hugo’s static/ folder.

If you know Hugo, you already know where this is going. Everything in static/ gets copied, verbatim, into the final site. Which means I had been cheerfully publishing my whole build toolchain - megabytes of it - to the live site on every single deploy. So, yeah. Production was shipping node_modules. Coding is my passion.

Nobody noticed, nothing broke, and the world kept turning. But it’s the kind of thing that, once you see it, you can’t un-see. To be fair to myself, when I originally put this Hugo site together, I only learned enough to get something shipped.

Letting Hugo do the heavy lifting.

The reason that mess existed in the first place was that I’d wired up Tailwind as a separate, manual build step that spat out a compiled stylesheet for Hugo to pick up. It worked, but it was a second moving part I had to remember existed.

The good news is that recent versions of Hugo can drive Tailwind itself, natively, as part of the normal asset pipeline. Combined with Tailwind v4 - which finally ditches the JavaScript config file in favour of configuring everything in CSS - I got to delete a lot of stuff:

$ git diff --shortstat main...modernize
64 files changed, 1872 insertions(+), 40197 deletions(-)

The whole stylesheet now starts its life as a single entry point:

@import "tailwindcss";
@plugin "@tailwindcss/typography";
@source "hugo_stats.json";

That hugo_stats.json bit is the clever part. Hugo writes out a list of every utility class it actually emits, and Tailwind reads that to decide what to generate. No more pointing Tailwind at my templates and hoping it guesses right.

Then a small partial hands it all off to Hugo to compile, minify and fingerprint:

{{- with (templates.Defer (dict "key" "css")) }}
  {{- with resources.Get "css/main.css" }}
    {{- $opts := dict "minify" (not hugo.IsDevelopment) }}
    {{- with . | css.TailwindCSS $opts }}
      {{- /* ...do very cool things... */ -}}
    {{- end }}
  {{- end }}
{{- end }}

The templates.Defer wrapper is there because the CSS can’t compiled ( transpiled? ) until Hugo has finished rendering every page and knows the full list of classes. So, in a very real way, Hugo solves an annoying 🐔 and 🥚 problem.

This means no more standalone Tailwind config, no committed stylesheet, no node_modules in static/, and no build toolchain leaking onto the live site. Very cool!

So. Many. Deprecations.

Of course, nothing that’s been left alone for a year comes back to life cleanly. Bumping Hugo to the latest release lit up the console like a Christmas tree.

A few of my templates were leaning on things that have since been politely shown the door:

• resources.GetRemote ... .Err for the changelog page - that pattern was removed in favour of a shiny new try keyword.
• .Language.LanguageCode and .Language.LanguageDirection, both deprecated in favour of .Locale and .Direction.
• The _build and cascade._target front matter keys, now just build and cascade.target.

None of it was hard to fix, but it’s a good reminder that “it still builds” and “it builds without complaints” are two very different bars.

A Footgun!

Here’s a fun one. The changelog page pulls in a Github event fixture file during local development by fetching it over http://localhost:1313. Effectively, from the very dev server that’s trying to build the page.

Those of you who are familiar with such things can probably see the problem. Hugo builds the site before it starts listening on that port, so the build sits there waiting for a server that doesn’t exist yet. A deadlock of my own making. The obvious fix was to just read the file off disk instead of asking the network nicely.

Some honourable mentions.

A grab-bag of smaller wins while I had the hood up:

• Fonts are now served as woff2 instead of raw ttf, with font-display: swap so text shows up immediately instead of hanging around invisible. This alone shaved the font payload down by about 60%.
• I also deleted a few MB of fluff that wasn’t being loaded by anything.
• Prettier got a nice version bump and I taught it to sort my Tailwind classes, so I can stop pretending I do that consistently by hand.

Was the juice worth the squeeze?

Aside from a few cosmetic updates here and there, the site should look pretty much the same as it did before. Which is, weirdly, the whole point; all of this work was about the parts you can’t see. I got a leaner build, faster page loads, and a project I can come back to in another year without wincing; famous last words, etc…

For now, the house is clean. Very nice.

If you use Github, you know it quietly records nearly everything you do as a stream of events. Better still, for a public repository, that stream is available over a public, unauthenticated API endpoint through a simple GET request.

So the plan more or less writes itself. We fetch the stream at build time, group the events by day, and let Hugo render them. Hugo even has a tidy little function for pulling in remote resources that works like so:

{{ $url := "https://api.github.com/repos/wilhelm-murdoch/wilhelm.codes/events" }}
{{ with try (resources.GetRemote $url) }}
  {{ $events := .Value | transform.Unmarshal }}
  {{/* ...range over them and render... */}}
{{ end }}

Because this all happens at build time, the visitor never waits on GitHub. By the time the page reaches a browser it’s just static HTML like everything else. The changelog is always current as of the last deploy, and I never have to think about it.

That’s the part that worked. Now for the parts that didn’t.

… Then, production happened.

I’ve been more or less absent from this site for close to a year, but this past week I’ve decided to breathe a bit more life into it. I made a few changes to modernise the stack, pushed my changes up and waited on Cloudflare to do its thing and…

ERROR error building site: ... error calling GetRemote:
failed to fetch remote resource from 'https://api.github.com/...': Forbidden

I was greeted with a 403. Except… I could paste that exact URL into my browser and get a perfectly happy wall of JSON back. So what gives?

Rate limiting, that’s what. GitHub’s unauthenticated API is capped at 60 requests per hour, per IP address. My build doesn’t run on my IP. It runs on a shared build machine alongside who-knows-how-many other people’s deploys, all of them hammering GitHub from the same handful of addresses. By the time my build rolled around, that bucket was bone dry.

Honestly, a 429 instead here would have saved me a bit of investigating, but I digress.

The solution was to created a properly scoped Github PAT. Using an authenticated request gets 5000 requests per hour, so I went about implementing support and handed it to the build as an environment variable, and taught the template to send it along:

{{ $opts := dict }}
{{ with os.Getenv "HUGO_GITHUB_TOKEN" }}
  {{ $opts = dict "headers" (dict "Authorization" (printf "Bearer %s" .)) }}
{{ end }}
{{ $remote := try (resources.GetRemote $url $opts) }}

It’s worth noting that Hugo won’t read just any environment variable. Its security policy only lets os.Getenv see variables prefixed with HUGO_. Name your token GITHUB_TOKEN and you’ll get a confusing fistful of nothing; name it HUGO_GITHUB_TOKEN and it works. Ask me how I know.

Hark! Another Footgun!

With the token in place, the fetch succeeded and the build promptly fell over somewhere new:

error calling len: reflect: call of reflect.Value.Type on zero Value

This never showed up locally, which tells me I should do myself a favour and do a quick local production deployment as a preflight before I push to Github and trigger an automated build. Originally, in development I rendered the changelog from a single large fixture file stored locally on disk. This was before I implemented the PAT so I wouldn’t rate limit myself locally when testing. The live stream, however, contains event shapes my fixture simply didn’t. One of them was a push event carrying no commits at all, and my template did this without a second thought:

{{ if eq (len .payload.commits) 1 }}

Call len on something that exists and isn’t there and Go has a small panic about it. The fix is boring; stop assuming the field is there. Hugo’s with only enters the block when its argument is actually… something:

{{ with .payload.commits }}
  {{ if eq (len .) 1 }}
    {{/* the one-commit case */}}
  {{ else }}
    {{/* the many-commits case */}}
  {{ end }}
{{ end }}

Since I’ve implemented PAT support, I’m not so concerned about being rate-limited during local development, though I now use the fixtures as a fallback if any request fails. This allows me to keep working locally without disruption.

Don’t let someone else’s server break your build.

The real personal lesson is the moment your build depends on a third party at build time, you’ve effectively ceded control of your deployment to them. GitHub rate-limits you, or has a wobble, or changes a payload shape, and suddenly your perfectly good site won’t deploy and you’ll have a hard time.

So the changelog no longer treats GitHub as load-bearing. If the fetch fails for any reason it shrugs, logs a warning, and falls back to that saved fixture instead of taking the whole deploy down with it:

{{ $remote := try (resources.GetRemote $url $opts) }}
{{ with $remote.Err }}
  {{ warnf "changelog: GitHub fetch failed (%s); using the fixture" . }}
  {{ $response = os.ReadFile "static/github.json" | transform.Unmarshal }}
{{ else with $remote.Value }}
  {{ $response = . | transform.Unmarshal }}
{{ end }}

The try keyword is the hero here as it catches the error that GetRemote would otherwise throw and hands it back to me as a value I can actually do something with, rather than a smoking crater where my build used to be. Worst case, the page shows slightly stale history. That’s a trade I’ll take every single time over a red deploy or disruption to my local flow.

In conclusion…

Ironically, this is something I’ve dealt with quite frequently at work. Github & AWS API rate limits ( yes, we frequently got rate limited by the VPC API of all things ), Yarn will throw a 5xx and even Sentry will chimp out during source map uploads. All of these things can disrupt deployments.

The point is just assume these things will happen and then build defensively around them. I, ehhhh, just need to apply this wisdom to my personal stuff. 😅

Anyway, the changelog still builds itself. It just doesn’t get to take the rest of the site down with it anymore.

In this case, what I really needed was a simple way to assign an ordinal to an arbitrary number. For instance, if I have a value of 2, I might want to tack on a nd as a suffix, eg; 1st, 540th or 9001st and so on.

Specifically, I’d like to use this with dates, but from what I can tell, Hugo doesn’t support this out-of-the-box. Luckily, the framework gives you a few options to extend its functionality.

Shortcodes

With shortcodes you can create snippets that act as functions which can be used to dynamically inject HTML – among other things – directly into your rendered markdown. You can even pass both named and positional arguments through the shortcode to modify their behaviour as needed.

For instance, I have this custom shortcode used for displaying styled block quotes like:

“Out of all the things I have lost, I miss my mind the most.” Mark Twain

I also have this one that let’s me print out some fancy callouts:

Hugo ships with a default set of shortcodes covering a variety of additional kinds of embeddings.

Unfortunately, for my purposes, I’m not planning on using dynamic numeric ordinals in my markdown files. I need this to work with templates and for that, we use…

Partials

Unlike shortcodes, partials are small re-usable HTML components that are typically used to keep code duplication down. They are effectively context-aware templates that can accept arbitrary data which can be used in generating desired output.

Here is a small example of how I use partials for this blog. It’s a small data card component you might find scattered throughout the site. Partials let you quickly change a UI component in one place while having it propagate everywhere else.

For the purpose of this article, they can also be used to create custom template “functions” like this which solves my very specific problem:

{{- if and ( eq ( mod . 10 ) 1 ) ( ne ( mod . 100 ) 11 ) -}}
    st
{{ else if and ( eq ( mod . 10 ) 2 ) ( ne ( mod . 100 ) 12 ) -}}
    nd
{{ else if and ( eq ( mod . 10 ) 3 ) ( ne ( mod . 100 ) 13 ) -}}
    rd
{{ else -}}
    th
{{ end -}}

This may look a bit unreadable to people who aren’t super-familiar with Hugo’s template syntax, but it effectively resolves to:

const number = 10
let ordinal = "th"
if (number % 10 == 1 && number % 100 != 11) {
	ordinal = "st"
} else if (number % 10 == 2 && number % 100 != 12) {
	ordinal = "nd"
} else if (number % 10 == 3 && number % 100 != 13) {
	ordinal = "rd"
}

console.log(number + ordinal)

// prints 10th

Anyways, I have this saved as partials/functions/ordinal.html and can reference this in any of my templates like so:

{{ partial "functions/ordinal.html" $number }}

Where $number is any arbitrary number I’d like an ordinal suffix for. Take the following example template which generates a random range of numbers and prints them out using my ordinal partial:

{{ range seq 10 }}
  {{.}}{{ partial "functions/ordinal.html" . }} 
{{ end }}

Which generates:

1st 2nd 3rd 4th 5th 6th 7th 8th 9th 10th

Ground-breaking work! I can finally use ordinals in dates. Not exactly something I expected to write an article about, but here we are. 🤷

First, a Small Rant

Before you call me a Luddite, let’s be clear here, I don’t have an issue with the technology itself. I’m an engineer. I think it all has its uses, but force-feeding it into almost every aspect of our lives with no regard to any social and environmental impacts with frequently no chance of being able to easily opt-out does not sit right with me one bit.

We’ve seen this happen over the last decade with touchscreens, IoT, blockchain, NFTs, Web3 the list goes on. All promoted by the endless parade of interchangeable, nameless faceless talentless hacks and disingenuous grifters who only care about making “BIG LINE GO UP”.

It’s all so tiring, so perhaps you can understand why I’d immediately recoil in disgust and go my own way with an alternative I can control.

Or, to put it in even simpler terms:

The “Stack”

I suppose it’s technically a “stack”, but it seems a bit funny to use the term when referencing a blog setup. Just as with my Plant Smart project, the goal is to automate as much as possible while keeping maintenance requirements and hard- and soft-dollar costs as low as possible. For this, I need only use a handful of tools to keep this space operational.

As far as the question over “cost”, outside of time spent, it’s about ~$100 AUD per year for the domain name. This blog is a very small and inconsequential part of this domain, so the overall cost is absorbed by “other stuff”.

So, the greatest cost is… time? I guess 🤷

Obsidian

While I won’t be going into what Obsidian is and how it works, I will say it’s been a personal boon in how I keep notes and track of ideas. I have a lot of fleeting / ephemeral thoughts I that would otherwise lose instantly if I didn’t immediately jot them down using this app.

So, it makes perfect sense for me to use my personal vault to store all my blog content. The fact Obsidian content nothing more than Markdown files makes working with them using other bits of tech that much easier.

Oh, it’s free btw.

Hugo

This is probably one of the more popular options when it comes to static site generators. It’s easy to install, is incredibly extensible and has loads of documentation to help you along. The template syntax sits upon Golang’s templating engine, so if you have a strong background as a Go developer like myself, you’ll be right at home.

However, knowing Go isn’t a hard requirement though it will make getting on top of things quite a bit easier.

What I enjoyed while learning about this project was I got to control every aspect of how I wanted my blog to look and function. Building the current design from the ground up with Hugo was incredibly fun.

Also, free. Very cool.

Cloudflare

I already used Cloudflare to manage this domain’s DNS settings, so it only made sense to publish my static site to Cloudflare Pages over, say, Vercel or Github Pages. There’s nothing spectacular going on here outside of me just wanting to keep these two things in the same place.

That being said, I have no issues with migrating else where if the need should ever arise. Though, I doubt this blog will hit the 20k file limit of Cloudflare Pages any time soon.

Again, free for the purposes of this article. If you don’t have a dedicated personal domain, you will be provided a subdomain attached to the pages.dev apex.

The Good Stuff

How is this all put together for day-to-day usage? Luckily, the Cloudflare stuff runs on auto-pilot, so it’s pretty much set-and-forget. I suppose you could say the same about the other components, but they’re the parts I touch the most.

I have a single vault in Obsidian. It’s where all my thoughts go. Stored within is a top-level Blog/ directory. Care to hazard a guess as to what it may contain?

Hugo has the concept of page bundles where you can group all resources associated with your blog articles within a single directory. You’ll notice above all directories mirror the blog posts hosted on this site. In my case, we can consider the directory names to be the human-friendly slugs you see in your address bar that point to the associated content. Within these directories, you’d see any other resource I may link to; source code, images, etc… The best thing about this is I only have to perform a relative reference to these resources. If I want to link to image-1.png, I only have to reference it as ![](/blog/my-blog-publishing-setup/image-1.png) without worrying about specifying an absolute path; very nice.

When I want to write a new article, I first create a new directory here. I may already have the title of the article in mind, but it can be in flux until I decide to publish it. In the new directory, I create a file called index.md. This file contains all the content of the associated article written using Markdown.

I then use an Obsidian template dedicated to new blog posts and apply it to the new file. This ensures I have all front matter properties ready to go. This makes configuring my posts in the editor that much easier.

All that’s left is writing my articles! Well, not really. I need a way to synchronise these files with my local Hugo instance. As you can see here, the content mirrors what I have stored in my Obsidian vault.

This can be easily addressed with 2 small apps:

1. fswatch which monitors a specified directory for changes and allows you to trigger subsequent commands for specific file system events. Basically, I want to watch for any changes in my Obsidian vaults Blog/ directory.
2. rsync to dynamically keep the target Hugo directory in sync with the Obsidian source directory.

I, of course, have this placed in a script that I have running in the background while I write. This is a great help during the drafting phase as I have Hugo dynamically processing and displaying these changes locally. This allows me to easily proof my work visually as I carry on writing.

Here’s the full script:

#!/usr/bin/env bash

SOURCE=${SOURCE:="${1}"}
export SOURCE

DESTINATION=${DESTINATION:="${2}"}
export DESTINATION

fswatch -o "${SOURCE}" | while read -r event; do 
    rsync -av --delete "${SOURCE}" "${DESTINATION}";
done

I keep this running whenever I’m writing a new article. Obsidian saves in almost realtime, so as I type the changes almost immediately sync across to the local Hugo server so I can review my changes.

You effectively run it as:

$ ./sync.sh /path/to/obsidian/blog/ /path/to/hugo/content/

For rsync to work as intended and keep the destination directory completely in sync with the source directory, remember to add a trailing slash to both directory arguments.

No Auto-Commit?

Nope. Not here. There are a few other peeps with blogs out there that “commit on change”, but I prefer to have a bit more control over what actually gets published. For that, I simply use plain-old git commands to publish my new article:

$ git add -A
$ git commit -m 'some banal new bullshit screed'
$ git push origin main

Cloudflare monitors the associated repository, picks up the change and the auto-magic-ally builds and pushes everything to the public eye.

With this flow, I can see my changes locally in near realtime and push to “production” only when I’m happy with the results.

In Closing …

I’m stoked to have my very own place again that I have complete control over. I’ve thoroughly enjoyed creating this new blog design and I may even open-source it one day.

That said, I’ve made a very loose commitment to post a new article every week this year of our lord 2025 and I’m hoping this will make things heaps simpler. I only want to worry about what I’m going to write and just get the content out there.

I sincerely hope I can keep this self-imposed weekly commitment. 😬👍

I have no idea if time is actually passing. Obviously, I can eyeball timestamps if available, but there are times when so many lines zip through the buffer that it’s easy to lose track or even go cross-eyed.

It can be tricky to notice that lines are still being tailed if the output doesn’t drastically change in some meaningful or noticeable way. Something I like to is intercept each line and then output some kind of divider whenever n lines have been added to the buffer.

It’s as simple as:

#!/usr/bin/env bash

lines=0
while IFS= read -r line; do
	echo "${line}"
	((lines++))

	if ((lines % 5 == 0)); then
		echo "----------"
	fi
done

Effectively, all this does is:

1. Intercept each line of output being piped into the script.
2. Keep a running tally of lines we’ve intercepted so far; $lines++.
3. If the current tally is divisible by n, or in this case 5, spit out an additional line containing a divider.
4. Keep doing this forever until the process is terminated.

If you were to save this in a file named as divider.sh and set it to execute with something like chmod a+x you could test it by doing something like:

$ while true; do echo "emitting a noop"; sleep 1; done | ./divider.sh

And you’ll see something like the following:

emitting a noop
emitting a noop
emitting a noop
emitting a noop
emitting a noop
----------
emitting a noop
emitting a noop
emitting a noop
emitting a noop
emitting a noop
----------

This is pretty dumb, but now you can see that output is still being placed in your terminal buffer. I’ve set it to every 5 lines, but you can update the script to make that configurable. You could also play a sound when the script places a divider in the buffer using something like tput bel.

Anyway… Enjoy.

Before we begin

Today, I went ahead and made my Instagram profile private. I have no desire to delete my posts just yet, but decided to go ahead and unfollow everyone as a start. There are several ways to do this:

• API calls using an SDK, which requires a developer account.
• Manually deleting “handraulically” via the native app or browser, but that can quickly become tedious if you have more than a hundred or so follows.
• Random “GreaseMonkey” scripts off the web that may, or may not, work properly if at all.
• Reverse-engineering XHR calls from your browser and writing a simple Bash script to automate the process.

If you’ve read some of my previous blog posts, you probably can guess which one I’ll be going with.

Requirements

This guide assumes you have some programming or scripting knowledge.

You will need the following on your machine:

• A shell terminal with Bash, or ZSH, support.
• curl to make API calls from the command line.
• jq to easily parse JSON-based responses from said API calls. This can easily be installed using Homebrew or some other supported package manager.
• A modern browser that supports a developer tools console to intercept XHR calls.

It’s worth mentioning that I am using a MacBook with the Chrome browser, but other webkit-based browsers should support similar functionality.

Fire up the browser!

Open up your browser, or new tab, and navigate to your Instagram profile. Open your browser’s developer console and navigate to the “Network” tab. Clear out whatever requests are currently listed and then filter by “Fetch/XHR”.

Your following list

We need to intercept an XHR request that returns a JSON body containing some of your followers. Click the “following” link and monitor the resulting requests that pop in the list on the “Network” tab.

You should see something like following/count=12... in the list. Right-click that item and select “Copy as cURL”. Paste the new value in your clipboard into a scratch file for reference later.

Be aware this command will contain all the headers required to make requests from your terminal on your behalf. Do NOT share these details with anyone. I haven’t checked how long until the tokens within the auth headers expire, so assume they are long(ish)-lived and treat them accordingly.

An unfollow request

Next, we need to intercept an XHR request to unfollow someone. Going back to your following list, select a random user and unfollow them while once again keeping an eye on the “Network” tab.

You’re going to see something like the following; an XHR event making a POST request to “destroy” your chosen follower. Once again, you’ll right-click and “Copy and cURL”. Paste this into your scratch file as well for future reference.

Putting it all together

Now that you have your two types of requests, we’ll need to script the following steps:

• Execute a curl request to fetch a JSON body containing the accounts you follow, one page of results at a time.
• Use jq to extract their Instagram IDs.
• Store the results in a variable named… drumroll, pls… $ids.
• Iterate through each $id.
• Execute a curl request to unfollow each $id.

Here is the meat of the script. I have included only the necessary headers required to successfully make these requests on your Instagram account’s behalf. You’ll need to swap out the ... with your own values.

You may also notice that I’ve change the count=... parameter to 100. It may be able to go higher, but I haven’t tested it.

ids=$(
  curl 'https://www.instagram.com/api/v1/friendships/.../following/?count=100&hl=en' \
    -H 'cookie: ...'           \
    -H 'x-asbd-id: ...'        \
    -H 'x-csrftoken: ...'      \
    -H 'x-ig-app-id: ...'      \
    -H 'x-ig-www-claim: ...'   \
    -H 'x-requested-with: ...' \
    -H 'x-web-session-id: ...' | jq -r '.users[].id'
)

while IFS= read -r id; do
  echo -n "Unfollowing user '${id}'"

  curl -s -o /dev/null "https://www.instagram.com/api/v1/friendships/destroy/${id}/?hl=en" \
    -H 'cookie: ...'           \
    -H 'x-asbd-id: ...'        \
    -H 'x-csrftoken: ...'      \
    -H 'x-ig-app-id: ...'      \
    -H 'x-ig-www-claim: ...'   \
    -H 'x-requested-with: ...' \
    -H 'x-web-session-id: ...' \
    --data-raw "container_module=profile&nav_chain=PolarisProfilePostsTabRoot%3AprofilePage%3A1%3Avia_cold_start%2CPolarisProfilePostsTabRoot%3AprofilePage%3A2%3Aunexpected&user_id=${id}"

  echo ' ... done!'
done <<< "${ids}"

echo "Finished"

You can save this as a file and make it executable, but I just paste this bad boy into your terminal. The above script will remove 100 accounts at a time. Just keep executing the script until you no longer see results like the following:

Unfollowing user '111'... done
Unfollowing user '222'... done
Unfollowing user '333'... done
Unfollowing user '444'... done
Unfollowing user '555'... done
Unfollowing user '666'... done
Unfollowing user '777'... done
Unfollowing user '888'... done
Finished

Caveats, gotchas & other things

I’ve only had a few hundred accounts that I followed, so I didn’t feel the need to make this a bullet-proof solution. There’s quite a bit missing and things can go wrong. Here are some things to consider off the top of my head :

• The session tokens will expire eventually, so you will have to refer back to your “Network” tab in your developer tools console to get new ones.
• As this example only unfollows a static amount at a time, re-executing the script may become tedious for very large follow numbers. Perhaps, you can modify this script to add some basic pagination.
• There is no error handling here, so keep an eye out for potential rate limiting or expired token issues with the Instagram API.
• Constantly updating session header values can be made a bit simpler by resorting to variables instead.

In closing …

Cleaning out old, unused or unwanted social media accounts can be liberating. I’ve recently gutted my Twitter presence and I haven’t been on Facebook in years, though I still do use Messenger to keep in touch with a handful of people. I’d prefer using Signal for this, but trying to convince other people to install yet another messaging app is like pulling teeth.

Anyway, I actually enjoy writing these kinds of posts. What other social networks should I cover? LinkedIn? Threads? What are some ways you can improve and build on the above? Lemme know in the comment section below.

Accumulation of Underlying Issues

Over time, systems and processes can accumulate small issues, technical debt, or unnoticed problems. These issues can build up, leading to a higher likelihood of a significant failure or incident occurring as time goes on.

Complacency and Reduced Vigilance

When a system or service has been running smoothly for an extended period, teams may become complacent and less vigilant. They might not be as proactive in monitoring, testing, and maintaining the system, which can increase the risk of failure.

Evolving Environments

As technology and business environments evolve — as they inevitably do in our space —, the context in which a system operates also changes. What was once a stable and reliable configuration may no longer be suitable, leading to unexpected issues or failures when the system is finally pushed to its limits.

Regression to the Mean

The law of large numbers suggests that over time, events tend to revert to their average or “mean” frequency. If you’ve experienced an unusually long period without incidents, statistics may suggest that you’re due for one soon, just as a run of heads in a coin toss doesn’t make tails any less likely on the next toss.

Maintaining Awareness

“The price of freedom stability is eternal vigilance.” Ancient Klingon Proverb ( probably )

Mitigation Strategies

Recognize the importance of proactively addressing issues before they accumulate. This involves regular monitoring, capacity planning, load testing, and maintenance to reduce the likelihood of a sudden failure.

Risk Management

Focus on identifying and managing risks, even during periods of relative stability. They plan for various failure scenarios and aim to minimize their impact through redundancy, graceful degradation, and fault-tolerant design.

Continuous improvement

Promote a culture of continuous improvement, encouraging teams to learn from past incidents, conduct post-mortems, even live-fire exercises and apply those lessons to prevent similar issues in the future.

Metrics & Monitoring

Use metrics and monitoring tools to maintain a vigilant eye on system health and performance. They set thresholds and alarms to detect anomalies early, regardless of how long it’s been since the last incident.

At any given point your production workloads may be operating under any number of unknown failure modes. Things break; it’s inevitable. Adjust your expectations accordingly and build around this fact.

In Conclusion

The idea that the longer something doesn’t happen, the sooner it will is a reminder of the importance of vigilance, proactive maintenance, and risk management not only in site reliability engineering, but software engineering as a whole.

This may not be a deterministic law as it highlights the tendency for issues to accumulate over time if not addressed, making it crucial to maintain a robust and resilient system.

It’s as simple as that?

That was the idea, at least. I just wanted a single place that had all the information I needed so I could make informed purchasing decisions. I also figured I couldn’t possibly be the only person who has this problem. So, I set off to make this project during the holiday break of 2022.

However, I didn’t want to make it too easy for myself…

A challenge!

I work as a weird combination of SRE and DevOps engineer. I have plenty of smarts when it comes to the technical side of hosting and running things, but it’s been years since I’ve done any front-end work. Career-wise, I need to be able to speak the same language as the teams I have to support. So, understanding how far behind that portion of my skillset had become, I thought this would be the perfect opportunity to brush up.

The Requirements:

• Must keep operational costs as close to zero as possible. The only money I’ve put down on this so far is the $20 AUD to purchase the domain name.
• Must be hosted on Cloudflare Pages. I could’ve chosen Github Pages to keep everything in one place, but automated Cloudflare builds & deployments work out of the box with minimal configuration. Besides, I’ve already worked with the latter and I wanted the challenge of trying something new.
• Must be completely static. There should be no server-side rendering, processing or other explicit backend dependencies to manage. For this, I use the pre-rendering functionality that comes packaged with SvelteKit. I’ve been working with this for only a few weeks now and I’m a full convert.
• All data must be served statically as well. The entirety of the data set is contained within a single JSON file, which you can view at /plants.json. This is the source of truth for all derivative data sets and lookup tables used on this site. It’s effectively what I call a “dumb API”. Check out the API documentation if you’d like to know more about it.
• Must use SvelteKit, TypeScript, TailwindCSS and Vite. I had zero working knowledge of any of these and my frontend peers can’t seem to shut up about them. So… why not?
• Must be fast. Everything is static, compressed, cached and sitting behind a world-class CDN. I’ve worked in a network performance and load testing SaaS for close to 5 years now. I wouldn’t be able to look myself in the mirror if I couldn’t easily do this one. 😅
• Must be open-source and community-driven. At one point, I would like to take the hands off the wheel and see if other interested parties would like to get involved and help out with managing datasets and fixing bugs. GitHub allows for pretty much all of this. Check how to contribute or see the latest contributions.

So far, it’s going quite well. I haven’t had any issues with meeting any of these self-imposed development constraints.

Any Caveats?

Definitely. The most obvious one would be image storage which has a direct effect on not only repository size — which currently clocks in at over 1.5GB — but, also build and deployment speeds. I could shell out $5 - $10 for object storage and image processing, but that would go against the first constraint. For now, image data will live in the GitHub repository as a perfectly reasonable compromise.

Where did you source all this data?

There are quite a few sources I’ve collated from, but these are the main ones.

1. iNaturalist is the best source of high-quality, community-driven creative commons license photography. All images have been sourced from this site along with licensing and attribution data.
2. ASPCA was used to initially prime the first dataset. This is also where I sourced most of the common name and symptom data.
3. Wikipedia is the best source of scientific classification data out there.

All of this disparate data was collated and munged together by several processing scripts written in Go as Magefiles. It got me about 95% there, but it still needs quite a bit of handraulic finessing.

Where to go from here?

I think I’ve met most, if not all, of my requirements. The datasets for the project still need a lot of love and I plan on supporting listings for a variety of other pet species. I already have the data; just need to go through it with a fine-toothed comb. Not to mention responsiveness for smaller screens needs a solid amount of work.

I’m having loads of fun at the moment learning new things. I hope I can keep doing this for a while longer. However, I do have other things planned for the future and will be using what I’ve learned here as a kind of launching pad.

I sincerely hope you find Plant Smart as useful as I had fun making it.

Let’s leave out all doubt regarding our god-tier levels of snarkiness.

Today, we’re going to write a laughably-simple Bash script that lets our coworkers know exactly what we think. Along the way, you’ll learn a little bit more about Bash such as:

1. Setting some default flags.
2. Reading text from stdin.
3. Parameter expansion.
4. Looping through each character of a string.
5. Creating random numbers.
6. Swapping character types using the tr command.
7. And, finally, making your long-suffering coworkers’ eyes roll.

Here We Go!

First, I’m going to just paste the entire script here and then we’ll go through all the important bits line-by-line:

#!/usr/bin/env bash

set -eo pipefail

read text

for (( i=0; i < "${#text}"; i++ )); do
  if [[ $(( "${RANDOM}" % 2 )) -eq 0 ]]; then
    echo -n "${text:${i}:1}" | tr '[:lower:]' '[:upper:]'
  else
    echo -n "${text:${i}:1}" | tr '[:upper:]' '[:lower:]'
  fi
done

echo

Alright, let’s take it from the top. All Bash scripts should start with a “shebang line”. This tells your terminal which environments and runtimes your script should run under. There are more traditional versions of this — #!/bin/bash, for instance — but, our way is considered to be the most portable overall.

#!/usr/bin/env bash

Next, we setup some environmental flags. I use at least the following for all of my personal scripts:

set -eo pipefail

Here is what they do:

1. set -e: Instructs Bash to immediately exit if any command has a non-zero exit status. This is how it works in most languages, but with Bash, it just keeps on trying to execute subsequent commands. This is generally acceptable on the command line, but not in a script. If we encounter an error, we want to exit immediately.
2. set -o pipefail: This tells Bash not to mask errors that may appear in a pipeline of commands. We want any failed command’s exit code in a pipeline to bubble up to the script itself and then exit with that code.

Speaking of pipelines, we want to be able to pipe text to this script so we can do something like the following:

echo "wilhelm, i asked you to patch the server." | ./spongebob
wiLHELm, I AsKEd yOu TO PatCh The seRvEr.

Let’s halt the script and wait for user input and then assign that input to variable text:

read text

Now that we have some text, we need to start randomly-swapping between upper and lower case characters. In order to do that, we need to know the number of characters in our string so we can build a nice loop:

for (( i=0; i < "${#text}"; i++ )); do
  # ... sweet code goes here 
done

We could do something in a sub-shell here like $(echo "${text}" | wc -c) to get the character count, but why do that when we could get the same result with "${#text}". This lovely bit of “shell parameter expansion” helps us avoid sub-commands and sub-shells.

Next, we want to be able to randomly swap the capitalisation of each character in the string to get the appropriate effect. A character is either upper- or lower-case, so minimum we need only to swap randomly between 2 values; 0 and 1. We can get this effect in Bash by using the internal ${RANDOM} function like so with the modulo ( or “mod” ) operator:

if [[ $(( "${RANDOM}" % 2 )) -eq 0 ]]; then
  # ... do something
else
  # ... do the opposite
fi

We’re now at the point where we want to modify the character associated with the for loop’s current iteration, but how do we get it from the text variable? Once again we use some parameter expansion in the form of ${text:offset:length}. We have the value for “offset” already; it’s ${i}. We only want a single character returned, so we use 1 for “length”.

echo -n "${text:${i}:1}"

This spits out the current character for each iteration of our loop. The -n in the echo statement simply stops Bash from adding a newline to the end of the result. Otherwise, you’d get a line per character as output.

Finally, we want to do case swapping. For this, we pipe the output of the above echo command into the tr command. Within the above if statement, if our random number equals 0, let’s swap a lowercase [:lower:] character with an uppercase [:upper:]:

echo -n "${text:${i}:1}" | tr '[:lower:]' '[:upper:]'

And, then, we do the opposite for any other result:

echo -n "${text:${i}:1}" | tr '[:upper:]' '[:lower:]'

You’ll notice a final echo command at the bottom of the script. Thanks to the final echo -n ... command from the previous for loop, you may find your results prepended to your command prompt. This ensures a newline makes it to the end of your 🧽 output.

Testing Time

Save the script as spongebob and make it executable with chmod a+x spongebob. That should be it! Here are a few of my results:

$ echo "abandon all hope, ye who enter here." | ./spongebob
abaNDON ALl Hope, yE wHO EnTer heRE.
$ echo "Wilhelm, that last deployment failed. Could you roll it back, please?" | ./spongebob
WilhelM, THAT LASt dEployMeNT fAILED. coUlD YOu RoLL It BaCk, PlEaSE?
$ echo "Wilhelm, I am your manager. Please, stop mocking me." | ./spongebob
wIlHElM, i aM yOUr MANagER. PlEasE, sTOP mOCKINg mE.
$ echo "Wilhelm, should we use Kubernetes for this?" | ./spongebob
WilHELM, should We Use KuBerNeTEs for THiS?

In Conclusion…

Ok, obviously this was all a clever ploy to get you to learn a few more Bash things. Definitely do not use this new knowledge to frustrate and annoy your coworkers. Please, be considerate of other people’s mental well being. I mean, what I meant to say was:

“DefInIteLY dO Not UsE THis nEw knoWleDGE to FRUStRATe aND aNnOY YoUR cOWorkers. pLeAsE, be ConsiDeRAtE oF oThEr peOPle’s mEnTal well beiNG.” Me.

There are any number of ways you could change this script. Instead of liberal use of echo, you could just build a string assigned to a variable and spit that out at the end. Instead of using else you could use continue to skip the final if fallback. Try a few and see what changes.

For the purposes of this article I felt the above sequence of commands was clear enough for most people to follow along.

I hope you learned something!

1. Bash for portability.
2. jq to filter JSON objects from RESTful APIs.
3. AWS CLI as I cannot stand dealing with the web console.
4. curl ( with wget as a reasonable fallback ) to interact with various APIs.
5. Terraform for most of my IaC needs.

It wouldn’t be an exaggeration to say I literally use these tools every day. Specifically, the first 3 items. Furthermore, to say that manually sorting through Docker containers is a practice in tedium would be a massive understatement. Because of this, not a day passes where I am not grateful to have found jq.

In this case, my favourite duo is Docker and jq… which is what this article is about. Let’s learn some fun filtering patterns to make your life a bit easier if you’re stuck in a terminal all day like myself.

Setting Up

First and foremost, you’re going to need to install jq in your test environment. If you’re on most common Linux distributions:

apt-get install jq

Or, on MacOS:

brew update && brew install jq

I’m assuming that if you’re reading this you already have Docker installed in your test environment. If not, head out to their website and read their installation documentation. It tends to vary wildly depending on your OS.

Let’s spin up some containers we can play around with. I use redis for stuff like this:

for i in {1..5}; do docker run -it --rm -d redis; done

Environmental Variables

Sourceable Blocks

There may be times I wish to collate all environmental variables associated with a set of containers and stick them a .env file to source later. Here we create a block for all the redis containers we just spun up:

docker inspect $(docker ps -q) | jq -r '
    .[].Config.Env
  | flatten 
  | .[]
'

This one will generate a lot of duplicates as all our test containers are effectively clones. What if we want a distinct set of instead? We can use the unique filter:

docker inspect $(docker ps -q) | jq -r '
	[
	  .[].Config.Env
	] 
  | flatten 
  | unique 
  | .[]
'

What if I want a set from specific containers?

docker inspect $(docker ps -q) | jq -r '
    [
	    .[] 
      | select([.Id] | inside([
	      "747c2c92855f88a8e9aa9709dcdf3a01f1677e70bf4c0bf0e520eb38ac502876",
	      "2cf70261ef62bc36d19aba02f8ef7b8d7aabfcb1e9f4593a919baa17f80aca5b"
	    ]))
      | .Config.Env
	] 
  | flatten 
  | unique 
  | .[]
'

How about filtering by a specific ip address for the default bridge network? Using a different network? Just replace bridge with the alternate network name.

docker inspect $(docker ps -q) | jq -r '
    .[] 
  | select(.NetworkSettings.Networks.bridge.IPAddress == "172.17.0.2") 
  | .Config.Env 
  | .[]
'

Multiple ip addresses?

docker inspect $(docker ps -q) | jq -r '
    .[] 
  | select([.NetworkSettings.Networks.bridge.IPAddress] 
  | inside([
      "172.17.0.2",
      "172.17.0.3"
    ])) 
  | .Config.Env 
  | .[]
'

As Flags Instead

Here’s a strange one I’ve had to do. What if we want to recreate a list of --env flags we could pass to other docker run ... commands?

docker inspect $(docker ps -q) | jq -r '
    [
        [
          .[].Config.Env
        ] 
      | flatten 
      | unique 
      | "--env=\(.[])"
    ] 
  | join(" ")
'

You’ll get a string that looks similar to the following.

--env=GOSU_VERSION=1.14 --env=PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin --env=REDIS_DOWNLOAD_SHA=f0e65fda74c44a3dd4fa9d512d4d4d833dd0939c934e946a5c622a630d057f2f --env=REDIS_DOWNLOAD_URL=http://download.redis.io/releases/redis-7.0.4.tar.gz --env=REDIS_VERSION=7.0.4

Filtering By Relative Time

I’ve had to use these several times for things like garbage collecting and cleaning up long-running containers. This uses a bit of Bash to help generate the relative timestamps. We use the date command for this, but if you’re on a darwin-based OS you’ll need to install gdate first via Homebrew with:

brew install coreutils

Substitute the following calls to date with gdate below if on MacOS.

Newer Than …

Filtering out recent containers with relative time is fairly straight-forward. The following gives us all containers created in the last hour as JSON output from docker inspect ...:

threshold=$(date -d "1 hour ago" +%s)
docker inspect $(docker ps -q) | jq --argjson threshold "${threshold}" -r '
    .[] 
  | select((.State.StartedAt | split(".")[0] | "\(.)Z" | fromdate) > $threshold)
'

Perhaps you just want the names of the containers instead:

threshold=$(date -d "1 hour ago" +%s)
docker inspect $(docker ps -q) | jq --argjson threshold "${threshold}" -r '
    .[] 
  | select((.State.StartedAt | split(".")[0] | "\(.)Z" | fromdate) > $threshold)
  | .Name[1:]
'

Docker likes to put a / at the beginning of each generated container pet name, so we use .Name[1:] to snip it off.

What about just returning the associated ids?

threshold=$(date -d "1 hour ago" +%s)
docker inspect $(docker ps -q) | jq --argjson threshold "${threshold}" -r '
    .[] 
  | select((.State.StartedAt | split(".")[0] | "\(.)Z" | fromdate) > $threshold)
  | .Id
'

Or, ip addresses:

threshold=$(date -d "1 hour ago" +%s)
docker inspect $(docker ps -q) | jq --argjson threshold "${threshold}" -r '
    .[] 
  | select((.State.StartedAt | split(".")[0] | "\(.)Z" | fromdate) > $threshold)
  | .NetworkSettings.Networks.bridge.IPAddress
'

Older Than???

The same filters apply, but you’re just flipping the compare operator from > to <:

threshold=$(date -d "1 hour ago" +%s)
docker inspect $(docker ps -q) | jq --argjson threshold "${threshold}" -r '
    .[] 
  | select((.State.StartedAt | split(".")[0] | "\(.)Z" | fromdate) < $threshold)
'

Other Common Patterns

There have been times where I need a block of container ip addresses so I can dynamically update some Nginx upstreams somewhere. With jq it’s as easy as:

docker inspect $(docker ps -q) | jq -r '
  .[].NetworkSettings.Networks.bridge.IPAddress
'

The output would look something like this:

172.17.0.6
172.17.0.5
172.17.0.4
172.17.0.3
172.17.0.2

Perhaps, I want to find a specific container id by it’s ip address:

docker inspect $(docker ps -q) | jq -r '
	.[]
  | select(.NetworkSettings.Networks.bridge.IPAddress == "172.17.0.3")
  | .Id
'

Or, just give me the names of all running containers:

docker inspect $(docker ps -q) | jq -r '
  .[].Name[1:]
'

Which would give you something like:

zealous_hertz
fervent_dijkstra
focused_galileo
zealous_poincare
naughty_wescoff

Cleaning Up …

We’re considerate people, so let’s clean up after ourselves by killing all containers created using the redis image:

docker kill $(docker inspect $(docker ps -q) | jq -r '
	.[] 
  | select(.Config.Image == "redis") 
  | .Id
')

Keep in mind this will kill all redis containers.

In Closing …

Having a toolbox filled with utilities you can mix and match together is great if you spend most of your day working in a terminal on a glowing rectangle. Hopefully, I’ve made a strong enough case to for Docker and jq as a great combo.

But, Wait! There’s More!

Why copy and paste these commands in your terminal, when you could just download and install a handy little Bash script to do it all for you? dq has all the above built-in as well as heaps more bells and whistles.

Here are a few command examples if you’re interested:

dq older-than 4 months --only-return ips --network pebkac
dq newer-than 2 fortnights --only-return ids
dq filter '.[].Name[1:]'
dq find-by-ip-address 172.17.0.2 --only-return names --network foo

Gathering Requirements

Since we’re doing some destructive operations, we’ll need to make sure our Slack user has the appropriate permissions to manage emoji; adding & deleting. This means using a different section of the Slack interface which uses an entirely seperate set of API endpoints.

You’ll still be using the following, which can be found using instructions from the previous article:

• An API request token.
• A session cookie.
• A workspace, or team, id.

In addition, you will need your workspaces’s subdomain, or URL. This can easily be found within the Slack app itself:

Testing Your Findings

This is an incredibly straight-forward process as it’s quite similar to what we already know. The primary difference is this is a multipart/form-data upload. So, instead of shipping of a JSON payload, it’s a form with associated fields.

With the information gathered above, you can upload directly to Slack using a simple cURL command like so:

curl -s --compressed "https://<domain>.slack.com/api/emoji.add" \
  -H 'content-type: multipart/form-data' \
  -H "cookie: d=<cookie>;" \
  -F "token=<token>" \
  -F "name=<name>" \
  -F mode=data \
  -F "image=@<local-emoji-path>"

Replace the following:

• <domain> is your workspace’s, or team’s, private URL.
• <cookie> is your session cookie value.
• <token> is your request token.
• <name> will be the named reference of your new emoji.
• <path> is the relative, or absolute, local path of your emoji file to upload.

The JSON response to this request will have the following structure if all goes well:

{ "ok": true }

And, if something went wrong:

{ 
	"ok": false,
	"error": "err_code"
}

That’s it! Now that we know how to upload from the command line.

The Implementation

If you’re only uploading a handful of emojis, it might make sense to just do it via Slack’s UI. However, this can get a bit tedious if you have dozens, or even hundreds, to upload. We can automate things even further by:

• Gathering all supported images from a specific source directory.
• Iterating through our findings and bulk-upload them all in one go.

So, let’s do just that. But, first, let’s set up some environmental variables so we can easily configure out script:

: ${SLACK_COOKIE:=}
: ${SLACK_TOKEN:=}
: ${SLACK_DOMAIN:=}

Slack only supports png, gif and jpg image formats, so lets see what we can find from our present working directory:

files=$(find . -iname \*.gif -o -iname \*.png -o -iname \*.png -maxdepth 1)
[[ "${files}" == "" ]] && exit 0

Exit if we can’t find any results. No need to continue if we haven’t a thing to upload.

Next, we just iterate through our findings using read and some more Bash parameter expansion to determine what will be the new emoji’s reference name:

echo "${files}" | while read -r path; do 
  name=$(basename "${path%.*}")
done

We can easily parse the response bodies with jq for some error checking and we’re good to go:

if [[ $(echo "${result}" | jq -r ".ok") == false ]]; then
  echo "${result}" | jq -r ".error"
  exit 1
fi

We now have all the information we need to upload each file in bulk. Put it all together and you’ve got a working bulk emoji uploader:

#!/usr/bin/env bash

set -eo pipefail

: ${SLACK_COOKIE:=}
: ${SLACK_TOKEN:=}
: ${SLACK_DOMAIN:=}

files=$(find . -iname \*.gif -o -iname \*.png -o -iname \*.png -maxdepth 1)
[[ "${files}" == "" ]] && exit 1

echo "${files}" | while read -r path; do 
  name=$(basename "${path%.*}")

  result=$(
    curl -s --compressed "https://${SLACK_DOMAIN}.slack.com/api/emoji.add" \
      -H 'content-type: multipart/form-data' \
      -H "cookie: d=${SLACK_COOKIE};" \
      -F "token=${SLACK_TOKEN}" \
      -F "name=${name}" \
      -F mode=data \
      -F "image=@${path}" 
  )

  if [[ $(echo "${result}" | jq -r ".ok") == false ]]; then
    echo "${result}" | jq -r ".error"
    exit 1
  fi

  echo "uploaded :${name}:!"
done

There you go! To test it out yourself, save this as an executable script and make sure your present working directory has some supported images in it. Pass through the your environmental variables when executing and you’re off:

$ SLACK_DOMAIN=*** SLACK_COOKIE=*** SLACK_TOKEN=*** ./upload.sh
uploaded :stonks:!
uploaded :boop:!
uploaded :derp:!
uploaded :booyah:!
uploaded :merp-flakes:!

Nice! 😊

Once Again, Something Better!

While perfectly functional, there’s not a lot of flexibility. No error checking, filtering or confirmation checks. If you’re looking for something a bit more fleshed out, look no further!

The Final Piece …

So far, we’ve gone over how to download and upload large sets of emoji, but what if you want to nuke them from orbit? The final article in this series will cover how to bulk delete while using advanced filters to pin-point specific sets of emoji you wish to remove.

Hope you’ve learned something useful!

But, when it comes time to part ways it’s only understandable to want to gather your stuff for the next place to start the process again. Over the years one can collect and curate quite a nice hoard of stupid images.

I’ve been using Slack prolifically for the past few years. Unfortunately, there is no simple way of exporting your precious collection outside of the old “right-click and save” method. This is effective for smaller collections of a couple dozen or so, but becomes impractical when there are hundreds or even thousands.

And, of course, I’m not alone. There have been a non-zero number of times when someone left the company only to hit me up later asking for an emoji archive.

Yes, this really happens and something must be done about it!

What follows is an unnecessarily long dive into a little bit of reverse engineering one of Slack’s API endpoints, subverting it for our own use and writing a stupid little tool to help ourselves ( and others ) automate this process for the future.

Some Investigative Work

There just has to be a straightforward way we can mostly automate this. So, I did some cursory digging around the Chrome Developer Tools console and found this endpoint in the “Network” tab:

https://edgeapi.slack.com/cache/T0XXXX/emojis/list?fp=97

The T0XXXX will be your workspace, or team, id. Make note of it, you’ll need it later.

If you intercept one of these requests and take a peak at the “Response” tab in the “Network” panel, you’ll see a JSON response with a structure similar to:

{
	"ok": true,
	"next_marker": "a-custom-emoji",
	"results": [
		{
			"name": "another-custom-emoji",
			"value": "https://emoji.slack-edge.com/T0XXXX/another-custom-emoji/xxxxxxx.png"
		}
		... more emojis ...
	]
}

• ok tells you whether the request was successful or not. If it returns false there will be an additional field named error that should clue you into what went wrong.
• next_marker tells you where in the list of custom emoji the next page of results should start. We use this as the value of marker in subsequent request payloads ( see below ). This is effectively how you page through large lists of emoji.
• results should be obvious, but it contains an array of objects with name and value fields.
  • name is the alias of the emoji.
  • value is the direct CDN URL to the emoji.

You may see other fields, but next_marker and value are the only fields we really care about.

Next, If you take a look at the “Payload” tab in the “Network” panel for the same request, you’ll see something similar to:

{
	"token": "xoxc-xxxx-xxxx-xxxx",
	"count": 100,
	"marker": "some-other-emoji"
}

• token is the API request token the Slack client uses to make the emoji list request on your behalf.
• count is the amount of emojis to return as a page.
• marker tells the API where in the emoji list to start paging.

The final piece of the puzzle is getting the value of your session cookie. You can grab this by clicking the “Request Headers” section under the “Headers” tab. You should see a header labeled cookie.

There’s going to be a fairly large block of text to parse through, but you’re looking for the cookie named d. Save everything between the d= and the closing ;. You’re going to need this to authenticate with the Slack API itself.

Testing Your Findings

We now understand the structure of our payload. We also have the endpoint to hit as well as our tokens and session cookie to authenticate. Piecing it all together, you can now use cURL to hit the API directly from your terminal:

$ curl --silent https://edgeapi.slack.com/cache/<team>/emojis/list?fp=97
  -H 'cookie: d=<cookie>;' \
  --data-raw '{"token":"<token>","count":10}'

Replace the following:

• <team> is your workspace, or team, id.
• <cookie> is your session cookie value.
• <token> is your request token.

If everything worked out, you should have successful API response like the one above. If the response has root field named marker_next, make note of the associated value to help page through subsequent results. Keep doing this in a loop until your response no longer returns a marker_next field.

Once that happens, you’re at the end of the list.

With a little bit of help from jq to parse and filter the JSON responses, we can grab a simple list of URLs.

$ <previous_curl_command> | jq -r '.results[].value'
https://emoji.slack-edge.com/T0XXXX/no/060fca9aa2581a93.png
https://emoji.slack-edge.com/T0XXXX/nods/c7fe54342ab5b8b6.gif
https://emoji.slack-edge.com/T0XXXX/nods-back/fbd1b5384cdd0905.gif
https://emoji.slack-edge.com/T0XXXX/nomnomnom/612cfc74c785010d.gif
https://emoji.slack-edge.com/T0XXXX/octocat/627964d7c9.png
https://emoji.slack-edge.com/T0XXXX/ohyou/6a352df984d9c076.gif
https://emoji.slack-edge.com/T0XXXX/one-sec-cooking/e749627cb088859d.png
https://emoji.slack-edge.com/T0XXXX/oof/db94710445cbb206.png
https://emoji.slack-edge.com/T0XXXX/petrol/35ed3db238f795fc.png
https://emoji.slack-edge.com/T0XXXX/piggy/b7762ee8cd.png

We now understand how to collect everything we need, so let’s automate this a bit more.

The Implementation

We’re going to write a little script that’ll download all the custom emojis from the target Slack workspace. It’s going to perform the following steps:

1. Define a starting JSON payload.
2. Create a run loop.
3. With each loop we make a request using the API endpoint.
4. Do some error checking.
5. Iterate through the results and save them to disk.
6. If the results contain a next_marker update the JSON payload with a marker field and continue the run loop.
7. Keep going until the last request no longer provides a next_marker.

First, we want to define some environmental variables so we can easily configure the script:

: ${SLACK_WORKSPACE_ID:=}
: ${SLACK_COOKIE:=}
: ${SLACK_TOKEN:=}

Now we define our JSON payload object. We use jq here to keep things easy. Not only is it great for parsing and filtering JSON in Bash, it also makes building and manipulating objects simple:

payload=$(
  jq -nc \
    --arg     token "${SLACK_TOKEN}" \
    --argjson count "100" \
    '{
      token: $token, 
      count: $count
    }'
)

Next, is the run loop where we make our API requests. This passes through the payload object we just defined while also targeting our workspace:

while true; do
  result=$(
    curl -s --compressed "https://edgeapi.slack.com/cache/${SLACK_WORKSPACE_ID}/emojis/list?fp=97" \
      -H "cookie: d=${SLACK_COOKIE};" \
      --data-raw "${payload}" 
  )
done

We obviously want to do some simple error checking so we know what we did wrong if the requests fail. Grab the error, spit it out and terminate the script.

if [[ $(echo "${result}" | jq -r ".ok") == false ]]; then
  echo "${result}" | jq -r ".error"
  exit 1
fi

We now start iterating through any of the results we got from the initial request. We really only care about the value field as the URL gives us all the pieces we need to give our emojis a human-readable filename.

Here we treat the resulting URL as a standard file path and use dirname paired with basename to get the name of the parent directory of the file. The parent directory holds the actual name of the emoji, while the file itself is only a random hash. We use Bash’s parameter expansion feature to construct our filename.

Finally, we download the remote file to the current directory:

echo "${result}"  | jq -r '.results[].value' | while read -r url; do 
name=$(basename $(dirname "${url}"))
curl -s -o "${name}.${url##*.}" "${url}"
done 

We need to check if there are more results to page through, so let’s look for the marker_next field. If we can’t find one, then mission accomplished; all done. However, if we do get a value, we take the previous payload we constructed and add the marker field.

The next loop will pick this change up and pass it on through to the API request telling Slack to give us the next page of precious emojis.

marker=$(echo "${result}" | jq -r ".next_marker")
[[ "${marker}" == "null" ]] && exit 0

payload=$(echo "${payload}" | jq --arg marker "${marker}" '. + { marker: $marker }')

The Finished Product

#!/usr/bin/env bash

set -eo pipefail

: ${SLACK_WORKSPACE_ID:=}
: ${SLACK_COOKIE:=}
: ${SLACK_TOKEN:=}

payload=$(
  jq -nc \
    --arg     token "${SLACK_TOKEN}" \
    --argjson count "100" \
    '{
      token: $token, 
      count: $count
    }'
)

while true; do
  result=$(
    curl -s --compressed "https://edgeapi.slack.com/cache/${SLACK_WORKSPACE_ID}/emojis/list?fp=97" \
      -H "cookie: d=${SLACK_COOKIE};" \
      --data-raw "${payload}" 
  )

  if [[ $(echo "${result}" | jq -r ".ok") == false ]]; then
    echo "${result}" | jq -r ".error"
    exit 1
  fi

  echo "${result}"  | jq -r '.results[].value' | while read -r url; do 
    name=$(basename $(dirname "${url}"))
    curl -s -o "${name}.${url##*.}" "${url}"
  done 

  marker=$(echo "${result}" | jq -r ".next_marker")
  [[ "${marker}" == "null" ]] && exit 0

  payload=$(echo "${payload}" | jq --arg marker "${marker}" '. + { marker: $marker }')
done

This is pretty much it. A barebones script that downloads all findings to its present working directory. To test it out yourself, make the script executable, define the environmental variables and fire it off like so:

$ SLACK_WORKSPACE_ID=*** SLACK_COOKIE=*** SLACK_TOKEN=*** ./fetch.sh

An Even Better Version!

I know I just made you read through all this, but you also could’ve just downloaded the tool from here. It’s got heaps more bells and whistles ( if you’re into that sort of thing):

The Bitter Irony

In typical bored engineer fashion, I’ve spent more time writing this article, documenting and publishing the code than I ever would have personally just “handraulically” doing this from time to time.

Hopefully, you’ve learned something new! 🤞

Bonus Relevant XKCD Image

source: xkcd: automation

Because we had so many to manage and not a whole lot of people power ( it was just 2 of us ), it got easy for a lot of hidden costs to pop up here and there if we weren’t vigilant enough with housekeeping.

We liked to keep roughly 95% of all static infrastructure nicely tucked away in Terraform, but over the years you get so many small extant changes that the dirt begins to pile up and, along with it, costs. Just a little bit here and there, but the cumulative effect becomes more and more obvious as time moves on.

So, what do you do? Do you manually audit every account? Every region in every account? AWS doesn’t give you an easy way to view all resources across the entirety of your org outside of what you can glean from the billing console. So, we do what comes natural; find a way to make the glowing rectangle do the job for you.

The goal here isn’t to find a solution that does everything, but just enough to make our jobs easier.

Enter AWS Nuke

I found this utility sometime last year when asked to find a way to trim some extra fat off our monthly bill. It is excellent and works precisely as advertised. It wound up only saving us a few hundred bucks per month, which is a drop in the bucket considering our total monthly spend, but saving money isn’t the only benefit.

Added Security

Cleaning up unused resources is a low touch way to keeping up your security posture. Fewer forgotten things, means a smaller attack surface.

The longer you leave something unchecked, the harder it becomes to manage. Entropy affects everything and while code can be immutable, the world acting upon it is not.

Think back and ask yourself how many times you’ve forgotten about a service with ageing dependencies, or an EC2 instance you haven’t patched in a while, or some critical, but undocumented, component stored away in some forgotten area of one of your accounts.

If you don’t need it, or don’t plan on actively maintaining it, find a way to get rid of it.

Soft-Dollar Cost

People tend to forget the hidden cost of maintaining lots of things; it requires human attention. Have a lot of random things you need to manage? Well, the more things there are the more time you spend on them. That literally translates to money spent at the end of the billing period and it won’t be showing up on the invoice.

Wouldn’t you rather spend your time on more important things at work? Reduce hidden costs by cleaning up after yourself.

Ok, Let’s Nuke Some Stuff

First things first, determine the account id you’re going to use.

This is a highly-destructive operation, so make sure you’re targeting the right account. Get yourself some credentials either through privileged IAM or an SSO user session. Assume from here on out that 11111111111 is your target account and ensure your user has administrative privileges. You’ll need this level of authorization if you’re going to be doing this kind of deep cleaning.

Ensure your credentials are associated with the proper account by doing something similar to the following. As a side note, if you’re using a credentials sourced by SSO, you’re going to need the AWS_SESSION_TOKEN, otherwise it’s safe to ignore.

export AWS_ACCESS_KEY_ID="***"
export AWS_SECRET_ACCESS_KEY="***"
export AWS_SESSION_TOKEN="***"
aws sts get-caller-identity
{
    "UserId": "XXXXXXXXXXXXXXXX:DEADBEEF",
    "Account": "11111111111",
    "Arn": "arn:aws:sts::11111111111:assumed-role/AWSReservedSSO_AdministratorAccess/DEADBEEF"
}

Configuration

There may be resources you wish to keep while cleaning your account. AWS Nuke exposes comprehensive filtering capabilities that allow you to ignore, or target, specific sets of resources. In our case, it would be stuff like GuardDuty, AWS Config, CloudTrail and S3 buckets dedicated to Terraform state storage. For resources we don’t want to wipe, the following configuration provides a basic example of how you may bootstrap the cleaning process.

Ultimately, this will be highly specific to your unique use case, but it’s a great way to demonstrate how it all works. Let’s go through all the important sections:

---
regions:
  - global
  - eu-north-1
  - ap-south-1
  - eu-west-3
  - eu-west-2
  - eu-west-1
  - ap-northeast-3
  - ap-northeast-2
  - ap-northeast-1
  - sa-east-1
  - ca-central-1
  - ap-southeast-1
  - ap-southeast-2
  - eu-central-1
  - us-east-1
  - us-east-2
  - us-west-1
  - us-west-2

account-blocklist:
- 12345678910

resource-types:
  excludes:
    - CloudWatchAlarm
    - Route53ResolverRule
    - Route53HostedZone
    - S3Object
    - S3Bucket
    - GuardDutyDetector
    - SNSSubscription
    - IAMSAMLProvider
    - CloudTrailTrail

accounts:
  11111111111:
    filters:
      IAMPolicy:
        - type: contains
          value: AWS-Chatbot-NotificationsOnly-Policy
      IAMRolePolicy:
        - type: contains
          value: CloudTrailCloudWatchLogsRole
        - type: contains
          value: OrganizationAccountAccessRole
      IAMRole:
        - type: contains
          value: AWSReservedSSO
        - type: contains
          value: CloudWatchAlarmToSlackRole
        - CloudTrailCloudWatchLogsRole
        - OrganizationAccountAccessRole
      IAMRolePolicyAttachment:
        - type: contains
          value: AWSReservedSSO
        - type: contains
          value: CloudWatchAlarmToSlackRole
      SNSTopic:
        - type: contains
          value: CloudWatchAlarms
      CloudWatchLogsLogGroup:
        - CloudTrail/DefaultLogGroup

regions

Lists the AWS regions to screen for resources. By default, we allow the tool to cycle through all supported regions. Unfortunately, there is no support for “all regions”, so this is a list that will have to be manually maintained. Also, depending on the amount of resources, or use-case for this tool, you may not want to scan all regions. Trim this list to suit your purposes.

global refers to services which are considered global and not tied to any specific region. For example, IAM users, policies and roles.

account-blocklist

A list of AWS accounts to completely avoid. If you’re not confident in your account selections, this is a great place to put accounts that run production workloads as an added failsafe. Remember, be explicit in your selections and use this tool only in accounts you wish to clean.

resource-types.excludes

A list of AWS resources we wish to ignore. For us, these are typically associated with the resources we’ve primed using Terraform. In this case, we just skip them completely. Trim this list to suit your purposes, or just manually delete these resources in the AWS console if you’re unsure.

accounts

Contains the id of the account you wish to clean. Within this block, you will see filters used to skip certain resources. In our case, the filter almost always contains a list of things we wish to keep in all accounts. Resources like CloudWatch alarms, GuardDuty and AWS Config settings, etc…

You can read more about configuring aws-nuke in the associated repository’s README. Feel free to customise this file locally to suit your own needs. There are a number of ways to filter and target resources with a high degree of precision.

Using the Tool

By default, aws-nuke does not perform any destructive operations. You must explicitly add the --no-dry-run flag with a subsequent run after performing the initial scan of the target account.

You can install the tool locally on your machine, but I prefer using Docker to invoke the tool:

docker run --rm -it \
    -v ~/aws-nuke-config.yml:/home/aws-nuke/config.yml \
    -e AWS_ACCESS_KEY_ID \
    -e AWS_SECRET_ACCESS_KEY \
    -e AWS_SESSION_TOKEN \
    quay.io/rebuy/aws-nuke:main \
    --config /home/aws-nuke/config.yml

You will be asked to manually type the alias of the target account for confirmation. For our purposes, the alias for account 11111111111 is aws-nuke-account:

aws-nuke version v2.19.0.15.gb46fbe0 - Wed Oct  5 10:01:13 UTC 2022 - b46fbe0e9f63266f56b5afd9635b4e4d5a3108d4

Do you really want to nuke the account with the ID 11111111111 and the alias 'aws-nuke-account'?
Do you want to continue? Enter account alias to continue.
> aws-nuke-account

You will then see a list of AWS resources as aws-nuke cycles through each region:

global - IAMRole - RDSBucketAccessRole - [Name: "RDSBucketAccessRole", Path: "/service-role/"] - filtered by config
global - IAMRole - S3BucketAccessRole - [Name: "S3BucketAccessRole", Path: "/"] - would remove
global - IAMRole - EBSSnapshotReplicationRole - [Name: "EBSSnapshotReplicationRole", Path: "/"] - would remove
global - IAMRole - VPCFlowLogsRole - [Name: "VPCFlowLogsRole", Path: "/"] - would remove
global - IAMRole - OrganizationAccountAccessRole - [Name: "OrganizationAccountAccessRole", Path: "/"] - filtered by config
us-west-2 - EC2RouteTable - rtb-0000000000 - [tag:ManagedBy: "terraform", tag:Name: "VPCRouteTablePublic", tag:Purpose: "vpc-route-table"] - would remove
us-west-2 - EC2RouteTable - rtb-1111111111 - [tag:ManagedBy: "terraform", tag:Name: "VPCRouteTablePrivate", tag:Purpose: "vpc-route-table"] - would remove
Scan complete: 161 total, 83 nukeable, 78 filtered.

The above resources would be deleted with the supplied configuration. Provide --no-dry-run to actually destroy resources.

It is important during the dry run that you closely-examine each resource that has been flagged for removal. Here are some common flags you should be paying attention to:

1. would remove: these resources will be removed by AWS Nuke on a subsequent run with --no-dry-run.
2. filtered by config: these are resources that have been skipped as a result of your filtering configurations.
3. cannot delete *: marks AWS-managed resources that simply cannot be deleted.

If you notice something is marked as would remove, but should be kept, add a filter for it to the configuration file. Take an iterative approach until you’re satisfied with the scanning results.

Once you have scanned the list and updated any relevant filters, you can execute the nuking process by passing the --no-dry-run flag. You will be presented with the same steps as above as well as a final confirmation message similar to the initial dry run scan.

In Conclusion …

This is a very destructive process that will permanently remove selected resources. Always make sure you have backups of selected data sources and they are properly-filtered within your configuration file.

There may also be times when resources just aren’t deleted as expected. That’s ok as this process isn’t 100% perfect. AWS Nuke is still under heavy development and AWS updates APIs around various services constantly. The point isn’t for this tool to do 100% of the heavy lifting, but to get you mostly there.

90% complete still means you’ve successfully avoided 90% of the work. 😊

100% Completion Speed-Run

Just stop paying the bill. Seriously. Just stop paying for the account. Cancel the card. Delete those billing alerts and just walk away. If you stop paying, AWS will eventually just tear down your account with everything in it. All of it. The entire account. Credit score be damned!

obviously, do not do this… Or, go nuts! After all, don’t you sometimes wish you could burn it all down and just walk away? j/k

Hope you’ve learned something useful!

Behave! (mostly) 🙉🙈🙊

For this particular service, we have an ETL process that stores raw data points for downstream ingestion by some other… thing. Anyway, the storage mechanism for this is a cross-region replicated S3 bucket containing just over 10 million objects.

All associated buckets and objects must be deleted. 😬

Have you ever had to delete a bucket with this many objects? Not exactly straight-forward. Even by AWS standards.

What’s the problem?

As you may or may not know, you can’t delete a bucket that still contains objects. For smaller buckets, this isn’t much of an issue as you can saunter on over to the “empty bucket” screen. This will take you to a confirmation interstitial where you have to sit and wait until the deletion process finishes. Though, you had better not close that tab unless you want to start the process all over.

Not exactly a feasible solution for our use case. No, for larger buckets there really is only one pragmatic solution.

Lifecycle Policies!

You may have noticed the tone of this article as being somewhat snarky. Well, that’s because what follows shouldn’t be nearly as convoluted as it is. This process should be a single button with a confirmation step where the work is done asynchronously.

That being said, I know better than to question another team’s design decisions. Who knows what weirdness they encountered during implementing this functionality. In all fairness, they do provide the mechanisms necessary drop millions, or even billions, of objects. The main concern is it’s just not at all intuitive.

So, let’s get to it! 🦾

Expire those objects!

Alright, we’re going to have to perform the following steps for the first policy:

1. Expire all current objects.
2. Permanently delete all non-current versions of objects.
3. Delete expired object delete markers.

First and foremost, give this policy and name and confirm your intent:

Select the following actions. We will need to select the last option in the list, but due to conflicting settings you will have to add it in separate, secondary, lifecycle policy:

Set the following option to 1 day:

Again, set this option to 1 day as well. Unfortunately, this is the lowest value you can select. You also have the option to specify how many versions of each object to keep. This is only really relevant if you have object versioning activated for this bucket, but you’re going do nuke it all, so just go ahead and leave this blank:

Look at you! All done except looking over your work and saving your new lifecycle policy:

Delete those objects!

A secondary policy is necessary to delete the objects you’ve just told AWS to expire. Begin the process by clicking the Create lifecycle rule button and performing the first step from the previous policy:

Next, you’ll select the last action in the list:

Select the options to delete all expired objects. Optionally, you can also delete impartial uploads older than 1 day as well.

And that’s it! Click Save and view the summary screen. It should look similar to this:

And now… we wait.

It will probably take an additional day for all pre-existing objects to be marked as “expired”. Next, AWS will trigger the lifecycle policies at 12am UTC. Depending on the size of your bucket, it could possibly take several days to empty. Just go on an extended ☕️ break.

Something to keep in mind is that you will not be charged for storage that has been marked as expired while AWS empties the associated bucket.

And that’s that!

Look, this works. But, it isn’t exactly intuitive. In my opinion, you shouldn’t have to google “how to empty large S3 bucket” or even go out of your way to create these policies. Ideally, this would all be neatly abstracted away from you and handled with a single button click and a confirmation interstitial ( one that you can walk away from ).

Anyway, it is what it is. I’ll keep checking back over the next few days to chase up its progress.

Hope this helped! Happy hacking! 🤘

Several days later…

Ok, it’s been a few days and we’re finally seeing progress on one of the buckets in question:

All it takes is a little bit of patience.

Wanna see a neat trick?

Open up your terminal of choice and invoke this dig command:

dig txt contact.wilhelm.codes +short

You should see, hopefully 🤞, the following block of text as a response:

"twitter: @wilhelm"
"github:  wilhelm-murdoch"
"keybase: 7F89 5036 2816 06F6"
"blog:    https://wilhelm.codes/"

What you’re looking at.

Simply put, you’re looking at my contact details using a simple DNS TXT record lookup. There are heaps of online tools out there to help you directly query DNS records, but dig tends to be the most common utility included included in most Linux distributions as well as MacOS.

Why might you do this?

Originally, TXT records were meant to allow DNS administrators to attach simple plain text human-readable “notes” to their zones. A bit later, it became common practice to use this flexible record type as a way to verify ownership of a domain.

If you’ve spent any time in an ops-related position, you’ve had to add these records when using custom domain names while integrating with some 3rd-party service. Especially, if you’ve ever wanted to configure an email service provider, like Google’s Gmail, to appear as if you’re sending emails from a personal domain name.

So, why not also create personal, human-readable way of declaring ownership over a domain as well? Like a globally-accessible low-tech “business card”.

DNS is effectively a globally-distributed, (mostly) fault-tolerant, eventually-consistent key / value store. These details can be easily looked up and verified by anyone with an Internet connection.

And as for “Why?”, why not?

Here’s you would implement it.

Honestly, there’s nothing special about how to implement this. It’s just another record. In my case, I ended up creating a separate TXT record per contact detail. Depending on which server software your DNS provider is using, you may or may not be able to create multi-line text blocks. So, your safest bet is just multiple one-liners attached to the same CNAME.

I recently migrated my zone for wilhelm.codes from Vultr to CloudFlare and this is pretty much what it all looks like:

Obviously, this interface will be different if you use a different service, but that’s all there is to it!

That’s it!

Yep. This isn’t groundbreaking stuff and I’m surely not the first person to do something like this. Just a bit of (mostly) harmless DNS fun!

But, good or bad, far too many of them will pass over your resume for the wildest reasons. We’re not here to go over all of them. No, today we’re going to address the somewhat recent trend of glancing at your public Github profile page and grading you based on how many green squares show up in your participation graph.

What’s your point?

Like basing your performance on LoC, using “commits made” as a recruiting metric is beyond useless. There can be any number of reasons why your graph isn’t as populated as they’d like:

1. You work on private repositories and you haven’t configured your profile to show anonymised private contributions.
2. You have an alternate account from your personal one to keep work and private life separate. Or, maybe it’s just company policy.
3. You’ve signed an NDA and can’t publish your contributions publicly.
4. You set personal boundaries for your work-life balance and don’t care to stare at glowing rectangles in your free time.
5. Or, maybe you’ve just had a slow year? 🤷

Personally, I have thousands of commits that will never see the light of day for some of the ☝️ points. If I’m out job hunting, I would hate to be passed over because I don’t tick some mundane box on an arbitrary checklist.

What to do?

Well, we light up that graph like it’s a switch board, of course. Make this information utterly useless. What do they care if your profile says you’ve had 2,345 commits in the last week? Maybe you did. If it’s 🟩 , they cross it off their checklist and move on to something else.

Ok, show me.

Crack your knuckles and get ready to do some coding. For this little project, we’ll be doing some simple Bash scripting. I’m currently working on MacOS, but this code should be fairly portable on other Linux-based distributions.

You will, naturally, need a Github account. Let’s start there.

Configure Your Participation Graph

Did you know you can configure your participation graph to display anonymised private commit statistics? Ensure that setting is ✅ and move on to the next step.

New Private Repository

Let’s create a new empty repository entitled graph-participation and make sure it’s private.

Checkout Local Working Copy

Follow the post-setup instructions Github gives you to check out a local working copy of your new 🔒 repository!

echo "Let's goooooooo!" >> README.md
git init
git add README.md
git commit -m "The first and final legitimate commit in this repository ..."
git branch -M main
git remote add origin git@github.com:<account>/github-participation.git
git push -u origin main

Of course, replace <account> with the name of your Github account.

Writing Some Code

Here comes the best part! What, you didn’t think you’d walk away from this article without writing a bit of code, did you?

This script is super-simple. There’s no need for elegance here as we’re probably only ever going to run this code once or twice. I’ll show you the full snippet and then walk you through the details.

start=2021-05-01
stop=2021-05-09
max=20
while [[ "${start}" != "${stop}" ]]; do 
  start=$(gdate -I -d "${start} + 1 day")
  for ((run=1; run <= $((RANDOM % max + 1)); run++)); do
    openssl rand -hex 32 > touch.txt
    git add -A
    export GIT_AUTHOR_DATE="${start}T00:01"
    export GIT_COMMITTER_DATE="${GIT_AUTHOR_DATE}"
    git commit -m "Touch #${run} for ${GIT_AUTHOR_DATE}"
  done
done

First things first, we only need to know three things to get started:

1. start: The day of our first “commit” in the format of YYYY-MM-DD.
2. stop: The day of our last “commit” in the format of YYYY-MM-DD.
3. max: We don’t want a single commit per day. Otherwise, the graph will be all one colour and not at all random enough to look legitimate. So, to account for this, we randomly create between 1 and 20 commits per day.

We start a while loop that keeps going until our start and stop strings are no longer different. With each iteration, we add 1 day to our start variable. Eventually, this date will increment until both start and stop strings are equal, thus ending the loop.

For each day iteration, we create another loop that randomly-iterates anywhere between 1 and max times. Within each nested iteration we replace the content of the file named touch.txt with a random value making it a unique commit.

We then stage this change and assign a date value to both the GIT_AUTHOR_DATE and GIT_COMMITTER_DATE environmental variables. These are what allow you to easily back date a commit.

Finally, we commit this unique change locally. This keeps happening for every day until the script stops.

Write this to an executable file, or just paste it into your terminal of choice and hit enter. If you see the following kind of log output, you know you’re doing the right thing:

[main 2ebb5b9] Touch #1 2021-05-07T00:01
 1 file changed, 1 insertion(+), 1 deletion(-)
[main e104b0a] Touch #2 2021-05-07T00:01
 1 file changed, 1 insertion(+), 1 deletion(-)
[main 873fa39] Touch #3 2021-05-07T00:01
 1 file changed, 1 insertion(+), 1 deletion(-)
[main 9def564] Touch #4 2021-05-07T00:01
 1 file changed, 1 insertion(+), 1 deletion(-)
[main 1b83187] Touch #1 2021-05-08T00:01
 1 file changed, 1 insertion(+), 1 deletion(-)
[main b9f00db] Touch #2 2021-05-08T00:01
 1 file changed, 1 insertion(+), 1 deletion(-)

Now, you simply have to push all your changes up to your repository on Github!

git push origin main

All you now have to do is sit back and wait for everything to update.

We’ve gone from barren landscape:

To a verdant garden paradise:

And that’s it! 🎉

If you’ve made a mistake, simply delete your private repository and try again with a fresh one.

Any final thoughts?

Job hunting is hard enough. Especially with more and more companies becoming increasingly risk-averse in their interviewing and recruiting processes. It’s my hope that this little bit of harmless mischief will make things a bit easier for you!