(2025-10-20) Another buzzword debunked: workflow orchestration
--------------------------------------------------------------
What I especially "love" about tech buzzwords is how they make really simple
(and obvious) concepts sound complicated and important. Recently, we've been 
forced to use a particular "workflow orchestration" tool at work, which, to 
be honest, only has added more complexity to the codebase without any real 
benefit. My personal attempt to tackle the same problem, on the other hand, 
took under 20 SLOC of a shell script, but first, let's determine what the 
problem is. According to IBM:

> Workflow orchestration is the practice of coordinating multiple automated
tasks across business applications and services to help ensure seamless 
execution. 

If you consider this definition vague AF, you're not alone. However, what it
usually translates to is:

1. Your processes are divided into isolated tasks taking some input and
returning some output.
2. These tasks are chained together using workflows, which, again, take some
input and return some output.
3. The "workflow orchestration" tools just provide a "standardized" way of
writing and calling the workflow and task code.

If you still consider this nothing special, well... You're absolutely right.
If we look at this with clear eyes, that's essentially how any procedural 
programming is done, including scripting. Yet somehow, when it comes to 
calling multiple programs instead of one, that's not called scripting 
anymore, but "workflow orchestration". And my "orchestration framework" in 
18 SLOC of shell, called ShellFlow (name it flow.sh), just shows all the 
absurdity that's going on:

#!/bin/sh

# print to stderr
debugout() { printf '%s\n' "$*" >&2; }

# format the message and print to stdout
flowout() { printf '%s\t%s' "$1" "$2"; }

# use this in all task definitions
run-task() {
  input="$(cat)"
  wf="$(printf '%s' "$input" | cut -f 1)"
  body="$(printf '%s' "$input" | cut -f 2-)"
  flowout "$wf" "$(do_task "$wf" "$body")"
}

# ShellFlow entry point
sfrun() {
  SFLIB="$(readlink -f "$0")"
  wfbase="$(basename "$1")"
  WFDIR="$(readlink -f $(dirname -- "$1"))"
  cd "$WFDIR"
  debugout "Starting workflow $wfbase..."
  flowout $wfbase "$(cat)" | SFLIB="$SFLIB" WFDIR="$WFDIR" $SHELL $wfbase \
    | cut -f 2-
}

[ "$(basename "$0")" == "flow.sh" ] && sfrun $*

This tool accepts the workflow script file as the argument and any stdin data
as the workflow input. Then, we can create a workflow script like this, 
using the most natural thing for the purpose, Unix pipelines:

./task1 | ./task2 | ./task3 ...

And each task script, provided it's also shell-based, would look like this:

#!/bin/sh
. $SFLIB # source the library part of the script
do_task() { # accepts workflow name into $1 and all input into $2
   # your actions here
}
run-task

In the task, just print your output data as usual and print your debug info
into stderr (e.g. using the provided debugout shell function). You can also 
vary your task logic depending on the particular workflow name. In fact, if 
your task is not shell-based, just extract the first tab-separated field of 
stdin as the workflow name and treat the rest as the actual data input.

That's it. That's all there is to it. Everything else is bells and whistles
around the same concept. It's pretty straightforward to add a server side to 
call any of the workflows remotely if you need to. You can even add a UI 
with call statistics and success metrics, and so on. It won't change the 
fundamentals in the slightest. Yet there are so many "products" competing 
exactly around all those bells and whistles and not offering anything really 
new. Now, of course, half of those "products" also claim to be "AI-driven". 
I think those claims can, but not necessarily need to, be easily debunked 
too.

Keep it simple, stupid. Just use shell.

--- Luxferre ---