Unconventional pipe captures

We often use pipes right away:

printf thequux | sed "s/^.../& /"  | tr tq TQ   #⇒ The Quux
#             │                  ⬆│          ⬆
#             └──────────────────┘└──────────┘

and, as above, it's usually implicit that the incoming contents come last, although some functions do require the use of - as a placeholder — which others, such as paste, can use repeatedly:

seq -w 42 | paste -d\  - - - - - - -

01 02 03 04 05 06 07
08 09 10 11 12 13 14
15 16 17 18 19 20 21
22 23 24 25 26 27 28
29 30 31 32 33 34 35
36 37 38 39 40 41 42

But what if we wanted the input to be somewhere other than at the last position?

Ok, so now we're piping not thequux, but the sed expression itself. How do we use it?

The simplest recipe is this:

printf "s/^.../& /" | sed "$(cat)" <<< thequux | tr tq TQ   #⇒ The Quux
#                  │          ⬆               │          ⬆
#                  └──────────┘               └──────────┘

See, the input of that sed command comes from the Here string <<< thequux.

But that cat is not complete: cat what? So the pipe's contents go there, and they're processed just like any regular Command Substitution.

Purists who don't like piping to cat can do this — same thing, but faster, but also noisier:

printf "s/^.../& /" | sed "$(</dev/stdin)" <<< thequux | tr tq TQ   #⇒ The Quux
#                  │          ⬆                       │          ⬆
#                  └──────────┘                       └──────────┘

Here's another example. Suppose we wanted The Quux to be blue. How?

Either this printf or this echo -e would do it:

BLUE="\e[0;34m"
NOCOLOR="\e[0m"

# These work
printf "$BLUE%s$NOCOLOR\n" "$(printf thequux | sed "s/^.../& /"  | tr tq TQ)"
echo -e "$BLUE$(printf thequux | sed "s/^.../& /"  | tr tq TQ)$NOCOLOR"

But none of the printf or echo below would work, because neither of them accepts "implicit pipe for missing last element", nor do they recognize - as its placeholder:

BLUE="\e[0;34m"
NOCOLOR="\e[0m"

# None of these work
printf thequux | sed "s/^.../& /"  | tr tq TQ | printf "$BLUE%s$NOCOLOR\n"
printf thequux | sed "s/^.../& /"  | tr tq TQ | printf "$BLUE%s$NOCOLOR\n" -
printf thequux | sed "s/^.../& /"  | tr tq TQ | echo -e "$BLUE"
printf thequux | sed "s/^.../& /"  | tr tq TQ | echo -e "$BLUE" -

Yet you want to use the pipe! You want to pipe to printf or echo. How?

BLUE="\e[0;34m"
NOCOLOR="\e[0m"

# These work
printf thequux | sed "s/^.../& /"  | tr tq TQ  | printf "$BLUE%s$NOCOLOR\n" "$(cat)"
printf thequux | sed "s/^.../& /"  | tr tq TQ  | echo -e "${BLUE}$(cat)${NOCOLOR}"
#             │                  ⬆│          ⬆│                     ⬆
#             └──────────────────┘└──────────┘└─────────────────────┘

The "cat solution" minimizes the amount of stuff that you see inside $(…).

But if you want to avoid a "useless cat" and you don't like the large </dev/stdin, and you prefer the shortest possible line, and you don't mind using the last part of your pipe, then you can replace the above with:

BLUE="\e[0;34m"
NOCOLOR="\e[0m"

# These work
printf thequux | sed "s/^.../& /"  | printf "$BLUE%s$NOCOLOR\n" "$(tr tq TQ)"
printf thequux | sed "s/^.../& /"  | echo -e "${BLUE}$(tr tq TQ)${NOCOLOR}"
#             │                  ⬆│                     ⬆
#             └──────────────────┘└─────────────────────┘

Here's a progression from maximum to minimum stuff inside "$(…)" (Command Substitution):

BLUE="\e[0;34m"
NOCOLOR="\e[0m"

# All of these work
echo -e "${BLUE}$(printf thequux | tr tq TQ | sed "s/^.../& /")${NOCOLOR}"
printf thequux | echo -e "${BLUE}$(tr tq TQ | sed "s/^.../& /")${NOCOLOR}"
printf thequux | sed "s/^.../& /" | echo -e "${BLUE}$(tr tq TQ)${NOCOLOR}"
printf thequux | sed "s/^.../& /" | tr tq TQ | echo -e "${BLUE}$(cat)${NOCOLOR}"

You'd normally use the first one, right?

You then progressively move the first piped expression to "the outside", until you run out of | symbols inside it. Up to there, the length would have been maintained. Moving the last one to the outside comes at the "cost" of an extra " | cat" and six characters. This is less "economical", and the cat is "useless" — but it's also arguably more generalizable (you can always use the cat), and therefore more readable and universably recognizable, and it's also "cleaner" in that it minimizes stuff inside "$(…)", which tends to look noisy.

Now you have more stuff to do with The Quux. In particular, you'll use it more than once.

But there's a snag:

Everything flows and nothing stays.
You cannot step twice into the same Bash pipe.
— Heraclitus, 5th century BC

So outside some exceptional cases where using tee is feasible (and not too convoluted), you'll want to do the cat trick inside braces — a group command:

printf thequux | sed "s/^.../& /" | tr tq TQ |
    { qx=$(cat)
      cut -c7- <<<"\
      ${qx: -4}ian Ventures specialize in cutting-edge operations
      to leverage your ${qx: -4} capitalization requirements.
      We're market leaders in:
      - ${qx^^}
      - ${qx^}
      - ${qx~}
      - ${qx~~}
      - ${qx,,}
      - ${qx,}
      - ${qx: -4}-related derivatives
      Sign up to our newsletter and receive
      the latest trends of ${qx,,}osphere!" ;}

echo "→${qx}←"

which should output:

Quuxian Ventures specialize in cutting-edge operations
to leverage your Quux capitalization requirements.
We're market leaders in:
- THE QUUX
- The Quux
- the Quux
- tHE qUUX
- the quux
- the Quux
- Quux-related derivatives
Sign up to our newsletter and receive
the latest trends of the quuxosphere!
→←

Notice the empty arrows at the end? A Bash trap: that's a pipeline, so the group command is in a subshell. Once you leave a subshell, any variables defined there are gone.

The canonical solution is to put the whole block at "top level" and use < <(…) (Process Substitution) as input.

So this would fix it: same output, but ending with →The Quux←:

{ qx=$(cat)
  cut -c3- <<<"\
  ${qx: -4}ian Ventures specialize in cutting-edge operations
  to leverage your ${qx: -4} capitalization requirements.
  We're market leaders in:
  - ${qx^^}
  - ${qx^}
  - ${qx~}
  - ${qx~~}
  - ${qx,,}
  - ${qx,}
  - ${qx: -4}-related derivatives
  Sign up to our newsletter and receive
  the latest trends of ${qx,,}osphere!"
} < <(printf thequux | sed "s/^.../& /" | tr tq TQ)

echo "→${qx}←"

Incidentally, this is also the preferable structure for processing lines in a loop when you want to preserve for later use some variable that you modified there:

x=()
while read -r line
do x+=("tag$line")
done < <(seq -w 10)
printf "%s:" ":${x[@]}"

:tag01:tag02:tag03:tag04:tag05:tag06:tag07:tag08:tag09:tag10:

What if we want to use the input in an if…then…fi?

The idea is similar to group commands — but we can add tests.

To recap:

• First, we saw a way to capture the pipe in a position other than last item.
  
• Then we saw that this also works with functions that can't do "implicit last item capture", such as printf and echo.
  
• Then we saw how to capture at once the contents of the last pipe, and then use that repeatedly, at our leisure.
  
• Then we saw how to split that into more than one variable, using read.
  
• Then we saw how to ditch the variable using : and $_, which is fine when the contents are used immediately.
  
• Then we saw how to ditch the braces by assigning to the captured pipe: "${quux=$(cat)}".
  
• Then we saw how to capture in an if…then…fi — and it'd also work with case…esac and others.
  

In all these cases, we captured the pipe into the first command that showed up. Sort of.

A question remains: what if we don't want to capture that pipe immediately?

Answer: it'll wait for you. Leave it clogged for a while. Take your time.

printf thequux | sed "s/^.../& /" | tr tq TQ |
    { Sp=sphere
      echo "Welcome, Ladies and Gentlemen."
      echo "Glad that you could be here."
      echo "Make yourselves comfortable."
      ((foo++))
      echo "We'll get to our topics when it's time to."
      echo "It's there, waiting for us."
      ((bar--))
      echo "There's a lot to talk about — but no hurry:"
      echo "we can all breathe and take our time."
      sleep 10
      echo "(Could you bring me a glass of water? Thank you.)" >/dev/null
      echo "Everyone seated? Good. Let's start, then."; echo
      : "$(grep -o Q.*)"  # ←guess who's being captured down here?
      echo "$_ and the ${_,}o$Sp are tonight's" "topics"
      echo "and each ${_:: -1} has its charms." ;} # ^new $_

Welcome, Ladies and Gentlemen.
Glad that you could be here.
Make yourselves comfortable.
We'll get to our topics when it's time to.
It's there, waiting for us.
There's a lot to talk about — but no hurry:
we can all breathe and take our time.
Everyone seated? Good. Let's start, then.

Quux and the quuxosphere are tonight's topics
and each topic has its charms.