The possibility to combine pipelines basically allows to modularize the pipeline creation process. This is especially useful when you have a set of pipelines that are used in different contexts and you want to avoid code duplication1.
Let’s define one pipeline that is used for data preprocessing and one that does the modelling2.
Next we combine the two pipelines using rbind().
pip <- rbind(pip1, pip2)
pip
# <pipeflow> preprocess-model (6 steps)
# -------------------------------------
# step params depends state
# 1: data x new
# 2: prep x data new
# 3: standardize x,scale prep new
# 4: data2 x new
# 5: fit x,k,b data2 new
# 6: predict x fit new
# -------------------------------------
# <ready> last run: neverNote that the data step of the second pipeline has been
renamed to data2 in both the step and the
depends columns (see line 4 above). That is, when
“rbinding” pipelines, {pipeflow} automatically ensures that all step
names stay unique by renaming any duplicates accordingly.
As is also visible from the graphical representation of the pipeline,
library(visNetwork)
do.call(visNetwork, args = pip_graph(pip)) |>
visHierarchicalLayout(direction = "LR")the two pipelines are not yet connected. To make sense of the
combined pipeline, we want to use the output of the
standardize step as the input of the data2
step, which we can do by applying the replace function,
which was introduced in the previous vignette modify the pipeline, as follows:
pip |> pip_replace("data2", \(x = ~standardize) x)
pip
# <pipeflow> preprocess-model (6 steps)
# -------------------------------------
# step params depends state
# 1: data x new
# 2: prep x data new
# 3: standardize x,scale prep new
# 4: data2 x standardize new
# 5: fit x,k,b data2 new
# 6: predict x fit new
# -------------------------------------
# <ready> last run: neverThe data2 step points to the output of the
standardize step, so that both pipelines are now
connected.
Since the name of the re-routed step might not always be known3, the
{pipeflow} package also provides a relative position indexing mechanism,
which allows to rewrite the above command using a number (instead of the
step name standardize) while having the same effect as
above.
pip |> pip_replace("data2", \(x = ~ -1) x)
pip
# <pipeflow> preprocess-model (6 steps)
# -------------------------------------
# step params depends state
# 1: data x new
# 2: prep x data new
# 3: standardize x,scale prep new
# 4: data2 x standardize new
# 5: fit x,k,b data2 new
# 6: predict x fit new
# -------------------------------------
# <ready> last run: neverThe relative indexing mechanism allows to refer to steps positioned
above the current step. The index ~-1 can be interpreted as
“go one step back”, ~-2 as “go two steps back”, and so
on.
Instead of replacing the step afterwards, the two pipelines can also
be connected right away with the bind argument of
rbind(). It takes a named list of one-sided formulas, which
binds unbound params of the second pipeline to steps of the first
one:
pip_bound <- rbind(pip1, pip2, bind = list(x = ~standardize))
pip_bound
# <pipeflow> preprocess-model (6 steps)
# -------------------------------------
# step params depends state
# 1: data x new
# 2: prep x data new
# 3: standardize x,scale prep new
# 4: data2 x standardize new
# 5: fit x,k,b data2 new
# 6: predict x fit new
# -------------------------------------
# <ready> last run: neverThe param x of the data2 step now refers to
the standardize step, as if it had been defined as
\(x = ~standardize) x. The same holds for any other step of
the second pipeline with an unbound param x. Steps where
x already refers to another step, like fit and
predict, are left as they are, and so are the steps of the
first pipeline: the data step still has its param
x. Steps are bound by name, not by relative position, and
bind can only be used when combining exactly two
pipelines.
Params can also be bound after combining, with
pip_set_params(): a one-sided formula binds the param to
that step in all selected steps that have it unbound. Here the
data step of the first pipeline has an unbound param
x as well, but it comes before the standardize
step, so it can’t be bound to it. Therefore, we select the
data2 step with a view:
pip_later <- rbind(pip1, pip2)
pip_set_params(pip_later["data2"], list(x = ~standardize))
pip_later
# <pipeflow> preprocess-model (6 steps)
# -------------------------------------
# step params depends state
# 1: data x new
# 2: prep x data new
# 3: standardize x,scale prep new
# 4: data2 x standardize new
# 5: fit x,k,b data2 new
# 6: predict x fit new
# -------------------------------------
# <ready> last run: neverWithout the view, pip_set_params() would signal an error
and leave the pipeline unchanged. Like with bind, the steps
are bound by name, not by relative position.
Let’s now run the combined pipeline and inspect the results of the final step.
pip_run(pip)
# info [2026-10-20 10:26:00.407 UTC]: Starting run of pipeflow 'preprocess-model'
# info [2026-10-20 10:26:00.407 UTC]: Step 1/6 data
# info [2026-10-20 10:26:00.407 UTC]: Step 2/6 prep
# info [2026-10-20 10:26:00.409 UTC]: Step 3/6 standardize
# info [2026-10-20 10:26:00.409 UTC]: Step 4/6 data2
# info [2026-10-20 10:26:00.410 UTC]: Step 5/6 fit
# info [2026-10-20 10:26:00.411 UTC]: Step 6/6 predict
# info [2026-10-20 10:26:00.412 UTC]: Finished run of pipeflow 'preprocess-model'As we can see, the outputs of the preprocessing pipeline flow into
the modelling pipeline. We can now go ahead and for example change the
multiplier of the fit step and rerun the pipeline.
pip_run(pip)
# info [2026-10-20 10:26:00.483 UTC]: Starting run of pipeflow 'preprocess-model'
# info [2026-10-20 10:26:00.483 UTC]: Step 1/6 data - skipping done step
# info [2026-10-20 10:26:00.483 UTC]: Step 2/6 prep - skipping done step
# info [2026-10-20 10:26:00.483 UTC]: Step 3/6 standardize - skipping done step
# info [2026-10-20 10:26:00.483 UTC]: Step 4/6 data2 - skipping done step
# info [2026-10-20 10:26:00.483 UTC]: Step 5/6 fit
# info [2026-10-20 10:26:00.486 UTC]: Step 6/6 predict
# info [2026-10-20 10:26:00.488 UTC]: Finished run of pipeflow 'preprocess-model'A pipeline, like the combined one above, can be built once and reused
later, also in another R session. To this end, it can be saved to a file
with saveRDS() and read back with readRDS()4.
file <- tempfile(fileext = ".rds")
saveRDS(pip, file)
pip_restored <- readRDS(file)
pip_restored
# <pipeflow> preprocess-model (6 steps)
# -------------------------------------
# step params depends state out
# 1: data x done 1,2,3,4,5
# 2: prep x data done 2,3,4,5,6
# 3: standardize x,scale prep done 4, 6, 8,10,12
# 4: data2 x standardize done 4, 6, 8,10,12
# 5: fit x,k,b data2 done 12,18,24,30,36
# 6: predict x fit done pred: 12,pred: 18,pred: 24,pred: 30,pred: 36
# -------------------------------------
# <ready> last run: 2026-10-20 10:26:00The restored pipeline has all steps, parameters, outputs and states of the saved one. That is, it can be modified and run right away, and only the steps that need an update are run again:
pip_set_params(pip_restored, params = list(k = 4))
pip_run(pip_restored)
# info [2026-10-20 10:26:00.545 UTC]: Starting run of pipeflow 'preprocess-model'
# info [2026-10-20 10:26:00.545 UTC]: Step 1/6 data - skipping done step
# info [2026-10-20 10:26:00.545 UTC]: Step 2/6 prep - skipping done step
# info [2026-10-20 10:26:00.545 UTC]: Step 3/6 standardize - skipping done step
# info [2026-10-20 10:26:00.545 UTC]: Step 4/6 data2 - skipping done step
# info [2026-10-20 10:26:00.545 UTC]: Step 5/6 fit
# info [2026-10-20 10:26:00.546 UTC]: Step 6/6 predict
# info [2026-10-20 10:26:00.547 UTC]: Finished run of pipeflow 'preprocess-model'
pip_restored[["predict", "out"]]
# [1] "pred: 16" "pred: 24" "pred: 32" "pred: 40" "pred: 48"Both the saved and the restored pipelines are independent of each
other, so the original pip still uses the multiplier
k = 3:
For more details see also the section “Saving pipelines” in the help
of pip_new().
Whether you combine them or not, in practice pipelines can get long quickly. The next vignette shows how you can easily focus on certain parts of your entire analysis workflow by Pipeline views.
Note that code duplication is not bad per se and can even be preferable, since it reduces entanglement and improves local readability, or as Sandi Metz put it, “duplication is far cheaper than the wrong abstraction”.↩︎
The step functions in these pipelines are minimal to keep the focus on the combine functionality.↩︎
A typical example would be appending several pipelines in a programmatic context.↩︎
Other ways to serialize R objects work as well, for example to send a pipeline to another R process.↩︎