GNU Parallel
To execute arbitrary commands in parallel (multi-process), making use of additional CPU cores, across input data, GNU parallel
can be used.
A basic example is as follows:
find . -name '*.html' | parallel gzip --best
and more examples can be found here.
For simple use cases, particularly where you do not need to collate stdout, xargs
with the -P
option may also be used.