Skip to content

GNU Parallel

To execute arbitrary commands in parallel (multi-process), making use of additional CPU cores, across input data, GNU parallel can be used.

A basic example is as follows:

find . -name '*.html' | parallel gzip --best

and more examples can be found here.

For simple use cases, particularly where you do not need to collate stdout, xargs with the -P option may also be used.