Parallel

    Allow to run command with different parameters simultaneously on CPU available

    Require parallel package

    Quick view

    Run echo and sleep under 4 cores

    parallel -j4 sleep {}\; echo {} ::: 2 1 4 3
    

    Semaphore

    In case you do not want to run your command as one, it is possible to use semaphore

    sem # equivalent to parallel --semaphore
    
    sem --jobs 2 --id id-for-sleep 'sleep 10s; echo "10s done"' # semaphore cannot benefit from :::
    sem --jobs 2 --id id-for-sleep sleep 5s\; echo "5s done"
    sem --wait --id id-for-sleep; echo "done" # wait all semaphore (commands) are done
    

    Note:

    • All options present in parallel are not necessarily guaranteed to work under sem, such as --joblog will be overwrite by each command from sem
    • --id ensure your working under the same semaphore (namespace)
    • --jobs need to be specified each time

    Go further

    See man parallel which got many examples