Releases: shenwei356/rush
Releases · shenwei356/rush
rush v0.0.6
Changes:
- Avoid mixed line from multiple processes without loss of performance,
e.g. the first half of a line is from one process
and the last half of the line is from another process.
similar withparallel --line-buffer
Benchmark:
===============================[ a few stdout ]===============================
--------------------------[ keep order: False ]--------------------------
commands:
seq 1 10 | gargs -p 4 "echo job:{}; seq 1 10" > t.gargs
seq 1 10 | rush -j 4 "echo job:{}; seq 1 10" > t.rush
seq 1 10 | parallel -j 4 "echo job:{}; seq 1 10" > t.parallel
time:
gargs: 0.0380618572235
parallel: 0.197314977646
rush: 0.0267391204834
md5sum:
2fd84d0e03f9d4402584914b2ba1dba9 t.gargs
6d82dfa19a9f0546bb376b2042139ded t.parallel
b5aecd901fb76f894674ee4be7ae90b7 t.rush
---------------------------[ keep order: True ]---------------------------
commands:
seq 1 10 | gargs -p 4 -o "echo job:{}; seq 1 10" > t.gargs
seq 1 10 | rush -j 4 -k "echo job:{}; seq 1 10" > t.rush
seq 1 10 | parallel -j 4 -k "echo job:{}; seq 1 10" > t.parallel
time:
gargs: 0.0253150463104
parallel: 0.163496017456
rush: 0.023738861084
md5sum:
6d82dfa19a9f0546bb376b2042139ded t.gargs
6d82dfa19a9f0546bb376b2042139ded t.parallel
6d82dfa19a9f0546bb376b2042139ded t.rush
==============================[ lots of stdout ]==============================
--------------------------[ keep order: False ]--------------------------
commands:
seq 1 10 | gargs -p 4 "echo job:{}; seq 1 1000000" > t.gargs
seq 1 10 | rush -j 4 "echo job:{}; seq 1 1000000" > t.rush
seq 1 10 | parallel -j 4 "echo job:{}; seq 1 1000000" > t.parallel
time:
gargs: 0.22896194458
parallel: 0.301723957062
rush: 0.243381023407
md5sum:
edc13e1bcc9700fccfa524ef4c078688 t.gargs
509086b10292759096d6db278bb2add8 t.parallel
8b0d20a76dfa9db701ae850968796478 t.rush
---------------------------[ keep order: True ]---------------------------
commands:
seq 1 10 | gargs -p 4 -o "echo job:{}; seq 1 1000000" > t.gargs
seq 1 10 | rush -j 4 -k "echo job:{}; seq 1 1000000" > t.rush
seq 1 10 | parallel -j 4 -k "echo job:{}; seq 1 1000000" > t.parallel
time:
gargs: 0.231871128082
parallel: 0.297834157944
rush: 0.237619161606
md5sum:
509086b10292759096d6db278bb2add8 t.gargs
509086b10292759096d6db278bb2add8 t.parallel
509086b10292759096d6db278bb2add8 t.rush
==========================[ a few stdout & run long ]==========================
--------------------------[ keep order: False ]--------------------------
commands:
seq 1 10 | gargs -p 4 "echo job:{}; sleep 2; seq 1 10" > t.gargs
seq 1 10 | rush -j 4 "echo job:{}; sleep 2; seq 1 10" > t.rush
seq 1 10 | parallel -j 4 "echo job:{}; sleep 2; seq 1 10" > t.parallel
time:
gargs: 6.02603578568
parallel: 6.1913690567
rush: 6.02895188332
md5sum:
053741317f4adcee133daf736b58cd8e t.gargs
6d82dfa19a9f0546bb376b2042139ded t.parallel
b336503c7faefd2602a356d99f3647c8 t.rush
---------------------------[ keep order: True ]---------------------------
commands:
seq 1 10 | gargs -p 4 -o "echo job:{}; sleep 2; seq 1 10" > t.gargs
seq 1 10 | rush -j 4 -k "echo job:{}; sleep 2; seq 1 10" > t.rush
seq 1 10 | parallel -j 4 -k "echo job:{}; sleep 2; seq 1 10" > t.parallel
time:
gargs: 6.03132200241
parallel: 6.19274401665
rush: 6.03151106834
md5sum:
6d82dfa19a9f0546bb376b2042139ded t.gargs
6d82dfa19a9f0546bb376b2042139ded t.parallel
6d82dfa19a9f0546bb376b2042139ded t.rush
=========================[ lots of stdout & run long ]=========================
--------------------------[ keep order: False ]--------------------------
commands:
seq 1 10 | gargs -p 4 "echo job:{}; sleep 2; seq 1 1000000" > t.gargs
seq 1 10 | rush -j 4 "echo job:{}; sleep 2; seq 1 1000000" > t.rush
seq 1 10 | parallel -j 4 "echo job:{}; sleep 2; seq 1 1000000" > t.parallel
time:
gargs: 6.15697097778
parallel: 6.33681201935
rush: 6.17427492142
md5sum:
e1987d0d7528b9607ac848f7cf81c630 t.gargs
509086b10292759096d6db278bb2add8 t.parallel
bce691db6deafe5bfc37bd99fe8aea6b t.rush
---------------------------[ keep order: True ]---------------------------
commands:
seq 1 10 | gargs -p 4 -o "echo job:{}; sleep 2; seq 1 1000000" > t.gargs
seq 1 10 | rush -j 4 -k "echo job:{}; sleep 2; seq 1 1000000" > t.rush
seq 1 10 | parallel -j 4 -k "echo job:{}; sleep 2; seq 1 1000000" > t.parallel
time:
gargs: 6.16847395897
parallel: 6.28346395493
rush: 6.16706514359
md5sum:
509086b10292759096d6db278bb2add8 t.gargs
509086b10292759096d6db278bb2add8 t.parallel
509086b10292759096d6db278bb2add8 t.rush
rush v0.0.5
Changes:
- support safe interrupt by
Ctrl + C
- support continue jobs
- fix some bugs
- fix tests for lots of stdout
rush v0.0.4
Changes
- implement retry interval
- do not send empty data
- update docs
Benchmark
===============================[ a few stdout ]===============================
--------------------------[ keep order: False ]--------------------------
commands:
seq 1 10 | gargs -p 4 "echo job:{}; seq 1 10" > t.gargs
seq 1 10 | rush -j 4 "echo job:{}; seq 1 10" > t.rush
seq 1 10 | parallel -j 4 "echo job:{}; seq 1 10" > t.parallel
time:
gargs: 0.0380549430847
parallel: 0.17822599411
rush: 0.0223121643066
md5sum:
93cd6eb1699145ea257a3f275dd63137 t.gargs
6d82dfa19a9f0546bb376b2042139ded t.parallel
d1647624415980e211be32dd20897252 t.rush
---------------------------[ keep order: True ]---------------------------
commands:
seq 1 10 | gargs -p 4 -o "echo job:{}; seq 1 10" > t.gargs
seq 1 10 | rush -j 4 -k "echo job:{}; seq 1 10" > t.rush
seq 1 10 | parallel -j 4 -k "echo job:{}; seq 1 10" > t.parallel
time:
gargs: 0.015750169754
parallel: 0.171111106873
rush: 0.0181651115417
md5sum:
6d82dfa19a9f0546bb376b2042139ded t.gargs
6d82dfa19a9f0546bb376b2042139ded t.parallel
6d82dfa19a9f0546bb376b2042139ded t.rush
==============================[ lots of stdout ]==============================
--------------------------[ keep order: False ]--------------------------
commands:
seq 1 10 | gargs -p 4 "echo job:{}; seq 1 1000000" > t.gargs
seq 1 10 | rush -j 4 "echo job:{}; seq 1 1000000" > t.rush
seq 1 10 | parallel -j 4 "echo job:{}; seq 1 1000000" > t.parallel
time:
gargs: 0.23122215271
parallel: 0.299681186676
rush: 0.221569061279
md5sum:
b2393a3e1f9c3110bf1f8199819241b2 t.gargs
509086b10292759096d6db278bb2add8 t.parallel
73e8ccfb9e959ac0fb585677ce913cfb t.rush
---------------------------[ keep order: True ]---------------------------
commands:
seq 1 10 | gargs -p 4 -o "echo job:{}; seq 1 1000000" > t.gargs
seq 1 10 | rush -j 4 -k "echo job:{}; seq 1 1000000" > t.rush
seq 1 10 | parallel -j 4 -k "echo job:{}; seq 1 1000000" > t.parallel
time:
gargs: 0.200814008713
parallel: 0.266680955887
rush: 0.224990129471
md5sum:
509086b10292759096d6db278bb2add8 t.gargs
509086b10292759096d6db278bb2add8 t.parallel
509086b10292759096d6db278bb2add8 t.rush
==========================[ a few stdout & run long ]==========================
--------------------------[ keep order: False ]--------------------------
commands:
seq 1 10 | gargs -p 4 "echo job:{}; sleep 2; seq 1 10" > t.gargs
seq 1 10 | rush -j 4 "echo job:{}; sleep 2; seq 1 10" > t.rush
seq 1 10 | parallel -j 4 "echo job:{}; sleep 2; seq 1 10" > t.parallel
time:
gargs: 6.01939082146
parallel: 6.229626894
rush: 6.04141998291
md5sum:
c647d7da2ac998c64e430578f500d28c t.gargs
6d82dfa19a9f0546bb376b2042139ded t.parallel
3eea9f793a62c39d3ccaaac1c9f0ac69 t.rush
---------------------------[ keep order: True ]---------------------------
commands:
seq 1 10 | gargs -p 4 -o "echo job:{}; sleep 2; seq 1 10" > t.gargs
seq 1 10 | rush -j 4 -k "echo job:{}; sleep 2; seq 1 10" > t.rush
seq 1 10 | parallel -j 4 -k "echo job:{}; sleep 2; seq 1 10" > t.parallel
time:
gargs: 6.03719902039
parallel: 6.25482797623
rush: 6.03823709488
md5sum:
6d82dfa19a9f0546bb376b2042139ded t.gargs
6d82dfa19a9f0546bb376b2042139ded t.parallel
6d82dfa19a9f0546bb376b2042139ded t.rush
=========================[ lots of stdout & run long ]=========================
--------------------------[ keep order: False ]--------------------------
commands:
seq 1 10 | gargs -p 4 "echo job:{}; sleep 2; seq 1 1000000" > t.gargs
seq 1 10 | rush -j 4 "echo job:{}; sleep 2; seq 1 1000000" > t.rush
seq 1 10 | parallel -j 4 "echo job:{}; sleep 2; seq 1 1000000" > t.parallel
time:
gargs: 6.20727992058
parallel: 6.34050798416
rush: 6.20419406891
md5sum:
ff0b53be0e22040c52fb56c7988d2d33 t.gargs
509086b10292759096d6db278bb2add8 t.parallel
04569f3ea8722ea77c104ae13b1fd7f0 t.rush
---------------------------[ keep order: True ]---------------------------
commands:
seq 1 10 | gargs -p 4 -o "echo job:{}; sleep 2; seq 1 1000000" > t.gargs
seq 1 10 | rush -j 4 -k "echo job:{}; sleep 2; seq 1 1000000" > t.rush
seq 1 10 | parallel -j 4 -k "echo job:{}; sleep 2; seq 1 1000000" > t.parallel
time:
gargs: 6.21313786507
parallel: 6.33985590935
rush: 6.20952105522
md5sum:
509086b10292759096d6db278bb2add8 t.gargs
509086b10292759096d6db278bb2add8 t.parallel
509086b10292759096d6db278bb2add8 t.rush
rush v0.0.3
Changes:
- fix timeout and other bugs
- update docs
Benchmark
===============================[ a few stdout ]===============================
--------------------------[ keep order: False ]--------------------------
commands:
seq 1 10 | gargs -p 4 "echo job:{}; seq 1 10" > t.gargs
seq 1 10 | parallel -j 4 "echo job:{}; seq 1 10" > t.parallel
seq 1 10 | ./rush -j 4 "echo job:{}; seq 1 10" > t.rush
time:
./rush: 0.0178070068359
gargs: 0.0277509689331
parallel: 0.2383248806
md5sum:
084141a7f21738027aaa890db8c5aa7f t.rush
6d85acb23f9711f59797ad37ed7c0cd5 t.gargs
6d82dfa19a9f0546bb376b2042139ded t.parallel
---------------------------[ keep order: True ]---------------------------
commands:
seq 1 10 | gargs -p 4 -o "echo job:{}; seq 1 10" > t.gargs
seq 1 10 | parallel -j 4 -k "echo job:{}; seq 1 10" > t.parallel
seq 1 10 | ./rush -j 4 -k "echo job:{}; seq 1 10" > t.rush
time:
./rush: 0.0186898708344
gargs: 0.0156490802765
parallel: 0.157879829407
md5sum:
6d82dfa19a9f0546bb376b2042139ded t.rush
6d82dfa19a9f0546bb376b2042139ded t.gargs
6d82dfa19a9f0546bb376b2042139ded t.parallel
==============================[ lots of stdout ]==============================
--------------------------[ keep order: False ]--------------------------
commands:
seq 1 10 | gargs -p 4 "echo job:{}; seq 1 1000000" > t.gargs
seq 1 10 | parallel -j 4 "echo job:{}; seq 1 1000000" > t.parallel
seq 1 10 | ./rush -j 4 "echo job:{}; seq 1 1000000" > t.rush
time:
./rush: 0.198143959045
gargs: 0.158150911331
parallel: 0.275547027588
md5sum:
3fad4f0c1d9030b1588155dd73613734 t.rush
482b3565e3f56ef39e27c2a3ee8c30d6 t.gargs
509086b10292759096d6db278bb2add8 t.parallel
---------------------------[ keep order: True ]---------------------------
commands:
seq 1 10 | gargs -p 4 -o "echo job:{}; seq 1 1000000" > t.gargs
seq 1 10 | parallel -j 4 -k "echo job:{}; seq 1 1000000" > t.parallel
seq 1 10 | ./rush -j 4 -k "echo job:{}; seq 1 1000000" > t.rush
time:
./rush: 0.187384128571
gargs: 0.163018941879
parallel: 0.26899600029
md5sum:
509086b10292759096d6db278bb2add8 t.rush
509086b10292759096d6db278bb2add8 t.gargs
509086b10292759096d6db278bb2add8 t.parallel
==========================[ a few stdout & run long ]==========================
--------------------------[ keep order: False ]--------------------------
commands:
seq 1 10 | gargs -p 4 "echo job:{}; sleep 2; seq 1 10" > t.gargs
seq 1 10 | parallel -j 4 "echo job:{}; sleep 2; seq 1 10" > t.parallel
seq 1 10 | ./rush -j 4 "echo job:{}; sleep 2; seq 1 10" > t.rush
time:
./rush: 6.01994299889
gargs: 6.02390599251
parallel: 6.18043088913
md5sum:
7860c93e9c8c716d60c43b19d5242fd8 t.rush
949c4a40db1f7dd8a0c79d17fb0a8857 t.gargs
6d82dfa19a9f0546bb376b2042139ded t.parallel
---------------------------[ keep order: True ]---------------------------
commands:
seq 1 10 | gargs -p 4 -o "echo job:{}; sleep 2; seq 1 10" > t.gargs
seq 1 10 | parallel -j 4 -k "echo job:{}; sleep 2; seq 1 10" > t.parallel
seq 1 10 | ./rush -j 4 -k "echo job:{}; sleep 2; seq 1 10" > t.rush
time:
./rush: 6.03327798843
gargs: 6.02552580833
parallel: 6.16181492805
md5sum:
6d82dfa19a9f0546bb376b2042139ded t.rush
6d82dfa19a9f0546bb376b2042139ded t.gargs
6d82dfa19a9f0546bb376b2042139ded t.parallel
=========================[ lots of stdout & run long ]=========================
--------------------------[ keep order: False ]--------------------------
commands:
seq 1 10 | gargs -p 4 "echo job:{}; sleep 2; seq 1 1000000" > t.gargs
seq 1 10 | parallel -j 4 "echo job:{}; sleep 2; seq 1 1000000" > t.parallel
seq 1 10 | ./rush -j 4 "echo job:{}; sleep 2; seq 1 1000000" > t.rush
time:
./rush: 6.16020393372
gargs: 6.15334606171
parallel: 6.32215213776
md5sum:
12475a467cd794017740a6d6a728c1e0 t.rush
416c083522744f81c1f5913ceeed6237 t.gargs
509086b10292759096d6db278bb2add8 t.parallel
---------------------------[ keep order: True ]---------------------------
commands:
seq 1 10 | gargs -p 4 -o "echo job:{}; sleep 2; seq 1 1000000" > t.gargs
seq 1 10 | parallel -j 4 -k "echo job:{}; sleep 2; seq 1 1000000" > t.parallel
seq 1 10 | ./rush -j 4 -k "echo job:{}; sleep 2; seq 1 1000000" > t.rush
time:
./rush: 6.17208790779
gargs: 6.17165708542
parallel: 6.30735492706
md5sum:
509086b10292759096d6db278bb2add8 t.rush
509086b10292759096d6db278bb2add8 t.gargs
509086b10292759096d6db278bb2add8 t.parallel
rush v0.0.2
doc improvement
rush v0.0.1
first release
Benchmark
===============================[ a few stdout ]===============================
--------------------------[ keep order: False ]--------------------------
commands:
seq 1 10 | gargs -p 4 "echo job:{}; seq 1 10" > t.gargs
seq 1 10 | parallel -j 4 "echo job:{}; seq 1 10" > t.parallel
seq 1 10 | ./rush -j 4 "echo job:{}; seq 1 10" > t.rush
time:
./rush: 0.027722120285
gargs: 0.0211288928986
parallel: 0.159312009811
md5sum:
7d1725c9f30dbc320a18a415e3c0c88f t.rush
10a6a0db132d64b397287327f7c45ea7 t.gargs
6d82dfa19a9f0546bb376b2042139ded t.parallel
---------------------------[ keep order: True ]---------------------------
commands:
seq 1 10 | gargs -p 4 -o "echo job:{}; seq 1 10" > t.gargs
seq 1 10 | parallel -j 4 -k "echo job:{}; seq 1 10" > t.parallel
seq 1 10 | ./rush -j 4 -k "echo job:{}; seq 1 10" > t.rush
time:
./rush: 0.0264530181885
gargs: 0.0268969535828
parallel: 0.158892154694
md5sum:
6d82dfa19a9f0546bb376b2042139ded t.rush
6d82dfa19a9f0546bb376b2042139ded t.gargs
6d82dfa19a9f0546bb376b2042139ded t.parallel
==============================[ lots of stdout ]==============================
--------------------------[ keep order: False ]--------------------------
commands:
seq 1 10 | gargs -p 4 "echo job:{}; seq 1 1000000" > t.gargs
seq 1 10 | parallel -j 4 "echo job:{}; seq 1 1000000" > t.parallel
seq 1 10 | ./rush -j 4 "echo job:{}; seq 1 1000000" > t.rush
time:
./rush: 0.227620840073
gargs: 0.208966970444
parallel: 0.289571046829
md5sum:
da7cbb8b35e32bd7f1899d9b5991d0ec t.rush
6da7ddaa1b8558cddc0bf1db9af8b94a t.gargs
509086b10292759096d6db278bb2add8 t.parallel
---------------------------[ keep order: True ]---------------------------
commands:
seq 1 10 | gargs -p 4 -o "echo job:{}; seq 1 1000000" > t.gargs
seq 1 10 | parallel -j 4 -k "echo job:{}; seq 1 1000000" > t.parallel
seq 1 10 | ./rush -j 4 -k "echo job:{}; seq 1 1000000" > t.rush
time:
./rush: 0.248844146729
gargs: 0.209996938705
parallel: 0.301899909973
md5sum:
509086b10292759096d6db278bb2add8 t.rush
509086b10292759096d6db278bb2add8 t.gargs
509086b10292759096d6db278bb2add8 t.parallel
==========================[ a few stdout & run long ]==========================
--------------------------[ keep order: False ]--------------------------
commands:
seq 1 10 | gargs -p 4 "echo job:{}; sleep 2; seq 1 10" > t.gargs
seq 1 10 | parallel -j 4 "echo job:{}; sleep 2; seq 1 10" > t.parallel
seq 1 10 | ./rush -j 4 "echo job:{}; sleep 2; seq 1 10" > t.rush
time:
./rush: 6.03018307686
gargs: 6.03072595596
parallel: 6.23056292534
md5sum:
c64cd09ec47558c2b22beaa4c4df3d95 t.rush
85e992184c94624714423305410d2274 t.gargs
6d82dfa19a9f0546bb376b2042139ded t.parallel
---------------------------[ keep order: True ]---------------------------
commands:
seq 1 10 | gargs -p 4 -o "echo job:{}; sleep 2; seq 1 10" > t.gargs
seq 1 10 | parallel -j 4 -k "echo job:{}; sleep 2; seq 1 10" > t.parallel
seq 1 10 | ./rush -j 4 -k "echo job:{}; sleep 2; seq 1 10" > t.rush
time:
./rush: 6.03470206261
gargs: 6.02221393585
parallel: 6.26596903801
md5sum:
6d82dfa19a9f0546bb376b2042139ded t.rush
6d82dfa19a9f0546bb376b2042139ded t.gargs
6d82dfa19a9f0546bb376b2042139ded t.parallel
=========================[ lots of stdout & run long ]=========================
--------------------------[ keep order: False ]--------------------------
commands:
seq 1 10 | gargs -p 4 "echo job:{}; sleep 2; seq 1 1000000" > t.gargs
seq 1 10 | parallel -j 4 "echo job:{}; sleep 2; seq 1 1000000" > t.parallel
seq 1 10 | ./rush -j 4 "echo job:{}; sleep 2; seq 1 1000000" > t.rush
time:
./rush: 6.16191077232
gargs: 6.13517904282
parallel: 6.30131387711
md5sum:
4f64c55403019193c41d6d9b9bba38a5 t.rush
03d18998ca3071fa54c4121e2095207b t.gargs
509086b10292759096d6db278bb2add8 t.parallel
---------------------------[ keep order: True ]---------------------------
commands:
seq 1 10 | gargs -p 4 -o "echo job:{}; sleep 2; seq 1 1000000" > t.gargs
seq 1 10 | parallel -j 4 -k "echo job:{}; sleep 2; seq 1 1000000" > t.parallel
seq 1 10 | ./rush -j 4 -k "echo job:{}; sleep 2; seq 1 1000000" > t.rush
time:
./rush: 6.15442299843
gargs: 6.13263082504
parallel: 6.24226307869
md5sum:
509086b10292759096d6db278bb2add8 t.rush
509086b10292759096d6db278bb2add8 t.gargs
509086b10292759096d6db278bb2add8 t.parallel