Skip to content

Commit

Permalink
app-arch/zstd: multithreaded compression by default
Browse files Browse the repository at this point in the history
Integration of Clear Linux's 'multi-thread-default.patch'

By default, zstd uses one core for compression. This patch
makes zstd use all physical cores detected for compression,
increasing performance and reducing compression time.

Below's results are from using zstd's built-in benchmark,
showing a decrease of 78.13% in compression time with -T0.

Default (-T1)
	19#linux-5.8.tar     : 983869440 -> 121381009 (8.106),  4.05 MB/s ,1771.3 MB/s
	zstd -T1 -b19 -i0 --priority=rt linux-5.8.tar  244.99s user 0.46s system 99% cpu 4:05.47 total

Patched default (-T0)
	19#linux-5.8.tar     : 983869440 -> 121384544 (8.105),  19.2 MB/s ,1756.7 MB/s
	zstd -T0 -b19 -i0 --priority=rt linux-5.8.tar  297.19s user 0.63s system 554% cpu 53.692 total

Source: https://github.com/clearlinux-pkgs/zstd/blob/1.4.5-60/multi-thread-default.patch
References: InBetweenNames#164
  • Loading branch information
jiblime committed Sep 11, 2020
1 parent dbeefd6 commit 076e587
Showing 1 changed file with 39 additions and 0 deletions.
Original file line number Diff line number Diff line change
@@ -0,0 +1,39 @@
diff --git a/programs/zstdcli.c b/programs/zstdcli.c
index 70e2b70..ebd5b33 100644
--- a/programs/zstdcli.c
+++ b/programs/zstdcli.c
@@ -695,7 +695,7 @@ int main(int const argCount, const char* argv[])
if ((filenames==NULL) || (file_of_names==NULL)) { DISPLAY("zstd: allocation error \n"); exit(1); }
programName = lastNameFromPath(programName);
#ifdef ZSTD_MULTITHREAD
- nbWorkers = 1;
+ nbWorkers = 0;
#endif

/* preset behaviors */
diff --git a/programs/README.md b/programs/README.md
index 53706de..c1b032f 100644
--- a/programs/README.md
+++ b/programs/README.md
@@ -168,7 +168,7 @@ Advanced arguments :
--stream-size=# : optimize compression parameters for streaming input of given number of bytes
--size-hint=# optimize compression parameters for streaming input of approximately this size
--target-compressed-block-size=# : make compressed block near targeted size
- -T# : spawns # compression threads (default: 1, 0==# cores)
+ -T# : spawns # compression threads (default: 0==# cores)
-B# : select size of each job (default: 0==automatic)
--rsyncable : compress using a rsync-friendly method (-B sets block size)
--no-dictID : don't write dictID into header (dictionary compression)
diff --git a/programs/zstdcli.c b/programs/zstdcli.c
index 70e2b70..4a872fe 100644
--- a/programs/zstdcli.c
+++ b/programs/zstdcli.c
@@ -160,7 +160,7 @@ static void usage_advanced(const char* programName)
DISPLAYOUT( "--fast[=#]: switch to very fast compression levels (default: %u) \n", 1);
DISPLAYOUT( "--adapt : dynamically adapt compression level to I/O conditions \n");
# ifdef ZSTD_MULTITHREAD
- DISPLAYOUT( " -T# : spawns # compression threads (default: 1, 0==# cores) \n");
+ DISPLAYOUT( " -T# : spawns # compression threads (default: 0==# cores) \n");
DISPLAYOUT( " -B# : select size of each job (default: 0==automatic) \n");
DISPLAYOUT( "--single-thread : use a single thread for both I/O and compression (result slightly different than -T1) \n");
DISPLAYOUT( "--rsyncable : compress using a rsync-friendly method (-B sets block size) \n");

0 comments on commit 076e587

Please sign in to comment.