Skip to content

Commit 65d1f55

Browse files
author
Nick Terrell
committed
zstd: Import upstream v1.5.7
In addition to keeping the kernel's copy of zstd up to date, this update was requested by Intel to expose upstream's APIs that allow QAT to accelerate the LZ match finding stage of Zstd. This patch is imported from the upstream tag v1.5.7-kernel [0], which is signed with upstream's signing key EF8FE99528B52FFD [1]. It was imported from upstream using this command: export ZSTD=/path/to/repo/zstd/ export LINUX=/path/to/repo/linux/ cd "$ZSTD/contrib/linux-kernel" git checkout v1.5.7-kernel make import LINUX="$LINUX" This patch has been tested on x86-64, and has been boot tested with a zstd compressed kernel & initramfs on i386 and aarch64. I benchmarked the patch on x86-64 with gcc-14.2.1 on an Intel i9-9900K by measruing the performance of compressed filesystem reads and writes. Component, Level, Size delta, C. time delta, D. time delta Btrfs , 1, +0.00%, -6.1%, +1.4% Btrfs , 3, +0.00%, -9.8%, +3.0% Btrfs , 5, +0.00%, +1.7%, +1.4% Btrfs , 7, +0.00%, -1.9%, +2.7% Btrfs , 9, +0.00%, -3.4%, +3.7% Btrfs , 15, +0.00%, -0.3%, +3.6% SquashFS , 1, +0.00%, N/A, +1.9% The major changes that impact the kernel use cases for each version are: v1.5.7: https://github.com/facebook/zstd/releases/tag/v1.5.7 * Add zstd_compress_sequences_and_literals() for use by Intel's QAT driver to implement Zstd compression acceleration in the kernel. * Fix an underflow bug in 32-bit builds that can cause data corruption when processing more than 4GB of data with a single `ZSTD_CCtx` object, when an input crosses the 4GB boundry. I don't believe this impacts any current kernel use cases, because the `ZSTD_CCtx` is typically reconstructed between compressions. * Levels 1-4 see 5-10% compression speed improvements for inputs smaller than 128KB. v1.5.6: https://github.com/facebook/zstd/releases/tag/v1.5.6 * Improved compression ratio for the highest compression levels. I don't expect these see much use however, due to their slow speeds. v1.5.5: https://github.com/facebook/zstd/releases/tag/v1.5.5 * Fix a rare corruption bug that can trigger on levels 13 and above. * Improve compression speed of levels 5-11 on incompressible data. v1.5.4: https://github.com/facebook/zstd/releases/tag/v1.5.4 * Improve copmression speed of levels 5-11 on ARM. * Improve dictionary compression speed. Signed-off-by: Nick Terrell <terrelln@fb.com>
1 parent 7eb1721 commit 65d1f55

Some content is hidden

Large Commits have some content hidden by default. Use the searchbox below for content that may be hidden.

60 files changed

+8749
-4381
lines changed

include/linux/zstd.h

Lines changed: 82 additions & 5 deletions
Original file line numberDiff line numberDiff line change
@@ -1,6 +1,6 @@
11
/* SPDX-License-Identifier: GPL-2.0+ OR BSD-3-Clause */
22
/*
3-
* Copyright (c) Yann Collet, Facebook, Inc.
3+
* Copyright (c) Meta Platforms, Inc. and affiliates.
44
* All rights reserved.
55
*
66
* This source code is licensed under both the BSD-style license (found in the
@@ -160,7 +160,6 @@ typedef ZSTD_parameters zstd_parameters;
160160
zstd_parameters zstd_get_params(int level,
161161
unsigned long long estimated_src_size);
162162

163-
164163
/**
165164
* zstd_get_cparams() - returns zstd_compression_parameters for selected level
166165
* @level: The compression level
@@ -173,9 +172,20 @@ zstd_parameters zstd_get_params(int level,
173172
zstd_compression_parameters zstd_get_cparams(int level,
174173
unsigned long long estimated_src_size, size_t dict_size);
175174

176-
/* ====== Single-pass Compression ====== */
177-
178175
typedef ZSTD_CCtx zstd_cctx;
176+
typedef ZSTD_cParameter zstd_cparameter;
177+
178+
/**
179+
* zstd_cctx_set_param() - sets a compression parameter
180+
* @cctx: The context. Must have been initialized with zstd_init_cctx().
181+
* @param: The parameter to set.
182+
* @value: The value to set the parameter to.
183+
*
184+
* Return: Zero or an error, which can be checked using zstd_is_error().
185+
*/
186+
size_t zstd_cctx_set_param(zstd_cctx *cctx, zstd_cparameter param, int value);
187+
188+
/* ====== Single-pass Compression ====== */
179189

180190
/**
181191
* zstd_cctx_workspace_bound() - max memory needed to initialize a zstd_cctx
@@ -190,6 +200,20 @@ typedef ZSTD_CCtx zstd_cctx;
190200
*/
191201
size_t zstd_cctx_workspace_bound(const zstd_compression_parameters *parameters);
192202

203+
/**
204+
* zstd_cctx_workspace_bound_with_ext_seq_prod() - max memory needed to
205+
* initialize a zstd_cctx when using the block-level external sequence
206+
* producer API.
207+
* @parameters: The compression parameters to be used.
208+
*
209+
* If multiple compression parameters might be used, the caller must call
210+
* this function for each set of parameters and use the maximum size.
211+
*
212+
* Return: A lower bound on the size of the workspace that is passed to
213+
* zstd_init_cctx().
214+
*/
215+
size_t zstd_cctx_workspace_bound_with_ext_seq_prod(const zstd_compression_parameters *parameters);
216+
193217
/**
194218
* zstd_init_cctx() - initialize a zstd compression context
195219
* @workspace: The workspace to emplace the context into. It must outlive
@@ -424,6 +448,16 @@ typedef ZSTD_CStream zstd_cstream;
424448
*/
425449
size_t zstd_cstream_workspace_bound(const zstd_compression_parameters *cparams);
426450

451+
/**
452+
* zstd_cstream_workspace_bound_with_ext_seq_prod() - memory needed to initialize
453+
* a zstd_cstream when using the block-level external sequence producer API.
454+
* @cparams: The compression parameters to be used for compression.
455+
*
456+
* Return: A lower bound on the size of the workspace that is passed to
457+
* zstd_init_cstream().
458+
*/
459+
size_t zstd_cstream_workspace_bound_with_ext_seq_prod(const zstd_compression_parameters *cparams);
460+
427461
/**
428462
* zstd_init_cstream() - initialize a zstd streaming compression context
429463
* @parameters The zstd parameters to use for compression.
@@ -583,6 +617,18 @@ size_t zstd_decompress_stream(zstd_dstream *dstream, zstd_out_buffer *output,
583617
*/
584618
size_t zstd_find_frame_compressed_size(const void *src, size_t src_size);
585619

620+
/**
621+
* zstd_register_sequence_producer() - exposes the zstd library function
622+
* ZSTD_registerSequenceProducer(). This is used for the block-level external
623+
* sequence producer API. See upstream zstd.h for detailed documentation.
624+
*/
625+
typedef ZSTD_sequenceProducer_F zstd_sequence_producer_f;
626+
void zstd_register_sequence_producer(
627+
zstd_cctx *cctx,
628+
void* sequence_producer_state,
629+
zstd_sequence_producer_f sequence_producer
630+
);
631+
586632
/**
587633
* struct zstd_frame_params - zstd frame parameters stored in the frame header
588634
* @frameContentSize: The frame content size, or ZSTD_CONTENTSIZE_UNKNOWN if not
@@ -596,7 +642,7 @@ size_t zstd_find_frame_compressed_size(const void *src, size_t src_size);
596642
*
597643
* See zstd_lib.h.
598644
*/
599-
typedef ZSTD_frameHeader zstd_frame_header;
645+
typedef ZSTD_FrameHeader zstd_frame_header;
600646

601647
/**
602648
* zstd_get_frame_header() - extracts parameters from a zstd or skippable frame
@@ -611,4 +657,35 @@ typedef ZSTD_frameHeader zstd_frame_header;
611657
size_t zstd_get_frame_header(zstd_frame_header *params, const void *src,
612658
size_t src_size);
613659

660+
/**
661+
* struct zstd_sequence - a sequence of literals or a match
662+
*
663+
* @offset: The offset of the match
664+
* @litLength: The literal length of the sequence
665+
* @matchLength: The match length of the sequence
666+
* @rep: Represents which repeat offset is used
667+
*/
668+
typedef ZSTD_Sequence zstd_sequence;
669+
670+
/**
671+
* zstd_compress_sequences_and_literals() - compress an array of zstd_sequence and literals
672+
*
673+
* @cctx: The zstd compression context.
674+
* @dst: The buffer to compress the data into.
675+
* @dst_capacity: The size of the destination buffer.
676+
* @in_seqs: The array of zstd_sequence to compress.
677+
* @in_seqs_size: The number of sequences in in_seqs.
678+
* @literals: The literals associated to the sequences to be compressed.
679+
* @lit_size: The size of the literals in the literals buffer.
680+
* @lit_capacity: The size of the literals buffer.
681+
* @decompressed_size: The size of the input data
682+
*
683+
* Return: The compressed size or an error, which can be checked using
684+
* zstd_is_error().
685+
*/
686+
size_t zstd_compress_sequences_and_literals(zstd_cctx *cctx, void* dst, size_t dst_capacity,
687+
const zstd_sequence *in_seqs, size_t in_seqs_size,
688+
const void* literals, size_t lit_size, size_t lit_capacity,
689+
size_t decompressed_size);
690+
614691
#endif /* LINUX_ZSTD_H */

include/linux/zstd_errors.h

Lines changed: 20 additions & 10 deletions
Original file line numberDiff line numberDiff line change
@@ -1,5 +1,6 @@
1+
/* SPDX-License-Identifier: GPL-2.0+ OR BSD-3-Clause */
12
/*
2-
* Copyright (c) Yann Collet, Facebook, Inc.
3+
* Copyright (c) Meta Platforms, Inc. and affiliates.
34
* All rights reserved.
45
*
56
* This source code is licensed under both the BSD-style license (found in the
@@ -12,13 +13,18 @@
1213
#define ZSTD_ERRORS_H_398273423
1314

1415

15-
/*===== dependency =====*/
16-
#include <linux/types.h> /* size_t */
16+
/* ===== ZSTDERRORLIB_API : control library symbols visibility ===== */
17+
#define ZSTDERRORLIB_VISIBLE
1718

19+
#ifndef ZSTDERRORLIB_HIDDEN
20+
# if (__GNUC__ >= 4) && !defined(__MINGW32__)
21+
# define ZSTDERRORLIB_HIDDEN __attribute__ ((visibility ("hidden")))
22+
# else
23+
# define ZSTDERRORLIB_HIDDEN
24+
# endif
25+
#endif
1826

19-
/* ===== ZSTDERRORLIB_API : control library symbols visibility ===== */
20-
#define ZSTDERRORLIB_VISIBILITY
21-
#define ZSTDERRORLIB_API ZSTDERRORLIB_VISIBILITY
27+
#define ZSTDERRORLIB_API ZSTDERRORLIB_VISIBLE
2228

2329
/*-*********************************************
2430
* Error codes list
@@ -43,33 +49,37 @@ typedef enum {
4349
ZSTD_error_frameParameter_windowTooLarge = 16,
4450
ZSTD_error_corruption_detected = 20,
4551
ZSTD_error_checksum_wrong = 22,
52+
ZSTD_error_literals_headerWrong = 24,
4653
ZSTD_error_dictionary_corrupted = 30,
4754
ZSTD_error_dictionary_wrong = 32,
4855
ZSTD_error_dictionaryCreation_failed = 34,
4956
ZSTD_error_parameter_unsupported = 40,
57+
ZSTD_error_parameter_combination_unsupported = 41,
5058
ZSTD_error_parameter_outOfBound = 42,
5159
ZSTD_error_tableLog_tooLarge = 44,
5260
ZSTD_error_maxSymbolValue_tooLarge = 46,
5361
ZSTD_error_maxSymbolValue_tooSmall = 48,
62+
ZSTD_error_cannotProduce_uncompressedBlock = 49,
63+
ZSTD_error_stabilityCondition_notRespected = 50,
5464
ZSTD_error_stage_wrong = 60,
5565
ZSTD_error_init_missing = 62,
5666
ZSTD_error_memory_allocation = 64,
5767
ZSTD_error_workSpace_tooSmall= 66,
5868
ZSTD_error_dstSize_tooSmall = 70,
5969
ZSTD_error_srcSize_wrong = 72,
6070
ZSTD_error_dstBuffer_null = 74,
71+
ZSTD_error_noForwardProgress_destFull = 80,
72+
ZSTD_error_noForwardProgress_inputEmpty = 82,
6173
/* following error codes are __NOT STABLE__, they can be removed or changed in future versions */
6274
ZSTD_error_frameIndex_tooLarge = 100,
6375
ZSTD_error_seekableIO = 102,
6476
ZSTD_error_dstBuffer_wrong = 104,
6577
ZSTD_error_srcBuffer_wrong = 105,
78+
ZSTD_error_sequenceProducer_failed = 106,
79+
ZSTD_error_externalSequences_invalid = 107,
6680
ZSTD_error_maxCode = 120 /* never EVER use this value directly, it can change in future versions! Use ZSTD_isError() instead */
6781
} ZSTD_ErrorCode;
6882

69-
/*! ZSTD_getErrorCode() :
70-
convert a `size_t` function result into a `ZSTD_ErrorCode` enum type,
71-
which can be used to compare with enum list published above */
72-
ZSTDERRORLIB_API ZSTD_ErrorCode ZSTD_getErrorCode(size_t functionResult);
7383
ZSTDERRORLIB_API const char* ZSTD_getErrorString(ZSTD_ErrorCode code); /*< Same as ZSTD_getErrorName, but using a `ZSTD_ErrorCode` enum argument */
7484

7585

0 commit comments

Comments
 (0)