API Reference

This document provides the definitions for the optimization macros and functions newly added to the Kunpeng LZ4 library, which is optimized based on open-source LZ4. These macros and functions take effect only on the AArch64 platform and fall back to the original implementation on other platforms.

Macro and Function Description

Table 1 Kunpeng optimization macros and functions lists the optimization macros and functions.

Table 1 Kunpeng optimization macros and functions

Macro Definitions

KZL_MEMCPY Series

Macro Function

Replaces small-block memory copies in LZ4 compression and decompression with NEON SIMD instructions on the AArch64 platform to improve memory copy throughput, and falls back to the standard memcpy or __builtin_memcpy on non-AArch64 platforms or compilers that do not support GCC 4+.

Macro Definition

#if defined(__GNUC__) && (__GNUC__ >= 4)
#    define KZL_MEMCPY_2(dst, src, size) __builtin_memcpy(dst, src, size)
#    define KZL_MEMCPY_4(dst, src, size) __builtin_memcpy(dst, src, size)
#    define KZL_MEMCPY_8(dst, src, size) vst1_u8((dst), vld1_u8(src))
#    define KZL_MEMCPY_16(dst, src, size) vst1q_u8((dst), vld1q_u8(src))
#    define KZL_MEMCPY_32(dst, src, size) vst1q_u8((dst), vld1q_u8(src)); \
                                          vst1q_u8(((dst)+16), vld1q_u8(((src)+16)))
#endif

Parameters

Platform Behavior

KZL_FORCE_INLINE

Macro Function

Forces the compiler to inline critical optimization functions, eliminating function call overhead.

Macro Definition

#ifndef KZL_FORCE_INLINE
#  ifdef _MSC_VER
#    define KZL_FORCE_INLINE static __forceinline
#  else
#    if defined (__cplusplus) || defined (__STDC_VERSION__) && __STDC_VERSION__ >= 199901L
#      ifdef __GNUC__
#        define KZL_FORCE_INLINE static inline __attribute__((always_inline))
#      else
#        define KZL_FORCE_INLINE static inline
#      endif
#    else
#      define KZL_FORCE_INLINE static
#    endif
#  endif
#endif

Function Definitions

KZL_LittleEndianfastHash5

Function Usage

Calculates a 5-byte fast hash in little-endian mode, using pre-computed constants G_KZLPRIME5BYTES and SEQUENCE_MOVE to optimize multiplication and shift operations, thereby improving hash calculation performance.

Function Syntax

KZL_FORCE_INLINE uint32_t KZL_LittleEndianfastHash5(uint64_t sequence, uint8_t HashLogUsage)
{
    return (uint32_t)(((sequence << SEQUENCE_MOVE) * G_KZLPRIME5BYTES) >> (HASH_SIZE - HashLogUsage));
}

Parameters

Return Value

Returns the hash index value (uint32_t), which is used to find the matching position in the hash table.

Related Constants

KZL_BigEndianfastHash5

Function Usage

Calculates a 5-byte fast hash in big-endian mode, using pre-computed constants G_KZLPRIME8BYTES and SEQUENCE_MOVE to optimize multiplication and shift operations.

Function Syntax

KZL_FORCE_INLINE uint32_t KZL_BigEndianfastHash5(uint64_t sequence, uint8_t HashLogUsage)
{
    return (uint32_t)(((sequence >> SEQUENCE_MOVE) * G_KZLPRIME8BYTES) >> (HASH_SIZE - HashLogUsage));
}

Parameters

Return Value

Returns the hash index value (uint32_t).

Related Constants

LZ4_hashPositionIgnoreBufferLength

Function Usage

Calculates the hash position by ignoring the buffer length limit. Unlike the original LZ4_hashPosition, this function forces the use of a 5-byte hash (LZ4_hash5) in all scenarios (including those where sizeof(reg_t)==4) to improve matching accuracy on the AArch64 platform.

Function Syntax

LZ4_FORCE_INLINE U32 LZ4_hashPositionIgnoreBufferLength(const void* const p, tableType_t const tableType)
{
    return LZ4_hash5(LZ4_read_ARCH(p), tableType);
}

Parameters

Return Value

Returns the hash index value (U32).

Application Scenario

In the Kunpeng-optimized version, all invocations of LZ4_hashPosition are replaced with LZ4_hashPositionIgnoreBufferLength to ensure that a 5-byte hash is always used for better matching quality.

skipTrigger

Function Usage

Calculates an adaptive skip threshold. It dynamically selects the skip step during compression based on the input data size: using a larger step size (4) for small data (< 64 KB + 11 bytes) to reduce ineffective match attempts, and a smaller step size (2) for large data to accelerate match lookup.

Function Syntax

static void skipTrigger(int srcSize, uint8_t *skipStep)
{
    if (skipStep == NULL || srcSize <= 0) {
        return;
    }

    if (srcSize >= G_64KLIMIT) {
        *skipStep = 2;
    } else {
        *skipStep = 4;
    }
}

Parameters

Return Value

None. The result is output via the skipStep pointer.

Output Value Description

Related Constants

LZ4_wildCopy16

Function Usage

Performs a fast memory copy with a 16-byte step, used for the fast copy of literal data on the compression side. Compared to the original LZ4_wildCopy8 (8-byte step), it copies 16 bytes per operation to improve copy throughput.

Function Syntax

LZ4_FORCE_INLINE
void LZ4_wildCopy16(void* dstPtr, const void* srcPtr, void* dstEnd)
{
    BYTE* d = (BYTE*)dstPtr;
    const BYTE* s = (const BYTE*)srcPtr;
    BYTE* const e = (BYTE*)dstEnd;

    do { LZ4_memcpy(d,s,16); d+=16; s+=16; } while (d<e);
}

Parameters

Return Value

None

LZ4_wildCopy64

Function Usage

Performs a fast memory copy with a 64-byte step, used for the fast copy of large-block matched data on the decompression side. Compared to the original LZ4_wildCopy32 (32-byte step), it copies 64 bytes per operation (via four 16-byte copies) to significantly improve decompression throughput.

Function Syntax

LZ4_FORCE_INLINE void
LZ4_wildCopy64(void* dstPtr, const void* srcPtr, void* dstEnd)
{
    BYTE* d = (BYTE*)dstPtr;
    const BYTE* s = (const BYTE*)srcPtr;
    BYTE* const e = (BYTE*)dstEnd;

    do {
        LZ4_memcpy(d,s,16);
        LZ4_memcpy(d+16,s+16,16);
        LZ4_memcpy(d+32,s+32,16);
        LZ4_memcpy(d+48,s+48,16);

        d+=64; s+=64;
    } while (d<e);
}

Parameters

Return Value

None

Compilation Optimization Options

The patch in this repository introduces the following compilation optimization options to lib/Makefile.