Skip to content

Conversation

@anthony-zy
Copy link
Contributor

This PR add RVV support for RV and optimized Memcopy64 function , improving compression speed by ~49.5%.

Optimize Snappy 1.2.2 performance

Added RVV support for RISC-V in Snappy, optimizing Memcopy64 by leveraging RVV vector load/store instructions (e.g., vle8_v_u8m1, vse8_v_u8m1) to reduce memory copy overhead and improve decompression performance. lzbench 2.1 tests onsilesia.tar (GCC 13.2.1, 64-bit Linux) show:
Compressor Compress Decompress Size Ratio
Snappy 1.2.2 (Before) 64.3 MB/s 186 MB/s 101403263 47.84%
Snappy 1.2.2 (After) 64.4 MB/s 278 MB/s 101403263 47.84%

Test Parameters:

  • Compression iterations (cIters): 1
  • Decompression iterations (dIters): 1
  • Compression time (cTime): 10.0s
  • Decompression time (dTime): 10.0s
  • Chunk size (chunkSize): 1706MB
  • Compression speed limit (cSpeed): 0MB

Notes:

  • Compression speed improved by ~49.5% (186 MB/s to 278 MB/s).
  • No change in decompression speed, size, or ratio, ensuring
    compatibility.
  • Environment: GCC 13.2.1, 64-bit Linux, silesia.tar.

Snappy 1.2.2 unittest ([ PASSED ] 21 tests.)

[==========] Running 21 tests from 3 test suites.
[----------] Global test environment set-up.
[----------] 1 test from CorruptedTest
[ RUN ] CorruptedTest.VerifyCorrupted
Crazy decompression lengths not checked on 64-bit build
[ OK ] CorruptedTest.VerifyCorrupted (14 ms)
[----------] 1 test from CorruptedTest (14 ms total)
[----------] 17 tests from Snappy
[ RUN ] Snappy.SimpleTests
[ OK ] Snappy.SimpleTests (22 ms)
[ RUN ] Snappy.AppendSelfPatternExtensionEdgeCases
[ OK ] Snappy.AppendSelfPatternExtensionEdgeCases (3 ms)
[ RUN ] Snappy.AppendSelfPatternExtensionEdgeCasesExhaustive
[ OK ] Snappy.AppendSelfPatternExtensionEdgeCasesExhaustive (4706 ms)
[ RUN ] Snappy.MaxBlowup
[ OK ] Snappy.MaxBlowup (10 ms)
[ DISABLED ] Snappy.DISABLED_MoreThan4GB
[ RUN ] Snappy.RandomData
[ OK ] Snappy.RandomData (20182 ms)
[ RUN ] Snappy.FourByteOffset
[ OK ] Snappy.FourByteOffset (1 ms)
[ RUN ] Snappy.IOVecSourceEdgeCases
[ OK ] Snappy.IOVecSourceEdgeCases (0 ms)
[ RUN ] Snappy.IOVecSinkEdgeCases
[ OK ] Snappy.IOVecSinkEdgeCases (0 ms)
[ RUN ] Snappy.IOVecLiteralOverflow
[ OK ] Snappy.IOVecLiteralOverflow (0 ms)
[ RUN ] Snappy.IOVecCopyOverflow
[ OK ] Snappy.IOVecCopyOverflow (0 ms)
[ RUN ] Snappy.ReadPastEndOfBuffer
[ OK ] Snappy.ReadPastEndOfBuffer (0 ms)
[ RUN ] Snappy.ZeroOffsetCopy
[ OK ] Snappy.ZeroOffsetCopy (0 ms)
[ RUN ] Snappy.ZeroOffsetCopyValidation
[ OK ] Snappy.ZeroOffsetCopyValidation (0 ms)
[ RUN ] Snappy.FindMatchLength
[ OK ] Snappy.FindMatchLength (0 ms)
[ RUN ] Snappy.FindMatchLengthRandom
[ OK ] Snappy.FindMatchLengthRandom (1089 ms)
[ RUN ] Snappy.VerifyCharTable
[ OK ] Snappy.VerifyCharTable (0 ms)
[ RUN ] Snappy.TestBenchmarkFiles
[ OK ] Snappy.TestBenchmarkFiles (409 ms)
[----------] 17 tests from Snappy (26426 ms total)
[----------] 3 tests from SnappyCorruption
[ RUN ] SnappyCorruption.TruncatedVarint
[ OK ] SnappyCorruption.TruncatedVarint (0 ms)
[ RUN ] SnappyCorruption.UnterminatedVarint
[ OK ] SnappyCorruption.UnterminatedVarint (0 ms)
[ RUN ] SnappyCorruption.OverflowingVarint
[ OK ] SnappyCorruption.OverflowingVarint (0 ms)
[----------] 3 tests from SnappyCorruption (0 ms total)
[----------] Global test environment tear-down
[==========] 21 tests from 3 test suites ran. (26441 ms total)

YOU HAVE 1 DISABLED TEST
[ PASSED ] 21 tests.

@anthony-zy
Copy link
Contributor Author

Hi @danilak-G ,
All unit tests for Snappy passed successfully. Could you please approve running the CI workflow?

@anthony-zy
Copy link
Contributor Author

Hi @danilak-G ,
All checks have passed. The CI is green.
Is a code review required before merging? Shall we proceed with the review?

@anthony-zy
Copy link
Contributor Author

anthony-zy commented Sep 5, 2025

@pwnall @haney @danilak-G , would you pls help to review? Thanks.

@danilak-G
Copy link
Collaborator

Hi, thanks for the contribution. We will take a look.

At a first glance, we don't like #if SNAPPY_HAVE_RVV in the middle of the decompression/compression loops and we need to think how to make it more readable. For now I can't provide you with a timeline when we will be able to review this PR

@anthony-zy anthony-zy closed this Sep 5, 2025
@anthony-zy anthony-zy reopened this Sep 5, 2025
@anthony-zy anthony-zy closed this Sep 5, 2025
@anthony-zy
Copy link
Contributor Author

anthony-zy commented Sep 5, 2025

Hi, thanks for the contribution. We will take a look.

At a first glance, we don't like #if SNAPPY_HAVE_RVV in the middle of the decompression/compression loops and we need to think how to make it more readable. For now I can't provide you with a timeline when we will be able to review this PR
thanks

Thanks a lot for the quick feedback!

I moved the #if SNAPPY_HAVE_RVV check to the same level as #if defined(x86_64) && defined(AVX),
turning it into #elif defined(__riscv) && SNAPPY_HAVE_RVV;
the #else branch remains unchanged. @danilak-G

@anthony-zy anthony-zy reopened this Sep 5, 2025
@anthony-zy anthony-zy closed this Sep 10, 2025
@anthony-zy anthony-zy reopened this Sep 10, 2025
@anthony-zy
Copy link
Contributor Author

@pwnall @haney @danilak-G , would you pls help to review? Thanks.

Hey, if anyone has a minute, I'd love a review of my code. We're working to improve Snappy's performance on RISC-V—feedback appreciated!

@danilak-G
Copy link
Collaborator

Hi, the feedback is still the same. There are multiple # if preprocessor directives in the middle of the decompression loops. This is not bad per se but requires quite rigorous review.

Timeline is still unknown from our side, snappy is mostly in maintenance mode for now

@anthony-zy anthony-zy closed this Sep 10, 2025
@anthony-zy anthony-zy reopened this Sep 12, 2025
@anthony-zy anthony-zy closed this Sep 12, 2025
@anthony-zy anthony-zy deleted the add_rvv_support branch September 12, 2025 03:04
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

2 participants