hardcoresushi/libgocryptfs
1
0
Fork 0
Code Issues Pull Requests Releases Activity
1,978 Commits 2 Branches 0 Tags 6 MiB
700ae685cc
Commit Graph

1978 Commits

This Branch
This Branch
All Branches
Author SHA1 Message Date
Jakob Unterwurzacher
f47e287c20 stupidgcm: revamp package documentation 
Maybe interesting for people following
https://github.com/rfjakob/gocryptfs/issues/452
 2021-09-07 18:15:04 +02:00
Jakob Unterwurzacher
d598536709 stupidgcm: unexport stupidGCM struct 
No need to have it exported.
 2021-09-07 18:15:04 +02:00
Jakob Unterwurzacher
3a80db953d stupidgcm: allow zero-length input data 
We used to panic in this case because it is useless.
But Go stdlib supports it, so we should as well.
 2021-09-07 18:15:04 +02:00
Jakob Unterwurzacher
738d5a2b3a stupidgcm: fix build with CGO_ENABLED=1 without_openssl 
We missed some "// +build" lines
 2021-09-07 18:15:04 +02:00
Jakob Unterwurzacher
d9510d0c0b stupidgcm: NewChacha20poly1305: avoid slice append 
I noticed that growslice() shows up in the cpuprofile.
Avoiding slice append for the private jey copy gives a 0.6% speedup:

gocryptfs/internal/speed$ benchstat old new
name             old time/op   new time/op   delta
StupidXchacha-4   5.68µs ± 0%   5.65µs ± 0%  -0.63%  (p=0.008 n=5+5)

name             old speed     new speed     delta
StupidXchacha-4  721MB/s ± 0%  725MB/s ± 0%  +0.63%  (p=0.008 n=5+5)
 2021-09-07 18:14:58 +02:00
Jakob Unterwurzacher
39b1070506 stupidgcm: add testConcurrency 
Verifies that we don't corrupt data when called concurrently.
 2021-09-07 18:14:05 +02:00
Jakob Unterwurzacher
f89b14ee3d stupidgcm: cache C.EVP_chacha20_poly1305() 
2% performance improvement, almost for free.

gocryptfs/internal/speed$ benchstat old new
name             old time/op   new time/op   delta
StupidXchacha-4   5.82µs ± 0%   5.68µs ± 0%  -2.37%  (p=0.008 n=5+5)

name             old speed     new speed     delta
StupidXchacha-4  704MB/s ± 0%  721MB/s ± 0%  +2.43%  (p=0.008 n=5+5)
 2021-09-07 18:14:05 +02:00
Jakob Unterwurzacher
6a0206897c stupidgcm: add BenchmarkCCall 
gocryptfs/internal/stupidgcm$ go test -bench .
goos: linux
goarch: amd64
pkg: github.com/rfjakob/gocryptfs/v2/internal/stupidgcm
cpu: Intel(R) Core(TM) i5-3470 CPU @ 3.20GHz
BenchmarkCCall-4   	15864030	        78.60 ns/op
PASS
ok  	github.com/rfjakob/gocryptfs/v2/internal/stupidgcm	1.898s
 2021-09-07 18:14:05 +02:00
Jakob Unterwurzacher
a2eaa5e3d1 speed: add BenchmarkStupidChacha 
gocryptfs/internal/speed$ go test -bench .
goos: linux
goarch: amd64
pkg: github.com/rfjakob/gocryptfs/v2/internal/speed
cpu: Intel(R) Core(TM) i5-3470 CPU @ 3.20GHz
BenchmarkStupidGCM-4              	  249396	      4722 ns/op	 867.50 MB/s
BenchmarkStupidGCMDecrypt-4       	  257872	      4616 ns/op	 887.35 MB/s
BenchmarkGoGCM-4                  	  290952	      4097 ns/op	 999.83 MB/s
BenchmarkGoGCMDecrypt-4           	  294106	      4060 ns/op	1008.84 MB/s
BenchmarkAESSIV-4                 	   46520	     25532 ns/op	 160.42 MB/s
BenchmarkAESSIVDecrypt-4          	   46974	     25478 ns/op	 160.76 MB/s
BenchmarkXchacha-4                	  244108	      4881 ns/op	 839.14 MB/s
BenchmarkXchachaDecrypt-4         	  249658	      4786 ns/op	 855.86 MB/s
BenchmarkStupidXchacha-4          	  205339	      5768 ns/op	 710.11 MB/s
BenchmarkStupidXchachaDecrypt-4   	  204577	      5836 ns/op	 701.84 MB/s
BenchmarkStupidChacha-4           	  227510	      5224 ns/op	 784.06 MB/s
BenchmarkStupidChachaDecrypt-4    	  222787	      5359 ns/op	 764.34 MB/s
PASS
ok  	github.com/rfjakob/gocryptfs/v2/internal/speed	15.328s
 2021-09-07 18:14:05 +02:00
Jakob Unterwurzacher
b8c56ccffc stupidgcm: replace naked panics 2021-09-07 18:14:05 +02:00
Jakob Unterwurzacher
8f820c429d stupidgcm: fix without_openssl build 
$ ./build-without-openssl.bash
internal/speed/speed.go:152:14: undefined: stupidgcm.NewXchacha20poly1305
 2021-09-07 18:14:05 +02:00
Jakob Unterwurzacher
c9728247ed test.bash: only check go files for naked panic 
This found a lot of panics in the new file openssl_aead.c.
 2021-09-07 18:14:05 +02:00
Jakob Unterwurzacher
e2ec048a09 stupidgcm: introduce stupidAEADCommon and use for both chacha & gcm 
Nice deduplication and brings the GCM decrypt speed up to par.

internal/speed$ benchstat old new
name                old time/op   new time/op   delta
StupidGCM-4          4.71µs ± 0%   4.66µs ± 0%   -0.99%  (p=0.008 n=5+5)
StupidGCMDecrypt-4   5.77µs ± 1%   4.51µs ± 0%  -21.80%  (p=0.008 n=5+5)

name                old speed     new speed     delta
StupidGCM-4         870MB/s ± 0%  879MB/s ± 0%   +1.01%  (p=0.008 n=5+5)
StupidGCMDecrypt-4  710MB/s ± 1%  908MB/s ± 0%  +27.87%  (p=0.008 n=5+5)
 2021-09-07 18:14:05 +02:00
Jakob Unterwurzacher
bf572aef88 stupidgcm: stupidChacha20poly1305.Open: batch C calls in aead_open 
Gets the decryption speed to the same level as the
encryption speed.

internal/speed$ benchstat old.txt new.txt
name                    old time/op    new time/op    delta
StupidXchacha-4          732MB/s ± 0%   740MB/s ± 0%   ~     (p=1.000 n=1+1)
StupidXchachaDecrypt-4   602MB/s ± 0%   741MB/s ± 0%   ~     (p=1.000 n=1+1)
 2021-09-07 18:14:05 +02:00
Jakob Unterwurzacher
3e27acb989 speed: add decryption benchmarks 
gocryptfs/internal/speed$ go test -bench .
goos: linux
goarch: amd64
pkg: github.com/rfjakob/gocryptfs/v2/internal/speed
cpu: Intel(R) Core(TM) i5-3470 CPU @ 3.20GHz
BenchmarkStupidGCM-4              	  263742	      4523 ns/op	 905.61 MB/s
BenchmarkStupidGCMDecrypt-4       	  204858	      5779 ns/op	 708.76 MB/s
BenchmarkGoGCM-4                  	  291259	      4095 ns/op	1000.25 MB/s
BenchmarkGoGCMDecrypt-4           	  293886	      4061 ns/op	1008.53 MB/s
BenchmarkAESSIV-4                 	   46537	     25538 ns/op	 160.39 MB/s
BenchmarkAESSIVDecrypt-4          	   46770	     25627 ns/op	 159.83 MB/s
BenchmarkXchacha-4                	  243619	      4893 ns/op	 837.03 MB/s
BenchmarkXchachaDecrypt-4         	  248857	      4793 ns/op	 854.51 MB/s
BenchmarkStupidXchacha-4          	  213717	      5558 ns/op	 736.99 MB/s
BenchmarkStupidXchachaDecrypt-4   	  176635	      6782 ns/op	 603.96 MB/s
PASS
ok  	github.com/rfjakob/gocryptfs/v2/internal/speed	12.871s
 2021-09-07 18:14:05 +02:00
Jakob Unterwurzacher
5046962634 speed: add bEncrypt helper, reuse dst buffer 
The bEncrypt helper massively deduplicates the code,
and reusing the dst buffer gives higher performance,
and that's what gocryptfs does in normal operation via
sync.Pool.

$ benchstat old.txt new.txt
name             old time/op   new time/op    delta
StupidGCM-4       6.24µs ± 1%    4.65µs ± 0%  -25.47%  (p=0.008 n=5+5)
GoGCM-4           4.90µs ± 0%    4.10µs ± 0%  -16.44%  (p=0.008 n=5+5)
AESSIV-4          26.4µs ± 0%    25.6µs ± 0%   -2.90%  (p=0.008 n=5+5)
Xchacha-4         5.76µs ± 0%    4.91µs ± 0%  -14.79%  (p=0.008 n=5+5)
StupidXchacha-4   7.24µs ± 1%    5.48µs ± 0%  -24.33%  (p=0.008 n=5+5)

name             old speed     new speed      delta
StupidGCM-4      656MB/s ± 1%   880MB/s ± 0%  +34.15%  (p=0.008 n=5+5)
GoGCM-4          835MB/s ± 0%  1000MB/s ± 0%  +19.68%  (p=0.008 n=5+5)
AESSIV-4         155MB/s ± 0%   160MB/s ± 0%   +2.99%  (p=0.008 n=5+5)
Xchacha-4        711MB/s ± 0%   834MB/s ± 0%  +17.35%  (p=0.008 n=5+5)
StupidXchacha-4  565MB/s ± 1%   747MB/s ± 0%  +32.15%  (p=0.008 n=5+5)
 2021-09-07 18:14:05 +02:00
Jakob Unterwurzacher
d9e89cd021 stupidgcm: use aead_seal for gcm as well 
$ benchstat old.txt new.txt
name         old time/op   new time/op   delta
StupidGCM-4   7.87µs ± 1%   6.64µs ± 2%  -15.65%  (p=0.000 n=10+10)

name         old speed     new speed     delta
StupidGCM-4  520MB/s ± 1%  617MB/s ± 2%  +18.56%  (p=0.000 n=10+10)
 2021-09-07 18:14:05 +02:00
Jakob Unterwurzacher
69d626b26f stupidgcm: replace chacha20poly1305_seal with generic aead_seal 2021-09-07 18:14:05 +02:00
Jakob Unterwurzacher
a3f5a8492a stupidgcm: batch C calls in chacha20poly1305_seal 
Go has a high overhead for each C call, so batch
all openssl operations in the new C function chacha20poly1305_seal.

Benchmark results:

internal/speed$ go test -bench BenchmarkStupidXchacha -count 10 > old.txt
internal/speed$ go test -bench BenchmarkStupidXchacha -count 10 > new.txt

internal/speed$ benchstat old.txt new.txt
name             old time/op   new time/op   delta
StupidXchacha-4   8.79µs ± 1%   7.25µs ± 1%  -17.54%  (p=0.000 n=10+10)

name             old speed     new speed     delta
StupidXchacha-4  466MB/s ± 1%  565MB/s ± 1%  +21.27%  (p=0.000 n=10+10)
 2021-09-07 18:14:05 +02:00
Jakob Unterwurzacher
9e1dd73e55 -speed: add XChaCha20-Poly1305-OpenSSL 
$ ./gocryptfs -speed
gocryptfs v2.1-56-gdb1466f-dirty.stupidchacha; go-fuse v2.1.1-0.20210825171523-3ab5d95a30ae; 2021-09-02 go1.17 linux/amd64
AES-GCM-256-OpenSSL       	 529.53 MB/s
AES-GCM-256-Go            	 833.85 MB/s	(selected in auto mode)
AES-SIV-512-Go            	 155.27 MB/s
XChaCha20-Poly1305-Go     	 715.33 MB/s	(use via -xchacha flag)
XChaCha20-Poly1305-OpenSSL	 468.94 MB/s

https://github.com/rfjakob/gocryptfs/issues/452
 2021-09-07 18:14:05 +02:00
Jakob Unterwurzacher
4017e4b22c stupidgcm: add stupidXchacha20poly1305 
Implementation copied from
32db794688/chacha20poly1305/xchacha20poly1305.go
 2021-09-07 18:14:05 +02:00
Jakob Unterwurzacher
591a56e7ae stupidgcm: stupidChacha20poly1305: normalize panic messages 2021-09-07 18:14:05 +02:00
Jakob Unterwurzacher
5df7ee815d stupidgcm: stupidChacha20poly1305: use byte array for key 
Follow what golang.org/x/crypto/chacha20poly1305 does
for easier integration in the next commit.
 2021-09-07 18:13:54 +02:00
Jakob Unterwurzacher
3ba74ac4fc stupidgcm: add testWipe test 
After looking at the cover profile, this was the only untested
code except panic cases.
 2021-09-02 10:17:01 +02:00
Jakob Unterwurzacher
961b8ca438 stupidgcm: deduplicate tests 2/2 
Deduplicate the cipher setup that was identical
for all tests for each cipher.
 2021-09-02 10:04:38 +02:00
Jakob Unterwurzacher
676a4ceb87 stupidgcm: deduplicate tests 1/2 
Pull the code shared between chacha and gcm into
generic functions.
 2021-09-02 09:57:20 +02:00
Jakob Unterwurzacher
c9b090770a stupidgcm: add chacha20poly1305 via openssl 
"stupidChacha20poly1305".

XChaCha will build upon this.
 2021-09-02 09:30:28 +02:00
Jakob Unterwurzacher
cbf282861b tests/matrix: don't leak fds in TestConcurrentReadCreate 
We leaked a file descriptor for each empty file we encountered.
 2021-09-01 10:28:33 +02:00
a1346054
7c2255be90 *: trim trailing whitespace 2021-09-01 10:22:01 +02:00
a1346054
6cb03b54fe *: fix spelling 2021-09-01 10:22:01 +02:00
a1346054
c63f7e9f64 shell scripts: fix shellcheck warnings 2021-09-01 10:22:01 +02:00
Jakob Unterwurzacher
c505e73a13 README: explain where -xchacha makes sense 2021-08-30 20:00:00 +02:00
Jakob Unterwurzacher
4e3b7702af fusefrontend: remove leftover Printf 
Commit b83ca9c921
inadveredly added a leftover debug Printf.

Delete it.
 2021-08-30 11:39:44 +02:00
Jakob Unterwurzacher
34d8a498c4 Unbreak hyperlinks broken by go mod v2 conversion 
Commit

  69d88505fd go mod: declare module version v2

translated all instances of "github.com/rfjakob/gocryptfs/" to
"github.com/rfjakob/gocryptfs/v2/".

Unfortunately, this included hyperlinks.

Unbreak the hyperlinks like this:

  find . -name \*.go | xargs sed -i s%https://github.com/rfjakob/gocryptfs/v2/%https://github.com/rfjakob/gocryptfs/v2/%
 2021-08-30 11:31:01 +02:00
Jakob Unterwurzacher
17fe50ef74 README: compress Installation section 
More content, less whitespace.
 2021-08-30 10:18:33 +02:00
Jakob Unterwurzacher
fab4ca07de README: update changelog 2021-08-30 10:18:23 +02:00
Jakob Unterwurzacher
a99051b324 Reimplement -serialize_reads flag using new SyncRead mount flag 
Let the kernel do the work for us.

See 15a8bb029a
for more info.
 2021-08-30 09:53:58 +02:00
Jakob Unterwurzacher
b83ca9c921 Remove serialize_reads package 
Will be replaced by go-fuse's new SyncRead flag.

More info: https://github.com/hanwen/go-fuse/issues/395
SyncRead commit: 15a8bb029a
 2021-08-30 09:41:38 +02:00
Jakob Unterwurzacher
e69a85769f go mod: upgrade go-fuse to fix darwin build failure 
Upgraded using

  go get -u github.com/hanwen/go-fuse/v2@master

to get 61df810860

Fixes https://github.com/rfjakob/gocryptfs/issues/597
 2021-08-29 19:43:26 +02:00
Jakob Unterwurzacher
91d3b30c1c doc: file-format.md: describe XChaCha20-Poly1305 
Different nonce size.
 2021-08-26 08:43:41 +02:00
Jakob Unterwurzacher
7b25ff39c8 fsstress-gocryptfs: fuse-xfstests now lives in /opt 
fuse-xfstests should be installed to /opt now to make
the terminal output independent of the user name
(as done in https://github.com/rfjakob/fuse-xfstests/wiki/results_2021-06-02 )
 2021-08-26 07:46:28 +02:00
Jakob Unterwurzacher
61ef6b00a6 -devrandom: make flag a no-op 
Commit f3c777d5ea added the `-devrandom` option:

    commit f3c777d5ea
    Author: @slackner
    Date:   Sun Nov 19 13:30:04 2017 +0100

    main: Add '-devrandom' commandline option

    Allows to use /dev/random for generating the master key instead of the
    default Go implementation. When the kernel random generator has been
    properly initialized both are considered equally secure, however:

    * Versions of Go prior to 1.9 just fall back to /dev/urandom if the
      getrandom() syscall would be blocking (Go Bug #19274)

    * Kernel versions prior to 3.17 do not support getrandom(), and there
      is no check if the random generator has been properly initialized
      before reading from /dev/urandom

    This is especially useful for embedded hardware with low-entroy. Please
    note that generation of the master key might block indefinitely if the
    kernel cannot harvest enough entropy.

We now require Go v1.13 and Kernel versions should have also moved on.
Make the flag a no-op.

https://github.com/rfjakob/gocryptfs/issues/596
 2021-08-25 12:39:17 +02:00
Jakob Unterwurzacher
b3d26b7264 go mod: update go-fuse 
We want /dev/fd/N support:

  74a933d6e8
  "fuse: support special /dev/fd/N mountpoint"

Fixes https://github.com/rfjakob/gocryptfs/issues/590
 2021-08-25 12:17:26 +02:00
Jakob Unterwurzacher
a4ed1aab00 README: add -xchacha to changelog 
https://github.com/rfjakob/gocryptfs/issues/452
 2021-08-24 14:06:01 +02:00
Jakob Unterwurzacher
24bb28a517 MANPAGE: add -xchacha 2021-08-24 14:05:52 +02:00
Jakob Unterwurzacher
5f1094b164 -speed: note that -xchacha is selectable 2021-08-24 14:02:12 +02:00
Jakob Unterwurzacher
ab7308639b tests/example_filesystems: add v2.2-xchacha-deterministic-names 
Combines both new flags.
 2021-08-24 13:53:50 +02:00
Jakob Unterwurzacher
d70875aaa9 tests/example_filesystems: add deterministic-names and xchacha 2021-08-24 13:50:07 +02:00
Jakob Unterwurzacher
62ed081c5e benchmark.bash: add -xchacha support 2021-08-24 13:30:09 +02:00
Jakob Unterwurzacher
abaa12992c xray: add xchacha support 
Also use the new cryptocore algo names.
 2021-08-23 22:19:30 +02:00