Commit Graph

803 Commits

Author SHA1 Message Date
Jakob Unterwurzacher
5046962634 speed: add bEncrypt helper, reuse dst buffer
The bEncrypt helper massively deduplicates the code,
and reusing the dst buffer gives higher performance,
and that's what gocryptfs does in normal operation via
sync.Pool.

$ benchstat old.txt new.txt
name             old time/op   new time/op    delta
StupidGCM-4       6.24µs ± 1%    4.65µs ± 0%  -25.47%  (p=0.008 n=5+5)
GoGCM-4           4.90µs ± 0%    4.10µs ± 0%  -16.44%  (p=0.008 n=5+5)
AESSIV-4          26.4µs ± 0%    25.6µs ± 0%   -2.90%  (p=0.008 n=5+5)
Xchacha-4         5.76µs ± 0%    4.91µs ± 0%  -14.79%  (p=0.008 n=5+5)
StupidXchacha-4   7.24µs ± 1%    5.48µs ± 0%  -24.33%  (p=0.008 n=5+5)

name             old speed     new speed      delta
StupidGCM-4      656MB/s ± 1%   880MB/s ± 0%  +34.15%  (p=0.008 n=5+5)
GoGCM-4          835MB/s ± 0%  1000MB/s ± 0%  +19.68%  (p=0.008 n=5+5)
AESSIV-4         155MB/s ± 0%   160MB/s ± 0%   +2.99%  (p=0.008 n=5+5)
Xchacha-4        711MB/s ± 0%   834MB/s ± 0%  +17.35%  (p=0.008 n=5+5)
StupidXchacha-4  565MB/s ± 1%   747MB/s ± 0%  +32.15%  (p=0.008 n=5+5)
2021-09-07 18:14:05 +02:00
Jakob Unterwurzacher
d9e89cd021 stupidgcm: use aead_seal for gcm as well
$ benchstat old.txt new.txt
name         old time/op   new time/op   delta
StupidGCM-4   7.87µs ± 1%   6.64µs ± 2%  -15.65%  (p=0.000 n=10+10)

name         old speed     new speed     delta
StupidGCM-4  520MB/s ± 1%  617MB/s ± 2%  +18.56%  (p=0.000 n=10+10)
2021-09-07 18:14:05 +02:00
Jakob Unterwurzacher
69d626b26f stupidgcm: replace chacha20poly1305_seal with generic aead_seal 2021-09-07 18:14:05 +02:00
Jakob Unterwurzacher
a3f5a8492a stupidgcm: batch C calls in chacha20poly1305_seal
Go has a high overhead for each C call, so batch
all openssl operations in the new C function chacha20poly1305_seal.

Benchmark results:

internal/speed$ go test -bench BenchmarkStupidXchacha -count 10 > old.txt
internal/speed$ go test -bench BenchmarkStupidXchacha -count 10 > new.txt

internal/speed$ benchstat old.txt new.txt
name             old time/op   new time/op   delta
StupidXchacha-4   8.79µs ± 1%   7.25µs ± 1%  -17.54%  (p=0.000 n=10+10)

name             old speed     new speed     delta
StupidXchacha-4  466MB/s ± 1%  565MB/s ± 1%  +21.27%  (p=0.000 n=10+10)
2021-09-07 18:14:05 +02:00
Jakob Unterwurzacher
9e1dd73e55 -speed: add XChaCha20-Poly1305-OpenSSL
$ ./gocryptfs -speed
gocryptfs v2.1-56-gdb1466f-dirty.stupidchacha; go-fuse v2.1.1-0.20210825171523-3ab5d95a30ae; 2021-09-02 go1.17 linux/amd64
AES-GCM-256-OpenSSL       	 529.53 MB/s
AES-GCM-256-Go            	 833.85 MB/s	(selected in auto mode)
AES-SIV-512-Go            	 155.27 MB/s
XChaCha20-Poly1305-Go     	 715.33 MB/s	(use via -xchacha flag)
XChaCha20-Poly1305-OpenSSL	 468.94 MB/s

https://github.com/rfjakob/gocryptfs/issues/452
2021-09-07 18:14:05 +02:00
Jakob Unterwurzacher
4017e4b22c stupidgcm: add stupidXchacha20poly1305
Implementation copied from
32db794688/chacha20poly1305/xchacha20poly1305.go
2021-09-07 18:14:05 +02:00
Jakob Unterwurzacher
591a56e7ae stupidgcm: stupidChacha20poly1305: normalize panic messages 2021-09-07 18:14:05 +02:00
Jakob Unterwurzacher
5df7ee815d stupidgcm: stupidChacha20poly1305: use byte array for key
Follow what golang.org/x/crypto/chacha20poly1305 does
for easier integration in the next commit.
2021-09-07 18:13:54 +02:00
Jakob Unterwurzacher
3ba74ac4fc stupidgcm: add testWipe test
After looking at the cover profile, this was the only untested
code except panic cases.
2021-09-02 10:17:01 +02:00
Jakob Unterwurzacher
961b8ca438 stupidgcm: deduplicate tests 2/2
Deduplicate the cipher setup that was identical
for all tests for each cipher.
2021-09-02 10:04:38 +02:00
Jakob Unterwurzacher
676a4ceb87 stupidgcm: deduplicate tests 1/2
Pull the code shared between chacha and gcm into
generic functions.
2021-09-02 09:57:20 +02:00
Jakob Unterwurzacher
c9b090770a stupidgcm: add chacha20poly1305 via openssl
"stupidChacha20poly1305".

XChaCha will build upon this.
2021-09-02 09:30:28 +02:00
Jakob Unterwurzacher
4e3b7702af fusefrontend: remove leftover Printf
Commit b83ca9c921
inadveredly added a leftover debug Printf.

Delete it.
2021-08-30 11:39:44 +02:00
Jakob Unterwurzacher
34d8a498c4 Unbreak hyperlinks broken by go mod v2 conversion
Commit

  69d88505fd go mod: declare module version v2

translated all instances of "github.com/rfjakob/gocryptfs/" to
"github.com/rfjakob/gocryptfs/v2/".

Unfortunately, this included hyperlinks.

Unbreak the hyperlinks like this:

  find . -name \*.go | xargs sed -i s%https://github.com/rfjakob/gocryptfs/v2/%https://github.com/rfjakob/gocryptfs/v2/%
2021-08-30 11:31:01 +02:00
Jakob Unterwurzacher
a99051b324 Reimplement -serialize_reads flag using new SyncRead mount flag
Let the kernel do the work for us.

See 15a8bb029a
for more info.
2021-08-30 09:53:58 +02:00
Jakob Unterwurzacher
b83ca9c921 Remove serialize_reads package
Will be replaced by go-fuse's new SyncRead flag.

More info: https://github.com/hanwen/go-fuse/issues/395
SyncRead commit: 15a8bb029a
2021-08-30 09:41:38 +02:00
Jakob Unterwurzacher
61ef6b00a6 -devrandom: make flag a no-op
Commit f3c777d5ea added the `-devrandom` option:

    commit f3c777d5ea
    Author: @slackner
    Date:   Sun Nov 19 13:30:04 2017 +0100

    main: Add '-devrandom' commandline option

    Allows to use /dev/random for generating the master key instead of the
    default Go implementation. When the kernel random generator has been
    properly initialized both are considered equally secure, however:

    * Versions of Go prior to 1.9 just fall back to /dev/urandom if the
      getrandom() syscall would be blocking (Go Bug #19274)

    * Kernel versions prior to 3.17 do not support getrandom(), and there
      is no check if the random generator has been properly initialized
      before reading from /dev/urandom

    This is especially useful for embedded hardware with low-entroy. Please
    note that generation of the master key might block indefinitely if the
    kernel cannot harvest enough entropy.

We now require Go v1.13 and Kernel versions should have also moved on.
Make the flag a no-op.

https://github.com/rfjakob/gocryptfs/issues/596
2021-08-25 12:39:17 +02:00
Jakob Unterwurzacher
5f1094b164 -speed: note that -xchacha is selectable 2021-08-24 14:02:12 +02:00
Jakob Unterwurzacher
20ca63cdbc contentenc: remove unused NonceMode constants
Looks like these are part of an abandoned plan.
2021-08-23 22:14:20 +02:00
Jakob Unterwurzacher
dfb7fae52a speed: use algo names from cryptocore 2021-08-23 22:13:49 +02:00
Jakob Unterwurzacher
806334eacf cryptocore: add NonceSize to AEADTypeEnum
Have the information in one centralized place,
and access it from main as needed.
2021-08-23 22:10:23 +02:00
Jakob Unterwurzacher
97d8340bd8 configfile: add Validate() function, support FlagXChaCha20Poly1305
We used to do validation using lists of mandatory feature flags.

With the introduction of XChaCha20Poly1305, this became too
simplistic, as it uses a different IV length, hence disabling
GCMIV128.

Add a dedicated function, Validate(), with open-coded validation
logic.

The validation and creation logic also gets XChaCha20Poly1305
support, and gocryptfs -init -xchacha now writes the flag into
gocryptfs.conf.
2021-08-23 16:00:41 +02:00
Jakob Unterwurzacher
4764a9bde0 Add partial XChaCha20-Poly1305 support (mount flag only)
Mount flag only at the moment, not saved to gocryptfs.conf.

https://github.com/rfjakob/gocryptfs/issues/452
2021-08-23 16:00:41 +02:00
Jakob Unterwurzacher
69d88505fd go mod: declare module version v2
Our git version is v2+ for some time now, but go.mod
still declared v1. Hopefully making both match makes
https://pkg.go.dev/github.com/rfjakob/gocryptfs/v2 work.

All the import paths have been fixed like this:

  find . -name \*.go | xargs sed -i s%github.com/rfjakob/gocryptfs/%github.com/rfjakob/gocryptfs/v2/%
2021-08-23 15:05:15 +02:00
Jakob Unterwurzacher
c9abfc8f06 ensurefds012: package comment should preceded package statement
This makes the comment visible to godoc.
2021-08-23 11:04:22 +02:00
Jakob Unterwurzacher
b603169d2c configfile: pass struct to Create 2/2
Drop Create and rename Create2 to Create.
2021-08-21 14:04:04 +02:00
Jakob Unterwurzacher
4b93525249 configfile: pass struct to Create 1/2
The argument list got too long.

Part 1: Replace with Create2
2021-08-21 14:01:58 +02:00
Jakob Unterwurzacher
2da0e13b1d cryptocore: drop IVLen helper var
The IVLen var seems be a net loss in clarity. Drop it.

Also add comments and normalize error messages.
2021-08-21 10:55:20 +02:00
Jakob Unterwurzacher
fbccb16043 -deterministic-names: implement for reverse mode, too 2021-08-20 17:06:18 +02:00
Jakob Unterwurzacher
2a9dea2973 -deterministic-names: accept flag on -init
And store it in gocryptfs.conf (=remove DirIV feature flag).
2021-08-20 15:57:40 +02:00
Jakob Unterwurzacher
195d9d18a9 Implement -deterministic-names: extended -zerodiriv
-deterministc-names uses all-zero dirivs but does not write
them to disk anymore.
2021-08-20 10:58:42 +02:00
Jose M Perez
8f94083a21 Flag -zerodiriv to create all diriv as all zero byte files 2021-08-19 18:05:54 +02:00
Jakob Unterwurzacher
02c91d73ce syscallcompat: use early return in asUser() 2021-08-19 09:01:58 +02:00
Jakob Unterwurzacher
be2bd4eec7 golangci-lint: fix issues found by "unused" and "deadcode"
Except xattrSupported, this is a false positive.

$ golangci-lint run --disable-all --enable unused --enable deadcode
gocryptfs-xray/xray_main.go:24:5: `GitVersionFuse` is unused (deadcode)
var GitVersionFuse = "[GitVersionFuse not set - please compile using ./build.bash]"
    ^
tests/symlink_race/main.go:47:6: `chmodLoop` is unused (deadcode)
func chmodLoop() {
     ^
internal/readpassword/extpass_test.go:11:5: `testPw` is unused (deadcode)
var testPw = []byte("test")
    ^
tests/reverse/xattr_test.go:13:6: func `xattrSupported` is unused (unused)
func xattrSupported(path string) bool {
     ^
internal/fusefrontend_reverse/rpath.go:20:22: func `(*RootNode).abs` is unused (unused)
func (rfs *RootNode) abs(relPath string, err error) (string, error) {
                     ^
tests/matrix/matrix_test.go:310:6: `sContains` is unused (deadcode)
func sContains(haystack []string, needle string) bool {
2021-08-19 08:34:49 +02:00
Jakob Unterwurzacher
c86981342b golangci-lint: fix issues found by gosimple
Everything except the

	if err2.Err == syscall.EOPNOTSUPP

case. Gets too confusing when collapsed into a single line.

Issues were:

$ golangci-lint run --disable-all --enable gosimple
mount.go:473:2: S1008: should use 'return strings.HasPrefix(v, "fusermount version")' instead of 'if strings.HasPrefix(v, "fusermount version") { return true }; return false' (gosimple)
	if strings.HasPrefix(v, "fusermount version") {
	^
cli_args.go:258:5: S1002: should omit comparison to bool constant, can be simplified to `args.forcedecode` (gosimple)
	if args.forcedecode == true {
	   ^
cli_args.go:263:6: S1002: should omit comparison to bool constant, can be simplified to `args.aessiv` (gosimple)
		if args.aessiv == true {
		   ^
cli_args.go:267:6: S1002: should omit comparison to bool constant, can be simplified to `args.reverse` (gosimple)
		if args.reverse == true {
		   ^
internal/stupidgcm/stupidgcm.go:227:6: S1002: should omit comparison to bool constant, can be simplified to `g.forceDecode` (gosimple)
		if g.forceDecode == true {
		   ^
gocryptfs-xray/xray_tests/xray_test.go:23:5: S1004: should use !bytes.Equal(out, expected) instead (gosimple)
	if bytes.Compare(out, expected) != 0 {
	   ^
gocryptfs-xray/xray_tests/xray_test.go:40:5: S1004: should use !bytes.Equal(out, expected) instead (gosimple)
	if bytes.Compare(out, expected) != 0 {
	   ^
gocryptfs-xray/paths_ctlsock.go:34:20: S1002: should omit comparison to bool constant, can be simplified to `!eof` (gosimple)
	for eof := false; eof == false; line++ {
	                  ^
tests/reverse/xattr_test.go:19:2: S1008: should use 'return err2.Err != syscall.EOPNOTSUPP' instead of 'if err2.Err == syscall.EOPNOTSUPP { return false }; return true' (gosimple)
	if err2.Err == syscall.EOPNOTSUPP {
	^
internal/fusefrontend/node.go:459:45: S1002: should omit comparison to bool constant, can be simplified to `!nameFileAlreadyThere` (gosimple)
		if nametransform.IsLongContent(cName2) && nameFileAlreadyThere == false {
		                                          ^
tests/xattr/xattr_integration_test.go:221:2: S1008: should use 'return err2.Err != syscall.EOPNOTSUPP' instead of 'if err2.Err == syscall.EOPNOTSUPP { return false }; return true' (gosimple)
	if err2.Err == syscall.EOPNOTSUPP {
	^
tests/test_helpers/helpers.go:338:19: S1002: should omit comparison to bool constant, can be simplified to `open` (gosimple)
	if err != nil && open == true {
	                 ^
tests/matrix/concurrency_test.go:121:7: S1004: should use !bytes.Equal(buf, content) instead (gosimple)
			if bytes.Compare(buf, content) != 0 {
			   ^
2021-08-19 08:34:44 +02:00
Jakob Unterwurzacher
2a25c3a8fd tlog: switch from golang.org/x/crypto/ssh/terminal to golang.org/x/term
$ golangci-lint run

internal/tlog/log.go:13:2: SA1019: package golang.org/x/crypto/ssh/terminal is deprecated: this package moved to golang.org/x/term. (staticcheck)
	"golang.org/x/crypto/ssh/terminal"
2021-08-19 07:38:56 +02:00
Jakob Unterwurzacher
a7fa91764a Update dependencies
Using

  go get -u
  go mod tidy
2021-08-18 17:30:01 +02:00
Jakob Unterwurzacher
64793fedf4 Fix issues found by ineffassign
gocryptfs$ ineffassign ./...

/home/jakob/go/src/github.com/rfjakob/gocryptfs/internal/configfile/config_file.go:243:2: ineffectual assignment to scryptHash
/home/jakob/go/src/github.com/rfjakob/gocryptfs/internal/configfile/config_file.go:272:2: ineffectual assignment to scryptHash
/home/jakob/go/src/github.com/rfjakob/gocryptfs/internal/fusefrontend/file.go:285:3: ineffectual assignment to fileID
/home/jakob/go/src/github.com/rfjakob/gocryptfs/internal/fusefrontend/node.go:367:3: ineffectual assignment to err
/home/jakob/go/src/github.com/rfjakob/gocryptfs/internal/fusefrontend/node_open_create.go:68:2: ineffectual assignment to fd
/home/jakob/go/src/github.com/rfjakob/gocryptfs/mount.go:308:2: ineffectual assignment to masterkey
/home/jakob/go/src/github.com/rfjakob/gocryptfs/gocryptfs-xray/xray_main.go:156:13: ineffectual assignment to err
/home/jakob/go/src/github.com/rfjakob/gocryptfs/internal/fusefrontend/prepare_syscall_test.go:65:16: ineffectual assignment to errno
/home/jakob/go/src/github.com/rfjakob/gocryptfs/internal/syscallcompat/open_nofollow_test.go:34:2: ineffectual assignment to fd
/home/jakob/go/src/github.com/rfjakob/gocryptfs/tests/defaults/acl_test.go:111:6: ineffectual assignment to err
/home/jakob/go/src/github.com/rfjakob/gocryptfs/tests/defaults/acl_test.go:181:2: ineffectual assignment to sz
/home/jakob/go/src/github.com/rfjakob/gocryptfs/tests/defaults/acl_test.go:198:2: ineffectual assignment to sz
/home/jakob/go/src/github.com/rfjakob/gocryptfs/tests/defaults/main_test.go:365:8: ineffectual assignment to err
/home/jakob/go/src/github.com/rfjakob/gocryptfs/tests/xattr/xattr_fd_test.go:30:6: ineffectual assignment to err
/home/jakob/go/src/github.com/rfjakob/gocryptfs/tests/xattr/xattr_fd_test.go:66:6: ineffectual assignment to err
2021-08-18 15:48:01 +02:00
Jakob Unterwurzacher
0bc9784508 reverse: fix "exclude all but" case
With test.

Fixes https://github.com/rfjakob/gocryptfs/issues/588
2021-08-18 11:38:56 +02:00
Jakob Unterwurzacher
b2724070d9 reverse mode: implement -one-file-system
Fixes https://github.com/rfjakob/gocryptfs/issues/475
2021-08-16 19:23:58 +02:00
Jakob Unterwurzacher
763499ee80 inomap: update outdated wording in comments 2021-08-16 17:14:14 +02:00
Jakob Unterwurzacher
b8d78d6a31 inomap: warn on first use of spillMap
We normally should not need it, warn if we do.
As the tests run with -wpanic, we would catch it.
2021-08-16 17:13:14 +02:00
Jakob Unterwurzacher
831e225616 syscallcompat: use BTRFS_SUPER_MAGIC from unix lib 2021-08-11 20:28:20 +02:00
Jakob Unterwurzacher
2d386fc92e syscallcompat: move quirks logic here & fix darwin
We need to look at f_fstypename acc. to
https://stackoverflow.com/a/52299141/1380267 :

> As filesystem type numbers are now assigned at runtime in
> recent versions of MacOS, you must use f_fstypename to
> determine the type.

https://github.com/rfjakob/gocryptfs/issues/585
2021-08-11 20:23:35 +02:00
Jakob Unterwurzacher
c3c9513e65 fusefrontend: add quirks for MacOS ExFAT
This also moves the quirks logic into fusefrontend.

Fixes https://github.com/rfjakob/gocryptfs/issues/585
2021-08-02 20:01:26 +02:00
Jakob Unterwurzacher
1bc1db620b fusefrontend: -sharedstorage: present stable inode numbers
Use the Gen field (inode generation) to distinguish hard links
while passing the real inode numbers to userspace.

Fixes https://github.com/rfjakob/gocryptfs/issues/584
2021-07-31 13:24:25 +02:00
Jakob Unterwurzacher
1dfd6b7b76 fusefrontend: prepareAtSyscall: handle error when opening ourselves
Error handling was missing here, so we would later get confusing
EBADF errors due to dirfd being -1.
2021-07-31 10:53:32 +02:00
Jakob Unterwurzacher
0ca302f12a fusefrontend: implement fsync on directories
Fixes https://github.com/rfjakob/gocryptfs/issues/587
2021-07-29 20:39:50 +02:00
Jakob Unterwurzacher
e83b79b4c2 fido2: actually drop -v flag
Commit 2a9d70d48f only
dropped the flag on mount but not on `-init`.

Also drop it on `-init`.

Fixes https://github.com/rfjakob/gocryptfs/issues/571 (part II)
2021-07-29 12:47:40 +02:00
Jakob Unterwurzacher
2a9d70d48f fido2: drop -v option (PIN request)
We used to pass `-v` on `gocryptfs -init` but not for
mount, which seems strange by itself, but more importantly,
`-v` does not work on Yubikeys.

Drop `-v`.

Fixes https://github.com/rfjakob/gocryptfs/issues/571
2021-06-27 11:17:29 +02:00