Commit Graph

859 Commits

Author SHA1 Message Date
Jakob Unterwurzacher
d74cf7c723 speed: add per-blocksize GoGCM benchmarks
Only visible when you run "go test -bench" like this:

$ cd gocryptfs/internal/speed
$ go test -bench .

goos: linux
goarch: amd64
pkg: github.com/rfjakob/gocryptfs/v2/internal/speed
cpu: Intel(R) Core(TM) i5-3470 CPU @ 3.20GHz
BenchmarkStupidGCM-4              	  202352	      5937 ns/op	 689.96 MB/s
BenchmarkStupidGCMDecrypt-4       	  206023	      5782 ns/op	 708.38 MB/s
BenchmarkGoGCM-4                  	  291878	      4098 ns/op	 999.45 MB/s
BenchmarkGoGCMBlockSize/1024-4    	 1000000	      1151 ns/op	 889.88 MB/s
BenchmarkGoGCMBlockSize/2048-4    	  561182	      2134 ns/op	 959.60 MB/s
BenchmarkGoGCMBlockSize/4096-4    	  292057	      4101 ns/op	 998.87 MB/s
BenchmarkGoGCMBlockSize/8192-4    	  149216	      8031 ns/op	1020.09 MB/s
BenchmarkGoGCMBlockSize/16384-4   	   75361	     15917 ns/op	1029.34 MB/s
BenchmarkGoGCMBlockSize/32768-4   	   37916	     31649 ns/op	1035.35 MB/s
BenchmarkGoGCMBlockSize/65536-4   	   19005	     63117 ns/op	1038.33 MB/s
BenchmarkGoGCMBlockSize/131072-4  	    9498	    126166 ns/op	1038.89 MB/s
BenchmarkGoGCMBlockSize/262144-4  	    4755	    252149 ns/op	1039.64 MB/s
BenchmarkGoGCMBlockSize/524288-4  	    2377	    504108 ns/op	1040.03 MB/s
BenchmarkGoGCMBlockSize/1048576-4 	    1188	   1008675 ns/op	1039.56 MB/s
BenchmarkGoGCMDecrypt-4           	  294664	      4059 ns/op	1009.02 MB/s
BenchmarkAESSIV-4                 	   46498	     25432 ns/op	 161.05 MB/s
BenchmarkAESSIVDecrypt-4          	   46908	     25509 ns/op	 160.57 MB/s
BenchmarkXchacha-4                	  244473	      4894 ns/op	 836.97 MB/s
BenchmarkXchachaDecrypt-4         	  249710	      4798 ns/op	 853.75 MB/s
BenchmarkStupidXchacha-4          	  166988	      7101 ns/op	 576.79 MB/s
BenchmarkStupidXchachaDecrypt-4   	  163093	      7240 ns/op	 565.72 MB/s
BenchmarkStupidChacha-4           	  184172	      6527 ns/op	 627.58 MB/s
BenchmarkStupidChachaDecrypt-4    	  179796	      6659 ns/op	 615.11 MB/s
PASS
ok  	github.com/rfjakob/gocryptfs/v2/internal/speed	30.068s
2023-03-08 16:54:56 +01:00
Jakob Unterwurzacher
8f3ec5dcaa fusefrontend: unbreak isConsecutiveWrite streaming write optimization
Commit 6196a5b5 got the logic inverted, hence we never
set the last position markers.

Fixes https://github.com/rfjakob/gocryptfs/issues/712
2023-02-21 22:08:41 +01:00
Jakob Unterwurzacher
85297cda97 fusefrontend: doWrite: report readFileID errors as I/O error
It used to be reported as "function not implemented", accompanied
with this log output:

  go-fuse: can't convert error type: ParseHeader: header is all-zero. Header hexdump: 000000000000000000000000000000000000

Now we report EIO and log this:

  doWrite 1372183: corrupt header: ParseHeader: header is all-zero. Header hexdump: 000000000000000000000000000000000000
2023-02-21 22:08:41 +01:00
Jakob Unterwurzacher
e9a5b8962b contentenc: simplify testRange tables
Get rid of this eyesore.
2023-02-21 22:08:41 +01:00
Jakob Unterwurzacher
88bc0aa607 MANPAGE: scryptn: list how much memory is needed
Calculated acc. to https://words.filippo.io/the-scrypt-parameters/ ,
and add benchmarks to double-check the numbers. They match.
2023-01-08 22:17:14 +01:00
Jakob Unterwurzacher
856ccaac10 make format
Run "make format" using
go version go1.19.4 linux/amd64
2022-12-29 15:00:37 +01:00
Christian Stewart
7ee4c8e9c3 go.mod: fix jacobsa/crypto build on riscv64
Replace dependency jacobsa/crypto with a fork with support for riscv64.

Issue: https://github.com/rfjakob/gocryptfs/issues/666

Upstream PR: https://github.com/jacobsa/crypto/issues/13

Unaddressed on jacobsa/crypto:

https://github.com/jacobsa/crypto/pull/14#issuecomment-1182744229

Signed-off-by: Christian Stewart <christian@paral.in>
2022-12-21 18:38:11 +01:00
Jakob Unterwurzacher
6677d8f1d5 Replace remaining golang.org/x/crypto/ssh/terminal ref with golang.org/x/term
Fixes https://github.com/rfjakob/gocryptfs/issues/681
Fixes 2a25c3a8fd
2022-08-28 12:03:34 +02:00
Jakob Unterwurzacher
003a7fa2e5 make format 2022-08-28 11:11:36 +02:00
NekoGirlSAIKOU
4808adc761 Add comment to pass Codacy Static Code Analysis 2022-08-28 11:09:06 +02:00
NekoGirlSAIKOU
1bff80b46c Fix invalid -longnamemax for reverse mode 2022-08-28 11:09:01 +02:00
Abirdcfly
702a2e19cc fix minor unreachable code caused by t.Fatal
Signed-off-by: Abirdcfly <fp544037857@gmail.com>
2022-08-15 14:24:18 +02:00
Yuta Hayashibe
e9ecff7f07 Fix typos 2022-06-26 10:59:06 +02:00
Val
c9e4e4f741 Fix reverse gocryptfs.conf access on macOS
Unlike the FUSE implementation on Linux, macFUSE doesn't cache the file
attributes from the `LOOKUP` call, so it calls `GETATTR` prior to
accessing a file.

In the case of the `VirtualConfNode` (reverse config file passthrough),
this resulted in the default `GETATTR` implementation returning an empty
result, ultimately resulting in a "permission denied" error.

    14:44:14.095207 rx 3: GETATTR n2
    14:44:14.095229 tx 3:     OK, {tA=1s {M0100000 SZ=0 L=0 0:0 0 0:8954996 A 0.000000 M 0.000000 C 0.000000}}
    14:44:14.099943 rx 4: ACCESS n2 {u=501 g=20 r}
    14:44:14.099990 tx 4:     13=permission denied

By impementing `Getattr` (from `fs.NodeGetattrer`) on `VirtualConfNode`
this solves the issue.
2022-04-02 15:15:04 +02:00
Jakob Unterwurzacher
ba75aa1ab0 root_test: add TestOverlay ; syscallcompat: add QuirkNoUserXattr 2022-01-27 15:44:09 +01:00
Jakob Unterwurzacher
b859bc96ef fusefrontend: fix "duplicate case" darwin build failure
$ ./crossbuild.bash
[...]
+ GOOS=darwin
+ GOARCH=amd64
+ build
+ go build -tags without_openssl -o /dev/null
internal/fusefrontend/node.go:397:2: duplicate case syscallcompat.RENAME_NOREPLACE (value 0) in switch
	previous case at internal/fusefrontend/node.go:397:7
internal/fusefrontend/node.go:397:2: duplicate case syscallcompat.RENAME_EXCHANGE (value 0) in switch
	previous case at internal/fusefrontend/node.go:397:7
internal/fusefrontend/node.go:397:2: duplicate case syscallcompat.RENAME_WHITEOUT (value 0) in switch
	previous case at internal/fusefrontend/node.go:397:7
internal/fusefrontend/node.go:399:38: duplicate case syscallcompat.RENAME_NOREPLACE | syscallcompat.RENAME_WHITEOUT (value 0) in switch
	previous case at internal/fusefrontend/node.go:397:7
2022-01-22 12:44:04 +01:00
Jakob Unterwurzacher
b7cac4ffd0 fusefrontend: support RENAME_WHITEOUT, RENAME_EXCHANGE
Both new internal test and xfstests generic/013 are happy.

https://github.com/rfjakob/gocryptfs/issues/641
2022-01-22 12:28:27 +01:00
Jakob Unterwurzacher
5f955423b7 fusefrontend: fix -force_owner not affecting MKNOD
Fixes https://github.com/rfjakob/gocryptfs/issues/629
2022-01-10 20:05:36 +01:00
Jakob Unterwurzacher
4b251f3ce1 readpassword: bubble up errors instead of exiting the process
This allows cleanups to happen in the caller, like removing
the control socket.

Fixes https://github.com/rfjakob/gocryptfs/issues/634
2022-01-03 15:18:59 +01:00
Jakob Unterwurzacher
5749e70c7c nametransform: fix oversight in comment 2021-12-19 14:50:52 +01:00
Jakob Unterwurzacher
64be5de75f fusefrontend: allow slashes in xattr names
xattr names have fewer restrictions than file names,
relax the validation.

Fixes https://github.com/rfjakob/gocryptfs/issues/627
2021-12-19 14:43:56 +01:00
Jakob Unterwurzacher
eb42e54182 tlog: only enable color if both stderr and stdout are a terminal
This

    gocryptfs -init /does/not/exist 2> err.log

used to write escape codes into err.log. Stop doing that.
2021-12-11 15:37:13 +01:00
Jakob Unterwurzacher
a1f01419e2 tlog: respect NO_COLOR
Fixes https://github.com/rfjakob/gocryptfs/issues/617
2021-12-11 15:35:01 +01:00
Jakob Unterwurzacher
bd1ecf5379 darwin: use O_NOFOLLOW for xattr opens
Running the tests we have lots of these:

Openat: O_NOFOLLOW missing: flags = 0x4
-wpanic turns this warning into a panic: Openat: O_NOFOLLOW missing: flags = 0x4
panic: -wpanic turns this warning into a panic: Openat: O_NOFOLLOW missing: flags = 0x4

goroutine 114 [running]:
log.(*Logger).Panic(0x14000118280, {0x14000313ca8, 0x1, 0x1})
	log/log.go:224 +0x90
github.com/rfjakob/gocryptfs/v2/internal/tlog.(*toggledLogger).Printf(0x14000076780, {0x1009dc2e8, 0x27}, {0x14000313d18, 0x1, 0x1})
	github.com/rfjakob/gocryptfs/v2/internal/tlog/log.go:78 +0x168
github.com/rfjakob/gocryptfs/v2/internal/syscallcompat.Openat(0x9, {0x1009d0747, 0x1}, 0x4, 0x0)
	github.com/rfjakob/gocryptfs/v2/internal/syscallcompat/sys_common.go:59 +0xf0
github.com/rfjakob/gocryptfs/v2/internal/fusefrontend.(*Node).getXAttr(0x14000142000, {0x1400001c140, 0x3a})
	github.com/rfjakob/gocryptfs/v2/internal/fusefrontend/node_xattr_darwin.go:30 +0x8c
github.com/rfjakob/gocryptfs/v2/internal/fusefrontend.(*Node).Getxattr(0x14000142000, {0x100a7eba0, 0x1400000c2e8}, {0x14000016348, 0x14}, {0x14000326000, 0x20, 0x4000})
	github.com/rfjakob/gocryptfs/v2/internal/fusefrontend/node_xattr.go:65 +0x1ac
github.com/hanwen/go-fuse/v2/fs.(*rawBridge).GetXAttr(0x1400008e140, 0x140001901e0, 0x140001133c0, {0x14000016348, 0x14}, {0x14000326000, 0x20, 0x4000})
	github.com/hanwen/go-fuse/v2@v2.1.1-0.20210825171523-3ab5d95a30ae/fs/bridge.go:685 +0x114
github.com/hanwen/go-fuse/v2/fuse.doGetXAttr(0x14000144000, 0x14000113200)
	github.com/hanwen/go-fuse/v2@v2.1.1-0.20210825171523-3ab5d95a30ae/fuse/opcode.go:270 +0x224
github.com/hanwen/go-fuse/v2/fuse.(*Server).handleRequest(0x14000144000, 0x14000113200)
	github.com/hanwen/go-fuse/v2@v2.1.1-0.20210825171523-3ab5d95a30ae/fuse/server.go:499 +0x214
created by github.com/hanwen/go-fuse/v2/fuse.(*Server).loop
	github.com/hanwen/go-fuse/v2@v2.1.1-0.20210825171523-3ab5d95a30ae/fuse/server.go:470 +0xac

https://github.com/rfjakob/gocryptfs/issues/625
2021-12-09 17:55:05 +01:00
Jakob Unterwurzacher
d530fbd400 docs: names longer than 175 bytes (not 176) are stored in longnames
Quoting fusefrontend_reverse/node_helpers.go :

	// File names are padded to 16-byte multiples, encrypted and
	// base64-encoded. We can encode at most 176 bytes to stay below the 255
	// bytes limit:
	// * base64(176 bytes) = 235 bytes
	// * base64(192 bytes) = 256 bytes (over 255!)
	// But the PKCS#7 padding is at least one byte. This means we can only use
	// 175 bytes for the file name.

Noticed by @bailey27 at https://github.com/rfjakob/gocryptfs/issues/499#issuecomment-955790427
2021-11-01 14:44:32 +01:00
Jakob Unterwurzacher
87a6bb370a nametransform: fix math.MaxInt build failure on older Go
Failure is:

  # github.com/rfjakob/gocryptfs/v2/internal/nametransform
  internal/nametransform/names.go:47:33: undefined: math.MaxInt

math.MaxInt was only introduced in Go 1.17. Use MaxInt32 instead,
which is good enough, even on amd64. It only has to be larger than
any name we might encounter.
2021-10-21 16:44:05 +02:00
Jakob Unterwurzacher
d14c9340d6 cli: add -longnamemax
Fixes https://github.com/rfjakob/gocryptfs/issues/499
2021-10-21 15:58:19 +02:00
Jakob Unterwurzacher
d583bdb79e configfile: add LongNameMax support
Feature flag + numeric paramater

https://github.com/rfjakob/gocryptfs/issues/499
2021-10-21 14:55:30 +02:00
Jakob Unterwurzacher
dc32710045 nametransform: add longNameMax parameter
Determines when to start hashing long names instead
of hardcoded 255. Will be used to alleviate "name too long"
issues some users see on cloud storage.

https://github.com/rfjakob/gocryptfs/issues/499
2021-10-21 14:55:30 +02:00
Jakob Unterwurzacher
a652be805e configfile: replace broken switch/case logic with if
Because switch only matches once, we could have missed invalid
cases.

Replace the switch statements with a straight if rake.
2021-10-21 14:55:30 +02:00
Charles Duffy
8ec872e330 fusefrontend: honor ForceOwner for LOOKUP and CREATE operations 2021-10-15 17:35:12 +02:00
Jakob Unterwurzacher
75cace0568 cryptocore: simplify declarations
Reported by codacity:

internal/cryptocore/cryptocore.go
Minor icon MINOR
Code Style
should omit type AEADTypeEnum from declaration of var BackendAESSIV; it will be inferred from the right-hand side
var BackendAESSIV AEADTypeEnum = AEADTypeEnum{"AES-SIV-512", "Go", siv_aead.NonceSize}
Minor icon MINOR
Code Style
should omit type AEADTypeEnum from declaration of var BackendXChaCha20Poly1305; it will be inferred from the right-hand side
var BackendXChaCha20Poly1305 AEADTypeEnum = AEADTypeEnum{"XChaCha20-Poly1305", "Go", chacha20poly1305.NonceSizeX}
Minor icon MINOR
Code Style
should omit type AEADTypeEnum from declaration of var BackendXChaCha20Poly1305OpenSSL; it will be inferred from the right-hand side
var BackendXChaCha20Poly1305OpenSSL AEADTypeEnum = AEADTypeEnum{"XChaCha20-Poly1305", "OpenSSL", chacha20poly1305.NonceSizeX}
Found 2 possible new issues
internal/cryptocore/cryptocore.go
Minor icon MINOR
Code Style
should omit type AEADTypeEnum from declaration of var BackendOpenSSL; it will be inferred from the right-hand side
var BackendOpenSSL AEADTypeEnum = AEADTypeEnum{"AES-GCM-256", "OpenSSL", 16}
Minor icon MINOR
Code Style
should omit type AEADTypeEnum from declaration of var BackendGoGCM; it will be inferred from the right-hand side
var BackendGoGCM AEADTypeEnum = AEADTypeEnum{"AES-GCM-256", "Go", 16}
2021-09-28 18:35:37 +02:00
Jakob Unterwurzacher
db1824a23a cryptocore: disentangle algorithm / library implementation name
Used in gocryptfs-xray, and will also be used in -info.
2021-09-28 18:09:31 +02:00
Jakob Unterwurzacher
2d0ba24eca -speed: print cpu model
When somebody posts "gocryptfs -speed" results, they are
most helpful together with the CPU model. Add the cpu model
to the output.

Example:

$ ./gocryptfs -speed
gocryptfs v2.2.0-beta1-5-g52b0444-dirty; go-fuse v2.1.1-0.20210825171523-3ab5d95a30ae; 2021-09-14 go1.17.1 linux/amd64
cpu: Intel(R) Core(TM) i5-3470 CPU @ 3.20GHz; with AES acceleration
AES-GCM-256-OpenSSL       	 862.79 MB/s
AES-GCM-256-Go            	 997.71 MB/s	(selected in auto mode)
AES-SIV-512-Go            	 159.58 MB/s
XChaCha20-Poly1305-OpenSSL	 729.65 MB/s
XChaCha20-Poly1305-Go     	 843.97 MB/s	(selected in auto mode)
2021-09-14 18:58:22 +02:00
Jakob Unterwurzacher
61e37b2439 stupidgcm: add CpuHasAES()
Makes the code clearer, and will be used in the next commit.
2021-09-14 18:58:04 +02:00
Jakob Unterwurzacher
cdbc48fe29 -speed: drop useless tab at end of line 2021-09-14 10:15:18 +02:00
Jakob Unterwurzacher
c9b825c58a inomap: deterministically set root device
We used to have "first Translate() wins". This is not deterministic,
as the LOOKUP for the root directory does not seem to reach us, so
the first user LOOKUP would win, which may be on a mountpoint.
2021-09-10 17:17:16 +02:00
Jakob Unterwurzacher
d023cd6c95 cli: drop -forcedecode flag
The rewritten openssl backend does not support this flag anymore,
and it was inherently dangerour. Drop it (ignored for compatibility)
2021-09-10 12:14:19 +02:00
Jakob Unterwurzacher
ad21647f25 -speed: show which xchacha implementation is preferred 2021-09-08 20:46:52 +02:00
Jakob Unterwurzacher
94e8004b6c Make -openssl also apply to xchacha
Now that stupidgcm supports xchacha, make it available
on mount.
2021-09-08 20:32:16 +02:00
Jakob Unterwurzacher
1a58667293 stupidgcm: add PreferOpenSSL{AES256GCM,Xchacha20poly1305}
Add PreferOpenSSLXchacha20poly1305,
rename PreferOpenSSL -> PreferOpenSSLAES256GCM.
2021-09-08 19:48:13 +02:00
Jakob Unterwurzacher
85c2beccaf stupidgcm: normalize constructor naming
New() -> NewAES256GCM()

Also add missing NewChacha20poly1305
constructor in without_openssl.go.
2021-09-07 18:15:04 +02:00
Jakob Unterwurzacher
f47e287c20 stupidgcm: revamp package documentation
Maybe interesting for people following
https://github.com/rfjakob/gocryptfs/issues/452
2021-09-07 18:15:04 +02:00
Jakob Unterwurzacher
d598536709 stupidgcm: unexport stupidGCM struct
No need to have it exported.
2021-09-07 18:15:04 +02:00
Jakob Unterwurzacher
3a80db953d stupidgcm: allow zero-length input data
We used to panic in this case because it is useless.
But Go stdlib supports it, so we should as well.
2021-09-07 18:15:04 +02:00
Jakob Unterwurzacher
738d5a2b3a stupidgcm: fix build with CGO_ENABLED=1 without_openssl
We missed some "// +build" lines
2021-09-07 18:15:04 +02:00
Jakob Unterwurzacher
d9510d0c0b stupidgcm: NewChacha20poly1305: avoid slice append
I noticed that growslice() shows up in the cpuprofile.
Avoiding slice append for the private jey copy gives a 0.6% speedup:

gocryptfs/internal/speed$ benchstat old new
name             old time/op   new time/op   delta
StupidXchacha-4   5.68µs ± 0%   5.65µs ± 0%  -0.63%  (p=0.008 n=5+5)

name             old speed     new speed     delta
StupidXchacha-4  721MB/s ± 0%  725MB/s ± 0%  +0.63%  (p=0.008 n=5+5)
2021-09-07 18:14:58 +02:00
Jakob Unterwurzacher
39b1070506 stupidgcm: add testConcurrency
Verifies that we don't corrupt data when called concurrently.
2021-09-07 18:14:05 +02:00
Jakob Unterwurzacher
f89b14ee3d stupidgcm: cache C.EVP_chacha20_poly1305()
2% performance improvement, almost for free.

gocryptfs/internal/speed$ benchstat old new
name             old time/op   new time/op   delta
StupidXchacha-4   5.82µs ± 0%   5.68µs ± 0%  -2.37%  (p=0.008 n=5+5)

name             old speed     new speed     delta
StupidXchacha-4  704MB/s ± 0%  721MB/s ± 0%  +2.43%  (p=0.008 n=5+5)
2021-09-07 18:14:05 +02:00
Jakob Unterwurzacher
6a0206897c stupidgcm: add BenchmarkCCall
gocryptfs/internal/stupidgcm$ go test -bench .
goos: linux
goarch: amd64
pkg: github.com/rfjakob/gocryptfs/v2/internal/stupidgcm
cpu: Intel(R) Core(TM) i5-3470 CPU @ 3.20GHz
BenchmarkCCall-4   	15864030	        78.60 ns/op
PASS
ok  	github.com/rfjakob/gocryptfs/v2/internal/stupidgcm	1.898s
2021-09-07 18:14:05 +02:00