I noticed that growslice() shows up in the cpuprofile.
Avoiding slice append for the private jey copy gives a 0.6% speedup:
gocryptfs/internal/speed$ benchstat old new
name old time/op new time/op delta
StupidXchacha-4 5.68µs ± 0% 5.65µs ± 0% -0.63% (p=0.008 n=5+5)
name old speed new speed delta
StupidXchacha-4 721MB/s ± 0% 725MB/s ± 0% +0.63% (p=0.008 n=5+5)
2% performance improvement, almost for free.
gocryptfs/internal/speed$ benchstat old new
name old time/op new time/op delta
StupidXchacha-4 5.82µs ± 0% 5.68µs ± 0% -2.37% (p=0.008 n=5+5)
name old speed new speed delta
StupidXchacha-4 704MB/s ± 0% 721MB/s ± 0% +2.43% (p=0.008 n=5+5)
Gets the decryption speed to the same level as the
encryption speed.
internal/speed$ benchstat old.txt new.txt
name old time/op new time/op delta
StupidXchacha-4 732MB/s ± 0% 740MB/s ± 0% ~ (p=1.000 n=1+1)
StupidXchachaDecrypt-4 602MB/s ± 0% 741MB/s ± 0% ~ (p=1.000 n=1+1)
Go has a high overhead for each C call, so batch
all openssl operations in the new C function chacha20poly1305_seal.
Benchmark results:
internal/speed$ go test -bench BenchmarkStupidXchacha -count 10 > old.txt
internal/speed$ go test -bench BenchmarkStupidXchacha -count 10 > new.txt
internal/speed$ benchstat old.txt new.txt
name old time/op new time/op delta
StupidXchacha-4 8.79µs ± 1% 7.25µs ± 1% -17.54% (p=0.000 n=10+10)
name old speed new speed delta
StupidXchacha-4 466MB/s ± 1% 565MB/s ± 1% +21.27% (p=0.000 n=10+10)
Commit f3c777d5ea added the `-devrandom` option:
commit f3c777d5ea
Author: @slackner
Date: Sun Nov 19 13:30:04 2017 +0100
main: Add '-devrandom' commandline option
Allows to use /dev/random for generating the master key instead of the
default Go implementation. When the kernel random generator has been
properly initialized both are considered equally secure, however:
* Versions of Go prior to 1.9 just fall back to /dev/urandom if the
getrandom() syscall would be blocking (Go Bug #19274)
* Kernel versions prior to 3.17 do not support getrandom(), and there
is no check if the random generator has been properly initialized
before reading from /dev/urandom
This is especially useful for embedded hardware with low-entroy. Please
note that generation of the master key might block indefinitely if the
kernel cannot harvest enough entropy.
We now require Go v1.13 and Kernel versions should have also moved on.
Make the flag a no-op.
https://github.com/rfjakob/gocryptfs/issues/596
We used to do validation using lists of mandatory feature flags.
With the introduction of XChaCha20Poly1305, this became too
simplistic, as it uses a different IV length, hence disabling
GCMIV128.
Add a dedicated function, Validate(), with open-coded validation
logic.
The validation and creation logic also gets XChaCha20Poly1305
support, and gocryptfs -init -xchacha now writes the flag into
gocryptfs.conf.
Our git version is v2+ for some time now, but go.mod
still declared v1. Hopefully making both match makes
https://pkg.go.dev/github.com/rfjakob/gocryptfs/v2 work.
All the import paths have been fixed like this:
find . -name \*.go | xargs sed -i s%github.com/rfjakob/gocryptfs/%github.com/rfjakob/gocryptfs/v2/%
Everything except the
if err2.Err == syscall.EOPNOTSUPP
case. Gets too confusing when collapsed into a single line.
Issues were:
$ golangci-lint run --disable-all --enable gosimple
mount.go:473:2: S1008: should use 'return strings.HasPrefix(v, "fusermount version")' instead of 'if strings.HasPrefix(v, "fusermount version") { return true }; return false' (gosimple)
if strings.HasPrefix(v, "fusermount version") {
^
cli_args.go:258:5: S1002: should omit comparison to bool constant, can be simplified to `args.forcedecode` (gosimple)
if args.forcedecode == true {
^
cli_args.go:263:6: S1002: should omit comparison to bool constant, can be simplified to `args.aessiv` (gosimple)
if args.aessiv == true {
^
cli_args.go:267:6: S1002: should omit comparison to bool constant, can be simplified to `args.reverse` (gosimple)
if args.reverse == true {
^
internal/stupidgcm/stupidgcm.go:227:6: S1002: should omit comparison to bool constant, can be simplified to `g.forceDecode` (gosimple)
if g.forceDecode == true {
^
gocryptfs-xray/xray_tests/xray_test.go:23:5: S1004: should use !bytes.Equal(out, expected) instead (gosimple)
if bytes.Compare(out, expected) != 0 {
^
gocryptfs-xray/xray_tests/xray_test.go:40:5: S1004: should use !bytes.Equal(out, expected) instead (gosimple)
if bytes.Compare(out, expected) != 0 {
^
gocryptfs-xray/paths_ctlsock.go:34:20: S1002: should omit comparison to bool constant, can be simplified to `!eof` (gosimple)
for eof := false; eof == false; line++ {
^
tests/reverse/xattr_test.go:19:2: S1008: should use 'return err2.Err != syscall.EOPNOTSUPP' instead of 'if err2.Err == syscall.EOPNOTSUPP { return false }; return true' (gosimple)
if err2.Err == syscall.EOPNOTSUPP {
^
internal/fusefrontend/node.go:459:45: S1002: should omit comparison to bool constant, can be simplified to `!nameFileAlreadyThere` (gosimple)
if nametransform.IsLongContent(cName2) && nameFileAlreadyThere == false {
^
tests/xattr/xattr_integration_test.go:221:2: S1008: should use 'return err2.Err != syscall.EOPNOTSUPP' instead of 'if err2.Err == syscall.EOPNOTSUPP { return false }; return true' (gosimple)
if err2.Err == syscall.EOPNOTSUPP {
^
tests/test_helpers/helpers.go:338:19: S1002: should omit comparison to bool constant, can be simplified to `open` (gosimple)
if err != nil && open == true {
^
tests/matrix/concurrency_test.go:121:7: S1004: should use !bytes.Equal(buf, content) instead (gosimple)
if bytes.Compare(buf, content) != 0 {
^
$ golangci-lint run
internal/tlog/log.go:13:2: SA1019: package golang.org/x/crypto/ssh/terminal is deprecated: this package moved to golang.org/x/term. (staticcheck)
"golang.org/x/crypto/ssh/terminal"
We used to pass `-v` on `gocryptfs -init` but not for
mount, which seems strange by itself, but more importantly,
`-v` does not work on Yubikeys.
Drop `-v`.
Fixes https://github.com/rfjakob/gocryptfs/issues/571
The new contrib/maxlen.bash showed that we have exponential
runtime with respect to directory depth.
The new recursive diriv caching is a lot smarter as it caches
intermediate lookups. maxlen.bash now completes in a few seconds.
xfstests results same as
2d158e4c82/screenlog.0 :
Failures: generic/035 generic/062 generic/080 generic/093 generic/099 generic/215 generic/285 generic/319 generic/426 generic/444 generic/467 generic/477 generic/523
Failed 13 of 580 tests
benchmark.bash results are identical:
$ ./benchmark.bash
Testing gocryptfs at /tmp/benchmark.bash.BdQ: gocryptfs v2.0.1-17-g6b09bc0; go-fuse v2.1.1-0.20210611132105-24a1dfe6b4f8; 2021-06-25 go1.16.5 linux/amd64
/tmp/benchmark.bash.BdQ.mnt is a mountpoint
WRITE: 262144000 bytes (262 MB, 250 MiB) copied, 0,4821 s, 544 MB/s
READ: 262144000 bytes (262 MB, 250 MiB) copied, 0,266061 s, 985 MB/s
UNTAR: 8,280
MD5: 4,564
LS: 1,745
RM: 2,244
This proposal is the counterpart of the modifications from the `-badname`
parameter. It modifies the plain -> cipher mapping for filenames when using
`-badname` parameter. The new function `EncryptAndHashBadName` tries to find a
cipher filename for the given plain name with the following steps:
1. If `badname` is disabled or direct mapping is successful: Map directly
(default and current behaviour)
2. If a file with badname flag has a valid cipher file, this is returned
(=File just ends with the badname flag)
3. If a file with a badname flag exists where only the badname flag was added,
this is returned (=File cipher name could not be decrypted by function
`DecryptName` and just the badname flag was added)
4. Search for all files which cipher file name extists when cropping more and
more characters from the end. If only 1 file is found, return this
5. Return an error otherwise
This allows file access in the file browsers but most important it allows that
you rename files with undecryptable cipher names in the plain directories.
Renaming those files will then generate a proper cipher filename One
backdraft: When mounting the cipher dir with -badname parameter, you can never
create (or rename to) files whose file name ends with the badname file flag
(at the moment this is " GOCRYPTFS_BAD_NAME"). This will cause an error.
I modified the CLI test function to cover additional test cases. Test [Case
7](https://github.com/DerDonut/gocryptfs/blob/badnamecontent/tests/cli/cli_test.go#L712)
cannot be performed since the cli tests are executed in panic mode. The
testing is stopped on error. Since the function`DecryptName` produces internal
errors when hitting non-decryptable file names, this test was omitted.
This implementation is a proposal where I tried to change the minimum amount
of existing code. Another possibility would be instead of creating the new
function `EncryptAndHashBadName` to modify the signature of the existing
function `EncryptAndHashName(name string, iv []byte)` to
`EncryptAndHashName(name string, iv []byte, dirfd int)` and integrate the
functionality into this function directly. You may allow calling with dirfd=-1
or other invalid values an then performing the current functionality.
gocryptfs 2.0 introduced the regression that the size
reported at symlink creation was the ciphertext size,
which is wrong.
Report the plaintext size.
Fixes https://github.com/rfjakob/gocryptfs/issues/574
xfstests generic/523 discovered that we allowed to set
xattrs with "/" in the name, but did not allow to read
them later.
With this change we do not allow to set them in the first
place.
In response to the discussion of the xfstests mailing list [1],
I looked at the Lseek implementation, which was naive and
did not handle all cases correctly.
The new implementation aligns the returned values to 4096 bytes
as most callers expect.
A lot of tests are added to verify that we handle all
cases correctly now.
[1]: https://www.spinics.net/lists/fstests/msg16554.html
Breakage was:
+GOOS=darwin
+GOARCH=amd64
+go build -tags without_openssl
internal/fusefrontend/node_dir_ops.go:45:34: cannot use context (type *fuse.Context) as type *fuse.Caller in argument to syscallcompat.MkdiratUser
internal/fusefrontend/node_dir_ops.go:83:35: cannot use context (type *fuse.Context) as type *fuse.Caller in argument to syscallcompat.MkdiratUser
