Commit Graph

844 Commits

Author SHA1 Message Date
Jakob Unterwurzacher
961b8ca438 stupidgcm: deduplicate tests 2/2
Deduplicate the cipher setup that was identical
for all tests for each cipher.
2021-09-02 10:04:38 +02:00
Jakob Unterwurzacher
676a4ceb87 stupidgcm: deduplicate tests 1/2
Pull the code shared between chacha and gcm into
generic functions.
2021-09-02 09:57:20 +02:00
Jakob Unterwurzacher
c9b090770a stupidgcm: add chacha20poly1305 via openssl
"stupidChacha20poly1305".

XChaCha will build upon this.
2021-09-02 09:30:28 +02:00
Jakob Unterwurzacher
4e3b7702af fusefrontend: remove leftover Printf
Commit b83ca9c921
inadveredly added a leftover debug Printf.

Delete it.
2021-08-30 11:39:44 +02:00
Jakob Unterwurzacher
34d8a498c4 Unbreak hyperlinks broken by go mod v2 conversion
Commit

  69d88505fd go mod: declare module version v2

translated all instances of "github.com/rfjakob/gocryptfs/" to
"github.com/rfjakob/gocryptfs/v2/".

Unfortunately, this included hyperlinks.

Unbreak the hyperlinks like this:

  find . -name \*.go | xargs sed -i s%https://github.com/rfjakob/gocryptfs/v2/%https://github.com/rfjakob/gocryptfs/v2/%
2021-08-30 11:31:01 +02:00
Jakob Unterwurzacher
a99051b324 Reimplement -serialize_reads flag using new SyncRead mount flag
Let the kernel do the work for us.

See 15a8bb029a
for more info.
2021-08-30 09:53:58 +02:00
Jakob Unterwurzacher
b83ca9c921 Remove serialize_reads package
Will be replaced by go-fuse's new SyncRead flag.

More info: https://github.com/hanwen/go-fuse/issues/395
SyncRead commit: 15a8bb029a
2021-08-30 09:41:38 +02:00
Jakob Unterwurzacher
61ef6b00a6 -devrandom: make flag a no-op
Commit f3c777d5ea added the `-devrandom` option:

    commit f3c777d5ea
    Author: @slackner
    Date:   Sun Nov 19 13:30:04 2017 +0100

    main: Add '-devrandom' commandline option

    Allows to use /dev/random for generating the master key instead of the
    default Go implementation. When the kernel random generator has been
    properly initialized both are considered equally secure, however:

    * Versions of Go prior to 1.9 just fall back to /dev/urandom if the
      getrandom() syscall would be blocking (Go Bug #19274)

    * Kernel versions prior to 3.17 do not support getrandom(), and there
      is no check if the random generator has been properly initialized
      before reading from /dev/urandom

    This is especially useful for embedded hardware with low-entroy. Please
    note that generation of the master key might block indefinitely if the
    kernel cannot harvest enough entropy.

We now require Go v1.13 and Kernel versions should have also moved on.
Make the flag a no-op.

https://github.com/rfjakob/gocryptfs/issues/596
2021-08-25 12:39:17 +02:00
Jakob Unterwurzacher
5f1094b164 -speed: note that -xchacha is selectable 2021-08-24 14:02:12 +02:00
Jakob Unterwurzacher
20ca63cdbc contentenc: remove unused NonceMode constants
Looks like these are part of an abandoned plan.
2021-08-23 22:14:20 +02:00
Jakob Unterwurzacher
dfb7fae52a speed: use algo names from cryptocore 2021-08-23 22:13:49 +02:00
Jakob Unterwurzacher
806334eacf cryptocore: add NonceSize to AEADTypeEnum
Have the information in one centralized place,
and access it from main as needed.
2021-08-23 22:10:23 +02:00
Jakob Unterwurzacher
97d8340bd8 configfile: add Validate() function, support FlagXChaCha20Poly1305
We used to do validation using lists of mandatory feature flags.

With the introduction of XChaCha20Poly1305, this became too
simplistic, as it uses a different IV length, hence disabling
GCMIV128.

Add a dedicated function, Validate(), with open-coded validation
logic.

The validation and creation logic also gets XChaCha20Poly1305
support, and gocryptfs -init -xchacha now writes the flag into
gocryptfs.conf.
2021-08-23 16:00:41 +02:00
Jakob Unterwurzacher
4764a9bde0 Add partial XChaCha20-Poly1305 support (mount flag only)
Mount flag only at the moment, not saved to gocryptfs.conf.

https://github.com/rfjakob/gocryptfs/issues/452
2021-08-23 16:00:41 +02:00
Jakob Unterwurzacher
69d88505fd go mod: declare module version v2
Our git version is v2+ for some time now, but go.mod
still declared v1. Hopefully making both match makes
https://pkg.go.dev/github.com/rfjakob/gocryptfs/v2 work.

All the import paths have been fixed like this:

  find . -name \*.go | xargs sed -i s%github.com/rfjakob/gocryptfs/%github.com/rfjakob/gocryptfs/v2/%
2021-08-23 15:05:15 +02:00
Jakob Unterwurzacher
c9abfc8f06 ensurefds012: package comment should preceded package statement
This makes the comment visible to godoc.
2021-08-23 11:04:22 +02:00
Jakob Unterwurzacher
b603169d2c configfile: pass struct to Create 2/2
Drop Create and rename Create2 to Create.
2021-08-21 14:04:04 +02:00
Jakob Unterwurzacher
4b93525249 configfile: pass struct to Create 1/2
The argument list got too long.

Part 1: Replace with Create2
2021-08-21 14:01:58 +02:00
Jakob Unterwurzacher
2da0e13b1d cryptocore: drop IVLen helper var
The IVLen var seems be a net loss in clarity. Drop it.

Also add comments and normalize error messages.
2021-08-21 10:55:20 +02:00
Jakob Unterwurzacher
fbccb16043 -deterministic-names: implement for reverse mode, too 2021-08-20 17:06:18 +02:00
Jakob Unterwurzacher
2a9dea2973 -deterministic-names: accept flag on -init
And store it in gocryptfs.conf (=remove DirIV feature flag).
2021-08-20 15:57:40 +02:00
Jakob Unterwurzacher
195d9d18a9 Implement -deterministic-names: extended -zerodiriv
-deterministc-names uses all-zero dirivs but does not write
them to disk anymore.
2021-08-20 10:58:42 +02:00
Jose M Perez
8f94083a21 Flag -zerodiriv to create all diriv as all zero byte files 2021-08-19 18:05:54 +02:00
Jakob Unterwurzacher
02c91d73ce syscallcompat: use early return in asUser() 2021-08-19 09:01:58 +02:00
Jakob Unterwurzacher
be2bd4eec7 golangci-lint: fix issues found by "unused" and "deadcode"
Except xattrSupported, this is a false positive.

$ golangci-lint run --disable-all --enable unused --enable deadcode
gocryptfs-xray/xray_main.go:24:5: `GitVersionFuse` is unused (deadcode)
var GitVersionFuse = "[GitVersionFuse not set - please compile using ./build.bash]"
    ^
tests/symlink_race/main.go:47:6: `chmodLoop` is unused (deadcode)
func chmodLoop() {
     ^
internal/readpassword/extpass_test.go:11:5: `testPw` is unused (deadcode)
var testPw = []byte("test")
    ^
tests/reverse/xattr_test.go:13:6: func `xattrSupported` is unused (unused)
func xattrSupported(path string) bool {
     ^
internal/fusefrontend_reverse/rpath.go:20:22: func `(*RootNode).abs` is unused (unused)
func (rfs *RootNode) abs(relPath string, err error) (string, error) {
                     ^
tests/matrix/matrix_test.go:310:6: `sContains` is unused (deadcode)
func sContains(haystack []string, needle string) bool {
2021-08-19 08:34:49 +02:00
Jakob Unterwurzacher
c86981342b golangci-lint: fix issues found by gosimple
Everything except the

	if err2.Err == syscall.EOPNOTSUPP

case. Gets too confusing when collapsed into a single line.

Issues were:

$ golangci-lint run --disable-all --enable gosimple
mount.go:473:2: S1008: should use 'return strings.HasPrefix(v, "fusermount version")' instead of 'if strings.HasPrefix(v, "fusermount version") { return true }; return false' (gosimple)
	if strings.HasPrefix(v, "fusermount version") {
	^
cli_args.go:258:5: S1002: should omit comparison to bool constant, can be simplified to `args.forcedecode` (gosimple)
	if args.forcedecode == true {
	   ^
cli_args.go:263:6: S1002: should omit comparison to bool constant, can be simplified to `args.aessiv` (gosimple)
		if args.aessiv == true {
		   ^
cli_args.go:267:6: S1002: should omit comparison to bool constant, can be simplified to `args.reverse` (gosimple)
		if args.reverse == true {
		   ^
internal/stupidgcm/stupidgcm.go:227:6: S1002: should omit comparison to bool constant, can be simplified to `g.forceDecode` (gosimple)
		if g.forceDecode == true {
		   ^
gocryptfs-xray/xray_tests/xray_test.go:23:5: S1004: should use !bytes.Equal(out, expected) instead (gosimple)
	if bytes.Compare(out, expected) != 0 {
	   ^
gocryptfs-xray/xray_tests/xray_test.go:40:5: S1004: should use !bytes.Equal(out, expected) instead (gosimple)
	if bytes.Compare(out, expected) != 0 {
	   ^
gocryptfs-xray/paths_ctlsock.go:34:20: S1002: should omit comparison to bool constant, can be simplified to `!eof` (gosimple)
	for eof := false; eof == false; line++ {
	                  ^
tests/reverse/xattr_test.go:19:2: S1008: should use 'return err2.Err != syscall.EOPNOTSUPP' instead of 'if err2.Err == syscall.EOPNOTSUPP { return false }; return true' (gosimple)
	if err2.Err == syscall.EOPNOTSUPP {
	^
internal/fusefrontend/node.go:459:45: S1002: should omit comparison to bool constant, can be simplified to `!nameFileAlreadyThere` (gosimple)
		if nametransform.IsLongContent(cName2) && nameFileAlreadyThere == false {
		                                          ^
tests/xattr/xattr_integration_test.go:221:2: S1008: should use 'return err2.Err != syscall.EOPNOTSUPP' instead of 'if err2.Err == syscall.EOPNOTSUPP { return false }; return true' (gosimple)
	if err2.Err == syscall.EOPNOTSUPP {
	^
tests/test_helpers/helpers.go:338:19: S1002: should omit comparison to bool constant, can be simplified to `open` (gosimple)
	if err != nil && open == true {
	                 ^
tests/matrix/concurrency_test.go:121:7: S1004: should use !bytes.Equal(buf, content) instead (gosimple)
			if bytes.Compare(buf, content) != 0 {
			   ^
2021-08-19 08:34:44 +02:00
Jakob Unterwurzacher
2a25c3a8fd tlog: switch from golang.org/x/crypto/ssh/terminal to golang.org/x/term
$ golangci-lint run

internal/tlog/log.go:13:2: SA1019: package golang.org/x/crypto/ssh/terminal is deprecated: this package moved to golang.org/x/term. (staticcheck)
	"golang.org/x/crypto/ssh/terminal"
2021-08-19 07:38:56 +02:00
Jakob Unterwurzacher
a7fa91764a Update dependencies
Using

  go get -u
  go mod tidy
2021-08-18 17:30:01 +02:00
Jakob Unterwurzacher
64793fedf4 Fix issues found by ineffassign
gocryptfs$ ineffassign ./...

/home/jakob/go/src/github.com/rfjakob/gocryptfs/internal/configfile/config_file.go:243:2: ineffectual assignment to scryptHash
/home/jakob/go/src/github.com/rfjakob/gocryptfs/internal/configfile/config_file.go:272:2: ineffectual assignment to scryptHash
/home/jakob/go/src/github.com/rfjakob/gocryptfs/internal/fusefrontend/file.go:285:3: ineffectual assignment to fileID
/home/jakob/go/src/github.com/rfjakob/gocryptfs/internal/fusefrontend/node.go:367:3: ineffectual assignment to err
/home/jakob/go/src/github.com/rfjakob/gocryptfs/internal/fusefrontend/node_open_create.go:68:2: ineffectual assignment to fd
/home/jakob/go/src/github.com/rfjakob/gocryptfs/mount.go:308:2: ineffectual assignment to masterkey
/home/jakob/go/src/github.com/rfjakob/gocryptfs/gocryptfs-xray/xray_main.go:156:13: ineffectual assignment to err
/home/jakob/go/src/github.com/rfjakob/gocryptfs/internal/fusefrontend/prepare_syscall_test.go:65:16: ineffectual assignment to errno
/home/jakob/go/src/github.com/rfjakob/gocryptfs/internal/syscallcompat/open_nofollow_test.go:34:2: ineffectual assignment to fd
/home/jakob/go/src/github.com/rfjakob/gocryptfs/tests/defaults/acl_test.go:111:6: ineffectual assignment to err
/home/jakob/go/src/github.com/rfjakob/gocryptfs/tests/defaults/acl_test.go:181:2: ineffectual assignment to sz
/home/jakob/go/src/github.com/rfjakob/gocryptfs/tests/defaults/acl_test.go:198:2: ineffectual assignment to sz
/home/jakob/go/src/github.com/rfjakob/gocryptfs/tests/defaults/main_test.go:365:8: ineffectual assignment to err
/home/jakob/go/src/github.com/rfjakob/gocryptfs/tests/xattr/xattr_fd_test.go:30:6: ineffectual assignment to err
/home/jakob/go/src/github.com/rfjakob/gocryptfs/tests/xattr/xattr_fd_test.go:66:6: ineffectual assignment to err
2021-08-18 15:48:01 +02:00
Jakob Unterwurzacher
0bc9784508 reverse: fix "exclude all but" case
With test.

Fixes https://github.com/rfjakob/gocryptfs/issues/588
2021-08-18 11:38:56 +02:00
Jakob Unterwurzacher
b2724070d9 reverse mode: implement -one-file-system
Fixes https://github.com/rfjakob/gocryptfs/issues/475
2021-08-16 19:23:58 +02:00
Jakob Unterwurzacher
763499ee80 inomap: update outdated wording in comments 2021-08-16 17:14:14 +02:00
Jakob Unterwurzacher
b8d78d6a31 inomap: warn on first use of spillMap
We normally should not need it, warn if we do.
As the tests run with -wpanic, we would catch it.
2021-08-16 17:13:14 +02:00
Jakob Unterwurzacher
831e225616 syscallcompat: use BTRFS_SUPER_MAGIC from unix lib 2021-08-11 20:28:20 +02:00
Jakob Unterwurzacher
2d386fc92e syscallcompat: move quirks logic here & fix darwin
We need to look at f_fstypename acc. to
https://stackoverflow.com/a/52299141/1380267 :

> As filesystem type numbers are now assigned at runtime in
> recent versions of MacOS, you must use f_fstypename to
> determine the type.

https://github.com/rfjakob/gocryptfs/issues/585
2021-08-11 20:23:35 +02:00
Jakob Unterwurzacher
c3c9513e65 fusefrontend: add quirks for MacOS ExFAT
This also moves the quirks logic into fusefrontend.

Fixes https://github.com/rfjakob/gocryptfs/issues/585
2021-08-02 20:01:26 +02:00
Jakob Unterwurzacher
1bc1db620b fusefrontend: -sharedstorage: present stable inode numbers
Use the Gen field (inode generation) to distinguish hard links
while passing the real inode numbers to userspace.

Fixes https://github.com/rfjakob/gocryptfs/issues/584
2021-07-31 13:24:25 +02:00
Jakob Unterwurzacher
1dfd6b7b76 fusefrontend: prepareAtSyscall: handle error when opening ourselves
Error handling was missing here, so we would later get confusing
EBADF errors due to dirfd being -1.
2021-07-31 10:53:32 +02:00
Jakob Unterwurzacher
0ca302f12a fusefrontend: implement fsync on directories
Fixes https://github.com/rfjakob/gocryptfs/issues/587
2021-07-29 20:39:50 +02:00
Jakob Unterwurzacher
e83b79b4c2 fido2: actually drop -v flag
Commit 2a9d70d48f only
dropped the flag on mount but not on `-init`.

Also drop it on `-init`.

Fixes https://github.com/rfjakob/gocryptfs/issues/571 (part II)
2021-07-29 12:47:40 +02:00
Jakob Unterwurzacher
2a9d70d48f fido2: drop -v option (PIN request)
We used to pass `-v` on `gocryptfs -init` but not for
mount, which seems strange by itself, but more importantly,
`-v` does not work on Yubikeys.

Drop `-v`.

Fixes https://github.com/rfjakob/gocryptfs/issues/571
2021-06-27 11:17:29 +02:00
Jakob Unterwurzacher
d6c8d892ff fido2: pretty-print fidoCommand in debug output
Related: https://github.com/rfjakob/gocryptfs/issues/571
2021-06-27 11:12:40 +02:00
Jakob Unterwurzacher
4fd95b718b fusefrontend: delete openBackingDir 2021-06-26 18:49:54 +02:00
Jakob Unterwurzacher
5306fc345b fusefrontend: convert last callers from openBackingDir to prepareAtSyscall 2021-06-26 18:49:54 +02:00
Jakob Unterwurzacher
45648e567a fusefrontend: ctlsock: get rid of unneccessary wrapper function 2021-06-26 18:49:54 +02:00
Jakob Unterwurzacher
f9f4bd214f fusefrontend: convert ctlsock from openBackingDir to prepareAtSyscall
openBackingDir will be removed.

Also, remove leftover debug printfs.
2021-06-26 18:49:54 +02:00
Jakob Unterwurzacher
ee59b5269b fusefrontend: convert openBackingDir tests to prepareAtSyscall
openBackingDir will be removed.
2021-06-26 16:28:30 +02:00
Jakob Unterwurzacher
84e702126a fusefrontend: implement recursive diriv caching
The new contrib/maxlen.bash showed that we have exponential
runtime with respect to directory depth.

The new recursive diriv caching is a lot smarter as it caches
intermediate lookups. maxlen.bash now completes in a few seconds.

xfstests results same as
2d158e4c82/screenlog.0 :

  Failures: generic/035 generic/062 generic/080 generic/093 generic/099 generic/215 generic/285 generic/319 generic/426 generic/444 generic/467 generic/477 generic/523
  Failed 13 of 580 tests

benchmark.bash results are identical:

  $ ./benchmark.bash
  Testing gocryptfs at /tmp/benchmark.bash.BdQ: gocryptfs v2.0.1-17-g6b09bc0; go-fuse v2.1.1-0.20210611132105-24a1dfe6b4f8; 2021-06-25 go1.16.5 linux/amd64
  /tmp/benchmark.bash.BdQ.mnt is a mountpoint
  WRITE: 262144000 bytes (262 MB, 250 MiB) copied, 0,4821 s, 544 MB/s
  READ:  262144000 bytes (262 MB, 250 MiB) copied, 0,266061 s, 985 MB/s
  UNTAR: 8,280
  MD5:   4,564
  LS:    1,745
  RM:    2,244
2021-06-25 13:56:53 +02:00
Jakob Unterwurzacher
05b813f202 nametransform: rename BadNameFlag to BadnameSuffix 2021-06-21 12:12:44 +02:00
Jakob Unterwurzacher
689b74835b nametransform: gather badname functions in badname.go 2021-06-21 12:10:04 +02:00
Jakob Unterwurzacher
2efef1e270 nametransform: delete NameTransformer interface
Useless layer of indirection.
2021-06-21 11:53:33 +02:00
Jakob Unterwurzacher
6b0e63c1a8 Improve startup debug output
The startup debug output was very verbose but still missing some
effective crypto settings.
2021-06-21 11:32:04 +02:00
Jakob Unterwurzacher
c5d8fa83ae nametransform: pass badname patterns via New
This means we can unexport the field.
2021-06-20 19:09:46 +02:00
Jakob Unterwurzacher
50630e9f3d fido2: hide "FIDO2" in gocryptfs.conf if not used
Result of:

$ gocryptfs -init foo
$ cat foo/gocryptfs.conf

Before:

{
	"Creator": "gocryptfs v2.0.1",
	"EncryptedKey": "FodEdNHD/cCwv1n5BuyAkbIOnJ/O5gfdCh3YssUCJ2DUr0A8DrQ5NH2SLhREeWRL3V8EMiPO2Ncr5IVwE4SSxQ==",
	"ScryptObject": {
		"Salt": "brGaw9Jg1kbPuSXFiwoxqK2oXFTgbniSgpiB+cu+67Y=",
		"N": 65536,
		"R": 8,
		"P": 1,
		"KeyLen": 32
	},
	"Version": 2,
	"FeatureFlags": [
		"GCMIV128",
		"HKDF",
		"DirIV",
		"EMENames",
		"LongNames",
		"Raw64"
	],
	"FIDO2": {
		"CredentialID": null,
		"HMACSalt": null
	}
}

After:

{
	"Creator": "gocryptfs v2.0.1-5-gf9718eb-dirty.DerDonut-badnamecontent",
	"EncryptedKey": "oFMj1lS1ZsM/vEfanNMeCTPw3PZr5VWeL7ap8Jd8YQm6evy2BAhtQ/pd6RzDx84wlCz9TpxqHRihuwSEMnOWWg==",
	"ScryptObject": {
		"Salt": "JZ/5mhy4a8EAQ/wDF1POIEe4/Ss38cfJgXgj26DuA4M=",
		"N": 65536,
		"R": 8,
		"P": 1,
		"KeyLen": 32
	},
	"Version": 2,
	"FeatureFlags": [
		"GCMIV128",
		"HKDF",
		"DirIV",
		"EMENames",
		"LongNames",
		"Raw64"
	]
}
2021-06-20 18:09:21 +02:00
DerDonut
a611810ff4 Badname file content access
This proposal is the counterpart of the modifications from the `-badname`
parameter. It modifies the plain -> cipher mapping for filenames when using
`-badname` parameter. The new function `EncryptAndHashBadName` tries to find a
cipher filename for the given plain name with the following steps:

1. If `badname` is disabled or direct mapping is successful: Map directly
(default and current behaviour)

2. If a file with badname flag has a valid cipher file, this is returned
(=File just ends with the badname flag)

3. If a file with a badname flag exists where only the badname flag was added,
this is returned (=File cipher name could not be decrypted by function
`DecryptName` and just the badname flag was added)

4. Search for all files which cipher file name extists when cropping more and
more characters from the end. If only 1 file is found, return this

5. Return an error otherwise

This allows file access in the file browsers but most important it allows that
you rename files with undecryptable cipher names in the plain directories.
Renaming those files will then generate a proper cipher filename One
backdraft: When mounting the cipher dir with -badname parameter, you can never
create (or rename to) files whose file name ends with the badname file flag
(at the moment this is " GOCRYPTFS_BAD_NAME"). This will cause an error.

I modified the CLI test function to cover additional test cases. Test [Case
7](https://github.com/DerDonut/gocryptfs/blob/badnamecontent/tests/cli/cli_test.go#L712)
cannot be performed since the cli tests are executed in panic mode. The
testing is stopped on error. Since the function`DecryptName` produces internal
errors when hitting non-decryptable file names, this test was omitted.

This implementation is a proposal where I tried to change the minimum amount
of existing code. Another possibility would be instead of creating the new
function `EncryptAndHashBadName` to modify the signature of the existing
function `EncryptAndHashName(name string, iv []byte)` to
`EncryptAndHashName(name string, iv []byte, dirfd int)` and integrate the
functionality into this function directly. You may allow calling with dirfd=-1
or other invalid values an then performing the current functionality.
2021-06-20 18:09:21 +02:00
Jakob Unterwurzacher
17f859d3c4 fusefronted: report plaintext size on symlink creation
gocryptfs 2.0 introduced the regression that the size
reported at symlink creation was the ciphertext size,
which is wrong.

Report the plaintext size.

Fixes https://github.com/rfjakob/gocryptfs/issues/574
2021-06-06 19:22:16 +02:00
Jakob Unterwurzacher
e48f2377ec syscallcompat: drop obsolete wrappers
These are now available cross-platform in the unix
package.
2021-06-05 15:06:30 +02:00
Jakob Unterwurzacher
015cd066e1 fido2: quote argument strings in debug output
Tested using

  gocryptfs -init -debug -fido2 "hello world" cipherdir

Output before:

  callFidoCommand: executing "/usr/bin/fido2-cred" with args [fido2-cred -M -h -v hello world]

After:

  callFidoCommand: executing "/usr/bin/fido2-cred" with args ["fido2-cred" "-M" "-h" "-v" "hello world"]

Related: https://github.com/rfjakob/gocryptfs/issues/571
2021-06-03 22:03:21 +02:00
Jakob Unterwurzacher
a38e5988ba fusefrontend: run acl Setxattr in user context
The result of setting an acl depends on who runs the
operation!

Fixes fuse-xfstests generic/375
(see https://github.com/rfjakob/fuse-xfstests/wiki/results_2021-05-19)
2021-06-02 19:10:36 +02:00
Jakob Unterwurzacher
b23e21f61f fusefrontend: catch ReadAt integer overflow
Discovered by xfstests generic/564 .

Failure was:

generic/564	- output mismatch (see /opt/fuse-xfstests/results//generic/564.out.bad)
    --- tests/generic/564.out	2021-05-08 21:11:05.307395966 +0200
    +++ /opt/fuse-xfstests/results//generic/564.out.bad	2021-05-19 19:01:16.912888879 +0200
    @@ -31,7 +31,7 @@
     source range beyond 8TiB returns 0

     destination range beyond 8TiB returns EFBIG
    -copy_range: File too large
    +copy_range: Function not implemented
2021-06-02 18:20:05 +02:00
Jakob Unterwurzacher
04858ddd22 nametransform: check name validity on encryption
xfstests generic/523 discovered that we allowed to set
xattrs with "/" in the name, but did not allow to read
them later.

With this change we do not allow to set them in the first
place.
2021-06-02 14:29:48 +02:00
Jakob Unterwurzacher
18befda0e6 fusefrontend: list "." and ".." in dir entries
Fixes xfstests generic/401
2021-05-29 16:44:38 +02:00
Jakob Unterwurzacher
738a9e006a fusefrontend: rewrite Lseek SEEK_DATA / SEEK_HOLE
In response to the discussion of the xfstests mailing list [1],
I looked at the Lseek implementation, which was naive and
did not handle all cases correctly.

The new implementation aligns the returned values to 4096 bytes
as most callers expect.

A lot of tests are added to verify that we handle all
cases correctly now.

[1]: https://www.spinics.net/lists/fstests/msg16554.html
2021-05-29 16:05:36 +02:00
Jakob Unterwurzacher
07164cbb3a contentenc: add PlainOffToCipherOff helper
Will be used for improving Lseek()
2021-05-26 18:28:59 +02:00
Jakob Unterwurzacher
b4794bedec contentenc: fix CipherSizeToPlainSize non-monoticity
For an illegal cipherSize, pretend we have an additional
1-byte block.

See code comment for details.
2021-05-26 13:17:56 +02:00
Jakob Unterwurzacher
bebd7ed81f contentenc: update comments
Also, replace one open-coded calculation with a
helper function.
2021-05-26 13:17:56 +02:00
Jakob Unterwurzacher
2a5ac3e9ba tests: contentenc: add TestSizeToSize
TestSizeToSize tests CipherSizeToPlainSize and PlainSizeToCipherSize.

Fails at the moment due to CipherSizeToPlainSize non-moniticity.
2021-05-26 13:17:56 +02:00
Jakob Unterwurzacher
1b3c3b1347 syscallcompat: add GetdentsSpecial()
GetdentsSpecial calls then Getdents syscall,
with normal entries and "." / ".." split into two slices.
2021-05-26 13:17:56 +02:00
Jakob Unterwurzacher
1d2ac1e589 stupidgcm: prefer Go stdlib over OpenSSL on Apple M1
https://github.com/rfjakob/gocryptfs/issues/556
2021-05-26 09:20:22 +02:00
Jakob Unterwurzacher
09870bfac5 syscallcompat: also refactor MkdiratUser on GOOS=darwin
Breakage was:

+GOOS=darwin
+GOARCH=amd64
+go build -tags without_openssl
internal/fusefrontend/node_dir_ops.go:45:34: cannot use context (type *fuse.Context) as type *fuse.Caller in argument to syscallcompat.MkdiratUser
internal/fusefrontend/node_dir_ops.go:83:35: cannot use context (type *fuse.Context) as type *fuse.Caller in argument to syscallcompat.MkdiratUser
2021-05-22 22:01:46 +02:00
Jakob Unterwurzacher
e1853e1011 syscallcompat: refactor MkdiratUser to take fuse.Context
Let's have MkdiratUser take fuse.Context like everybody
else.
2021-05-22 21:44:19 +02:00
Jakob Unterwurzacher
cb4f9f9e29 syscallcompat: deduplicate OpenatUser/MknodatUser/SymlinkatUser/MkdiratUser
Turns out the whole euid switching logic can be shared when
wrapping the syscall in a closure.
2021-05-22 21:39:29 +02:00
Jakob Unterwurzacher
0650a512bb fsck: redirect go-fuse noise to syslog 2021-05-18 18:38:23 +02:00
Jakob Unterwurzacher
5da20da977 -speed: note that XChaCha20 is not selectable
This is unclear to users, as seen on
https://github.com/rfjakob/gocryptfs/issues/452#issuecomment-828836651
2021-05-18 09:53:23 +02:00
Jakob Unterwurzacher
a91ad29d36 fusefrontend: fix RENAME_NOREPLACE darwin build failure
Error was:

internal/fusefrontend/node.go:371:2: duplicate case syscallcompat.RENAME_NOREPLACE (value 0) in switch
	previous case at internal/fusefrontend/node.go:368:7

Rewrite using "if"s instead.
2021-05-15 17:20:43 +02:00
Jakob Unterwurzacher
1ba2e42234 fusefrontend: avoid duplicate const definition
RENAME_NOREPLACE is already defined in syscallcompat.
2021-05-08 20:42:42 +02:00
Jakob Unterwurzacher
a267562d29 fusefrontend: reject broken RENAME_EXCHANGE and RENAME_WHITEOUT
Discovered by xfstests generic/013: or implementation for
RENAME_EXCHANGE and RENAME_WHITEOUT is incomplete. Reject the
flags so that the caller retries with regular rename.
2021-05-08 15:39:49 +02:00
Jakob Unterwurzacher
4a07d6598c fusefrontend: make dirCache work for "node itself"
"node itself" can be converted to node + child by
ascending one level.

Performance gains are spectacular, as will be seen
in the next commit.
2021-04-07 07:18:35 +02:00
Jakob Unterwurzacher
f73aee72f8 fusefrontend: print dirCache stats after unmount 2021-04-05 18:20:17 +02:00
Jakob Unterwurzacher
043f81dd01 fs: more idiomatic dirCache struct naming 2021-04-04 13:05:47 +02:00
Jakob Unterwurzacher
24d5d39300 fs: add initial dirfd caching
dirfd caching was temporarily removed when moving
to the v2api. Add it back to gain back some lost speed.
2021-04-03 13:08:28 +02:00
Jakob Unterwurzacher
6aae2aad97 tests: fusefronted: fix TestOpenBackingDir
This test only worked accidentially, and would
break once dirfd caching is added.

fs.Mkdir(..., "dir1/dir2", ...) is illegal
(child name cannot contain slashes).
2021-04-03 12:34:18 +02:00
Jakob Unterwurzacher
6b28c0be2c fusefronted: replace last rn.openBackingDir() calls
Use the n.prepareAtSyscall() wrapper instead. Prepares
for adding caching into n.prepareAtSyscall().
2021-04-03 11:19:04 +02:00
Jakob Unterwurzacher
3b9a1b628b fusefronted: move Create() and Open() to new file
And deduplicate the code a little.
2021-03-21 09:31:05 +01:00
Jakob Unterwurzacher
47a4d33f24 fusefrontend: -sharedstorage: fix TestRmwRace failure
The Open() and Create() paths used different inode numbers,
which broke openfiletable locking against concurred readers.
2021-03-21 09:02:50 +01:00
Jakob Unterwurzacher
f6036c429a syscallcompat: getdents: link to #483
Give a user receiving the Getdents warning some background info.
2021-03-14 14:43:11 +01:00
Jakob Unterwurzacher
e2dc52a965 v2api: -sharestorage: disable hard link tracking & add tests
Hard link tracking was not correctly disabled
since the migration to the go-fuse v2 api.

Add a test to ensure it stays off.

Fixes https://github.com/rfjakob/gocryptfs/issues/525
2021-03-07 17:22:29 +01:00
Jakob Unterwurzacher
eaca820e87 fusefrontend: do not encrypt ACLs
Pass through system.posix_acl_access and system.posix_acl_default
unencrypted to fix "cp -a" problems.

"cp -a" uses "setxattr" even to set normal permissions, see
https://www.spinics.net/lists/linux-nfs/msg63986.html .

Fixes https://github.com/rfjakob/gocryptfs/issues/543
2021-02-07 20:01:16 +01:00
Jakob Unterwurzacher
80a651a194 syscallcompat: MknodatUser: work around changed syscall.Setgroups semantics
Since go1.16beta1 (commit d1b1145cace8b968307f9311ff611e4bb810710c ,
https://go-review.googlesource.com/c/go/+/210639 )
syscall.{Setgroups,Setregid,Setreuid} affects all threads, which
is exactly what we not want.

We now use unix.{Setgroups,Setregid,Setreuid} instead.

Workarounds https://github.com/golang/go/issues/1435 .
2021-02-06 11:38:25 +01:00
Jakob Unterwurzacher
bed60101f4 nametransform: make gocryptfs.diriv and gocryptfs.xxx.name files world-readable
Make `gocryptfs.diriv` and `gocryptfs.xxx.name` files world-readable to make encrypted backups easier
when mounting via fstab.

Having the files follow chmod/chown of their parent does not seem
to be worth the hassle. The content of the diriv files is not
secret, and both diriv and name files are protected by the
perms of the parent dir.

Fixes https://github.com/rfjakob/gocryptfs/issues/539
2021-01-10 08:07:10 +01:00
Jakob Unterwurzacher
f3394ae286 nametransform: move permission constants to perms.go
Prep for https://github.com/rfjakob/gocryptfs/issues/539
2021-01-10 07:27:04 +01:00
gmd20
c20c7992a0 main: add "-kernel_cache" flag
This option is similar to fuse(8) kernel_cache

Verified using vmtouch.

Without -kernel_cache:

$ dd if=/dev/zero of=foo bs=1M count=10 ; vmtouch -t foo ; vmtouch foo
10+0 records in
10+0 records out
10485760 bytes (10 MB, 10 MiB) copied, 0,0242321 s, 433 MB/s
           Files: 1
     Directories: 0
   Touched Pages: 2560 (10M)
         Elapsed: 0.011159 seconds
           Files: 1
     Directories: 0
  Resident Pages: 0/2560  0/10M  0%
         Elapsed: 0.000993 seconds

With -kernel_cache:

$ dd if=/dev/zero of=foo bs=1M count=10 ; vmtouch -t foo ; vmtouch foo
10+0 records in
10+0 records out
10485760 bytes (10 MB, 10 MiB) copied, 0,0244015 s, 430 MB/s
           Files: 1
     Directories: 0
   Touched Pages: 2560 (10M)
         Elapsed: 0.011564 seconds
           Files: 1
     Directories: 0
  Resident Pages: 2560/2560  10M/10M  100%
         Elapsed: 0.000369 seconds
2020-12-20 09:55:04 +01:00
Jakob Unterwurzacher
1c1692c4d9 fusefrontend_reverse: fix GETATTR panic
We don't implement Getattr(), so don't try to call it.

Reported at https://github.com/rfjakob/gocryptfs/issues/519#issuecomment-718790790 :

15:22:53.414101 rx 3344: READ n2565 {Fh 7 [42143744 +131072)  L 0 RDONLY,0x8000}
15:22:53.414274 rx 3342: READ n2565 {Fh 7 [42012672 +131072)  L 0 RDONLY,0x8000}
15:22:53.414787 rx 3346: READ n2565 {Fh 7 [42274816 +131072)  L 0 RDONLY,0x8000}
15:22:53.414806 tx 3336:     OK,  131072b data "\xcb\xd3<\"!-\xa7\xc4"...
15:22:53.414874 rx 3348: GETATTR n1446 {Fh 0}
panic: interface conversion: *fusefrontend_reverse.File is not fs.FileGetattrer: missing method Getattr

goroutine 451 [running]:
github.com/rfjakob/gocryptfs/internal/fusefrontend_reverse.(*Node).Getattr(0xc00034c880, 0x5620579784c0, 0xc000593e60, 0x562057939800, 0xc000218050, 0xc0000fc108, 0x0)
        github.com/rfjakob/gocryptfs/internal/fusefrontend_reverse/node.go:69 +0x273
github.com/hanwen/go-fuse/v2/fs.(*rawBridge).getattr(0xc00011e000, 0x5620579784c0, 0xc000593e60, 0xc00034c880, 0x562057939800, 0xc000218050, 0xc0000fc108, 0xbfded1ef58ba7b13)
        github.com/hanwen/go-fuse/v2@v2.0.4-0.20200908172753-0b6cbc515082/fs/bridge.go:500 +0x2d4
github.com/hanwen/go-fuse/v2/fs.(*rawBridge).GetAttr(0xc00011e000, 0xc0000e0000, 0xc0000fc198, 0xc0000fc108, 0x0)
        github.com/hanwen/go-fuse/v2@v2.0.4-0.20200908172753-0b6cbc515082/fs/bridge.go:488 +0x11c
github.com/hanwen/go-fuse/v2/fuse.doGetAttr(0xc000120000, 0xc0000fc000)
        github.com/hanwen/go-fuse/v2@v2.0.4-0.20200908172753-0b6cbc515082/fuse/opcode.go:287 +0x6f
github.com/hanwen/go-fuse/v2/fuse.(*Server).handleRequest(0xc000120000, 0xc0000fc000, 0xc000000000)
        github.com/hanwen/go-fuse/v2@v2.0.4-0.20200908172753-0b6cbc515082/fuse/server.go:472 +0x2c1
github.com/hanwen/go-fuse/v2/fuse.(*Server).loop(0xc000120000, 0xc000288001)
        github.com/hanwen/go-fuse/v2@v2.0.4-0.20200908172753-0b6cbc515082/fuse/server.go:445 +0x18c
created by github.com/hanwen/go-fuse/v2/fuse.(*Server).readRequest
        github.com/hanwen/go-fuse/v2@v2.0.4-0.20200908172753-0b6cbc515082/fuse/server.go:312 +0x419
2020-11-10 19:37:49 +01:00
Jakob Unterwurzacher
832e58cad4 Drop two more generated files
These were committed by mistake.
2020-10-19 19:27:47 +02:00
Jakob Unterwurzacher
165bf6c849 Drop generated files
These were committed by mistake.
2020-10-19 19:25:47 +02:00
Jakob Unterwurzacher
6697ffd6e2 fusefronted: reject GETXATTR "security.capability"
Unless we are mounted with -suid, we can reject
these requests, and gain back some lost speed.

Closes https://github.com/rfjakob/gocryptfs/issues/515
2020-10-18 21:07:30 +02:00
Jakob Unterwurzacher
c943ed32aa syscallcompat: add getxattr fastpaths
The allocations from Lgetxattr were #1 in the tar extract
allocation profile (caused by security.capability lookups).
No more!

$ benchstat old.txt new.txt
name         old time/op  new time/op  delta
Lgetxattr-4  15.2µs ± 0%   1.8µs ± 0%   ~     (p=1.000 n=1+1)

$ ./benchmark.bash
Testing gocryptfs at /tmp/benchmark.bash.H8p: gocryptfs v2.0-beta1-4-g95ea738-dirty; go-fuse v2.0.4-0.20200908172753-0b6cbc515082 => github.com/rfjakob/go-fuse/v2 v2.0.4-0.20201015204057-88b12c99f8af; 2020-10-18 go1.15.3 linux/amd64
/tmp/benchmark.bash.H8p.mnt is a mountpoint
WRITE: 262144000 bytes (262 MB, 250 MiB) copied, 0,520109 s, 504 MB/s
READ:  262144000 bytes (262 MB, 250 MiB) copied, 0,255672 s, 1,0 GB/s
UNTAR: 30,238
MD5:   12,721
LS:    10,038
RM:    16,536
2020-10-18 00:25:42 +02:00
Jakob Unterwurzacher
83a324a46b syscallcompat: add Lgetxattr benchmark 2020-10-16 20:04:22 +02:00
Jakob Unterwurzacher
fcb28e4ff3 v2pai: delete fusefrontend_reverse_v1api
Served its mission a copy-paste source but can now be deleted.
2020-10-15 23:18:21 +02:00
Jakob Unterwurzacher
ec3eaf0b87 syscallcompat: don't retry Close()
After Close() returns, the fd is dead, even if we
received EINTR. Don't retry, we could shoot down
an unrelated fd that received the same fd number.
2020-10-14 13:40:12 +02:00