Commit Graph

124 Commits

Author SHA1 Message Date
Jakob Unterwurzacher
ba75aa1ab0 root_test: add TestOverlay ; syscallcompat: add QuirkNoUserXattr 2022-01-27 15:44:09 +01:00
Jakob Unterwurzacher
b859bc96ef fusefrontend: fix "duplicate case" darwin build failure
$ ./crossbuild.bash
[...]
+ GOOS=darwin
+ GOARCH=amd64
+ build
+ go build -tags without_openssl -o /dev/null
internal/fusefrontend/node.go:397:2: duplicate case syscallcompat.RENAME_NOREPLACE (value 0) in switch
	previous case at internal/fusefrontend/node.go:397:7
internal/fusefrontend/node.go:397:2: duplicate case syscallcompat.RENAME_EXCHANGE (value 0) in switch
	previous case at internal/fusefrontend/node.go:397:7
internal/fusefrontend/node.go:397:2: duplicate case syscallcompat.RENAME_WHITEOUT (value 0) in switch
	previous case at internal/fusefrontend/node.go:397:7
internal/fusefrontend/node.go:399:38: duplicate case syscallcompat.RENAME_NOREPLACE | syscallcompat.RENAME_WHITEOUT (value 0) in switch
	previous case at internal/fusefrontend/node.go:397:7
2022-01-22 12:44:04 +01:00
Jakob Unterwurzacher
b7cac4ffd0 fusefrontend: support RENAME_WHITEOUT, RENAME_EXCHANGE
Both new internal test and xfstests generic/013 are happy.

https://github.com/rfjakob/gocryptfs/issues/641
2022-01-22 12:28:27 +01:00
Jakob Unterwurzacher
bd1ecf5379 darwin: use O_NOFOLLOW for xattr opens
Running the tests we have lots of these:

Openat: O_NOFOLLOW missing: flags = 0x4
-wpanic turns this warning into a panic: Openat: O_NOFOLLOW missing: flags = 0x4
panic: -wpanic turns this warning into a panic: Openat: O_NOFOLLOW missing: flags = 0x4

goroutine 114 [running]:
log.(*Logger).Panic(0x14000118280, {0x14000313ca8, 0x1, 0x1})
	log/log.go:224 +0x90
github.com/rfjakob/gocryptfs/v2/internal/tlog.(*toggledLogger).Printf(0x14000076780, {0x1009dc2e8, 0x27}, {0x14000313d18, 0x1, 0x1})
	github.com/rfjakob/gocryptfs/v2/internal/tlog/log.go:78 +0x168
github.com/rfjakob/gocryptfs/v2/internal/syscallcompat.Openat(0x9, {0x1009d0747, 0x1}, 0x4, 0x0)
	github.com/rfjakob/gocryptfs/v2/internal/syscallcompat/sys_common.go:59 +0xf0
github.com/rfjakob/gocryptfs/v2/internal/fusefrontend.(*Node).getXAttr(0x14000142000, {0x1400001c140, 0x3a})
	github.com/rfjakob/gocryptfs/v2/internal/fusefrontend/node_xattr_darwin.go:30 +0x8c
github.com/rfjakob/gocryptfs/v2/internal/fusefrontend.(*Node).Getxattr(0x14000142000, {0x100a7eba0, 0x1400000c2e8}, {0x14000016348, 0x14}, {0x14000326000, 0x20, 0x4000})
	github.com/rfjakob/gocryptfs/v2/internal/fusefrontend/node_xattr.go:65 +0x1ac
github.com/hanwen/go-fuse/v2/fs.(*rawBridge).GetXAttr(0x1400008e140, 0x140001901e0, 0x140001133c0, {0x14000016348, 0x14}, {0x14000326000, 0x20, 0x4000})
	github.com/hanwen/go-fuse/v2@v2.1.1-0.20210825171523-3ab5d95a30ae/fs/bridge.go:685 +0x114
github.com/hanwen/go-fuse/v2/fuse.doGetXAttr(0x14000144000, 0x14000113200)
	github.com/hanwen/go-fuse/v2@v2.1.1-0.20210825171523-3ab5d95a30ae/fuse/opcode.go:270 +0x224
github.com/hanwen/go-fuse/v2/fuse.(*Server).handleRequest(0x14000144000, 0x14000113200)
	github.com/hanwen/go-fuse/v2@v2.1.1-0.20210825171523-3ab5d95a30ae/fuse/server.go:499 +0x214
created by github.com/hanwen/go-fuse/v2/fuse.(*Server).loop
	github.com/hanwen/go-fuse/v2@v2.1.1-0.20210825171523-3ab5d95a30ae/fuse/server.go:470 +0xac

https://github.com/rfjakob/gocryptfs/issues/625
2021-12-09 17:55:05 +01:00
Jakob Unterwurzacher
34d8a498c4 Unbreak hyperlinks broken by go mod v2 conversion
Commit

  69d88505fd go mod: declare module version v2

translated all instances of "github.com/rfjakob/gocryptfs/" to
"github.com/rfjakob/gocryptfs/v2/".

Unfortunately, this included hyperlinks.

Unbreak the hyperlinks like this:

  find . -name \*.go | xargs sed -i s%https://github.com/rfjakob/gocryptfs/v2/%https://github.com/rfjakob/gocryptfs/v2/%
2021-08-30 11:31:01 +02:00
Jakob Unterwurzacher
69d88505fd go mod: declare module version v2
Our git version is v2+ for some time now, but go.mod
still declared v1. Hopefully making both match makes
https://pkg.go.dev/github.com/rfjakob/gocryptfs/v2 work.

All the import paths have been fixed like this:

  find . -name \*.go | xargs sed -i s%github.com/rfjakob/gocryptfs/%github.com/rfjakob/gocryptfs/v2/%
2021-08-23 15:05:15 +02:00
Jakob Unterwurzacher
02c91d73ce syscallcompat: use early return in asUser() 2021-08-19 09:01:58 +02:00
Jakob Unterwurzacher
64793fedf4 Fix issues found by ineffassign
gocryptfs$ ineffassign ./...

/home/jakob/go/src/github.com/rfjakob/gocryptfs/internal/configfile/config_file.go:243:2: ineffectual assignment to scryptHash
/home/jakob/go/src/github.com/rfjakob/gocryptfs/internal/configfile/config_file.go:272:2: ineffectual assignment to scryptHash
/home/jakob/go/src/github.com/rfjakob/gocryptfs/internal/fusefrontend/file.go:285:3: ineffectual assignment to fileID
/home/jakob/go/src/github.com/rfjakob/gocryptfs/internal/fusefrontend/node.go:367:3: ineffectual assignment to err
/home/jakob/go/src/github.com/rfjakob/gocryptfs/internal/fusefrontend/node_open_create.go:68:2: ineffectual assignment to fd
/home/jakob/go/src/github.com/rfjakob/gocryptfs/mount.go:308:2: ineffectual assignment to masterkey
/home/jakob/go/src/github.com/rfjakob/gocryptfs/gocryptfs-xray/xray_main.go:156:13: ineffectual assignment to err
/home/jakob/go/src/github.com/rfjakob/gocryptfs/internal/fusefrontend/prepare_syscall_test.go:65:16: ineffectual assignment to errno
/home/jakob/go/src/github.com/rfjakob/gocryptfs/internal/syscallcompat/open_nofollow_test.go:34:2: ineffectual assignment to fd
/home/jakob/go/src/github.com/rfjakob/gocryptfs/tests/defaults/acl_test.go:111:6: ineffectual assignment to err
/home/jakob/go/src/github.com/rfjakob/gocryptfs/tests/defaults/acl_test.go:181:2: ineffectual assignment to sz
/home/jakob/go/src/github.com/rfjakob/gocryptfs/tests/defaults/acl_test.go:198:2: ineffectual assignment to sz
/home/jakob/go/src/github.com/rfjakob/gocryptfs/tests/defaults/main_test.go:365:8: ineffectual assignment to err
/home/jakob/go/src/github.com/rfjakob/gocryptfs/tests/xattr/xattr_fd_test.go:30:6: ineffectual assignment to err
/home/jakob/go/src/github.com/rfjakob/gocryptfs/tests/xattr/xattr_fd_test.go:66:6: ineffectual assignment to err
2021-08-18 15:48:01 +02:00
Jakob Unterwurzacher
831e225616 syscallcompat: use BTRFS_SUPER_MAGIC from unix lib 2021-08-11 20:28:20 +02:00
Jakob Unterwurzacher
2d386fc92e syscallcompat: move quirks logic here & fix darwin
We need to look at f_fstypename acc. to
https://stackoverflow.com/a/52299141/1380267 :

> As filesystem type numbers are now assigned at runtime in
> recent versions of MacOS, you must use f_fstypename to
> determine the type.

https://github.com/rfjakob/gocryptfs/issues/585
2021-08-11 20:23:35 +02:00
Jakob Unterwurzacher
e48f2377ec syscallcompat: drop obsolete wrappers
These are now available cross-platform in the unix
package.
2021-06-05 15:06:30 +02:00
Jakob Unterwurzacher
a38e5988ba fusefrontend: run acl Setxattr in user context
The result of setting an acl depends on who runs the
operation!

Fixes fuse-xfstests generic/375
(see https://github.com/rfjakob/fuse-xfstests/wiki/results_2021-05-19)
2021-06-02 19:10:36 +02:00
Jakob Unterwurzacher
1b3c3b1347 syscallcompat: add GetdentsSpecial()
GetdentsSpecial calls then Getdents syscall,
with normal entries and "." / ".." split into two slices.
2021-05-26 13:17:56 +02:00
Jakob Unterwurzacher
09870bfac5 syscallcompat: also refactor MkdiratUser on GOOS=darwin
Breakage was:

+GOOS=darwin
+GOARCH=amd64
+go build -tags without_openssl
internal/fusefrontend/node_dir_ops.go:45:34: cannot use context (type *fuse.Context) as type *fuse.Caller in argument to syscallcompat.MkdiratUser
internal/fusefrontend/node_dir_ops.go:83:35: cannot use context (type *fuse.Context) as type *fuse.Caller in argument to syscallcompat.MkdiratUser
2021-05-22 22:01:46 +02:00
Jakob Unterwurzacher
e1853e1011 syscallcompat: refactor MkdiratUser to take fuse.Context
Let's have MkdiratUser take fuse.Context like everybody
else.
2021-05-22 21:44:19 +02:00
Jakob Unterwurzacher
cb4f9f9e29 syscallcompat: deduplicate OpenatUser/MknodatUser/SymlinkatUser/MkdiratUser
Turns out the whole euid switching logic can be shared when
wrapping the syscall in a closure.
2021-05-22 21:39:29 +02:00
Jakob Unterwurzacher
f6036c429a syscallcompat: getdents: link to #483
Give a user receiving the Getdents warning some background info.
2021-03-14 14:43:11 +01:00
Jakob Unterwurzacher
80a651a194 syscallcompat: MknodatUser: work around changed syscall.Setgroups semantics
Since go1.16beta1 (commit d1b1145cace8b968307f9311ff611e4bb810710c ,
https://go-review.googlesource.com/c/go/+/210639 )
syscall.{Setgroups,Setregid,Setreuid} affects all threads, which
is exactly what we not want.

We now use unix.{Setgroups,Setregid,Setreuid} instead.

Workarounds https://github.com/golang/go/issues/1435 .
2021-02-06 11:38:25 +01:00
Jakob Unterwurzacher
832e58cad4 Drop two more generated files
These were committed by mistake.
2020-10-19 19:27:47 +02:00
Jakob Unterwurzacher
165bf6c849 Drop generated files
These were committed by mistake.
2020-10-19 19:25:47 +02:00
Jakob Unterwurzacher
6697ffd6e2 fusefronted: reject GETXATTR "security.capability"
Unless we are mounted with -suid, we can reject
these requests, and gain back some lost speed.

Closes https://github.com/rfjakob/gocryptfs/issues/515
2020-10-18 21:07:30 +02:00
Jakob Unterwurzacher
c943ed32aa syscallcompat: add getxattr fastpaths
The allocations from Lgetxattr were #1 in the tar extract
allocation profile (caused by security.capability lookups).
No more!

$ benchstat old.txt new.txt
name         old time/op  new time/op  delta
Lgetxattr-4  15.2µs ± 0%   1.8µs ± 0%   ~     (p=1.000 n=1+1)

$ ./benchmark.bash
Testing gocryptfs at /tmp/benchmark.bash.H8p: gocryptfs v2.0-beta1-4-g95ea738-dirty; go-fuse v2.0.4-0.20200908172753-0b6cbc515082 => github.com/rfjakob/go-fuse/v2 v2.0.4-0.20201015204057-88b12c99f8af; 2020-10-18 go1.15.3 linux/amd64
/tmp/benchmark.bash.H8p.mnt is a mountpoint
WRITE: 262144000 bytes (262 MB, 250 MiB) copied, 0,520109 s, 504 MB/s
READ:  262144000 bytes (262 MB, 250 MiB) copied, 0,255672 s, 1,0 GB/s
UNTAR: 30,238
MD5:   12,721
LS:    10,038
RM:    16,536
2020-10-18 00:25:42 +02:00
Jakob Unterwurzacher
83a324a46b syscallcompat: add Lgetxattr benchmark 2020-10-16 20:04:22 +02:00
Jakob Unterwurzacher
ec3eaf0b87 syscallcompat: don't retry Close()
After Close() returns, the fd is dead, even if we
received EINTR. Don't retry, we could shoot down
an unrelated fd that received the same fd number.
2020-10-14 13:40:12 +02:00
Jakob Unterwurzacher
af4c1fb7a3 syscallcompat: retry ops on EINTR
Retry operations that have been shown to throw EINTR
errors on CIFS.

Todo: Solution for this pain in the back:

	warning: unix.Getdents returned errno 2 in the middle of data
	rm: cannot remove 'linux-3.0.old3/Documentation/ABI/removed': Input/output error

Progress towards fixing https://github.com/rfjakob/gocryptfs/issues/483 .
2020-10-14 00:35:16 +02:00
Jakob Unterwurzacher
803fdf410b syscallcompat: Openat: retry on EINTR
Towards fixing https://github.com/rfjakob/gocryptfs/issues/507
2020-10-11 01:31:09 +02:00
Jakob Unterwurzacher
8b1df08b8a syscallcompat: add Renameat2 for Darwin 2020-09-09 11:16:29 +02:00
Jakob Unterwurzacher
84344834c4 v2api: remove OpenatUserCtx, MknodatUserCtx helpers
Instead, use the new toFuseCtx() function introduced
in an earlier commit.
2020-07-11 19:44:45 +02:00
Jakob Unterwurzacher
b971c75e67 v2api: implement Mknod 2020-07-11 19:23:04 +02:00
Jakob Unterwurzacher
192a29075a v2api: implement Mkdir 2020-06-21 13:46:08 +02:00
Jakob Unterwurzacher
f6ded09e36 v2api: implement Create 2020-06-21 13:25:12 +02:00
Jakob Unterwurzacher
6aa9f5636f v2api: implement Lookup()
Compiles, but untested otherwise. No caching.
2020-06-21 12:01:34 +02:00
Jakob Unterwurzacher
15ff79bf14 syscallcompat: warn when Getdents truncates data
On CIFS mounts, unix.Getdents can return sudden ENOENT
in the middle of data. This will not be reported as an error
by user space tools, so return EIO instead.

Also log it as a warning.

https://github.com/rfjakob/gocryptfs/issues/483
2020-05-24 23:30:25 +02:00
Jakob Unterwurzacher
25f1727de9 syscallcompat: getdents: retry on EINTR
Fixes: https://github.com/rfjakob/gocryptfs/issues/483
Related: https://github.com/golang/go/issues/38836
2020-05-23 22:54:23 +02:00
Jakob Unterwurzacher
ec74d1d2f4 Update go-fuse import path to github.com/hanwen/go-fuse/v2
We need
fd7328faf9
to fix a crash reported in https://github.com/rfjakob/gocryptfs/issues/430 :

  2019/10/30 17:14:16 Unknown opcode 2016
  panic: runtime error: invalid memory address or nil pointer dereference
  [signal SIGSEGV: segmentation violation code=0x1 addr=0x20 pc=0x508d38]

This patch is only in the v2.x.x branch. Upgrade to v2, as the
old API is also supported there.

Running

  git grep hanwen/go-fuse | grep -v hanwen/go-fuse/v2

to check for forgotten references comes back clean.
2020-05-17 14:23:47 +02:00
Jakob Unterwurzacher
b1468a732f Fix unix2syscall_darwin.go build failure
Error was

 +GOOS=darwin
 +GOARCH=amd64
 +go build -tags without_openssl
 # github.com/rfjakob/gocryptfs/internal/syscallcompat
 internal/syscallcompat/unix2syscall_darwin.go:22:32: u.Atimespec undefined (type unix.Stat_t has no field or method Atimespec)
 internal/syscallcompat/unix2syscall_darwin.go:23:32: u.Mtimespec undefined (type unix.Stat_t has no field or method Mtimespec)
 internal/syscallcompat/unix2syscall_darwin.go:24:32: u.Ctimespec undefined (type unix.Stat_t has no field or method Ctimespec)

caused by 87c872767d (diff-4913a9178621eadcdf191db17915fbcb)
2019-05-19 21:04:33 +02:00
Sebastian Lackner
a97d14c42d syscallcompat: fetch supplementary groups for OpenatUser & friends
Handled the same way in GlusterFS, disorderfs, libfuse.
Fixes https://github.com/rfjakob/gocryptfs/issues/394
2019-05-01 17:52:07 +02:00
Jakob Unterwurzacher
3ac9872230 tests: split testParentDir by UID
When we run tests as root, they will leave root-owned files
in testParentDir, which causes trouble when we run tests as
a normal user later on. Split by UID.
2019-05-01 13:12:44 +02:00
Jakob Unterwurzacher
3d6b2685fb Revert "syscallcompat: drop Faccessat AT_SYMLINK_NOFOLLOW helper"
Breaks mounting on MacOS: unix.Faccessat on Darwin does NOT (yet)
support AT_SYMLINK_NOFOLLOW. See d44fe89ba4 .

This reverts commit 0805a63df1.
2019-01-20 13:10:59 +01:00
Jakob Unterwurzacher
0805a63df1 syscallcompat: drop Faccessat AT_SYMLINK_NOFOLLOW helper
unix.Faccessat has added support for AT_SYMLINK_NOFOLLOW in July 2018,
bd9dbc187b (diff-341484dbbe3180cd7a31ef2ad2d679b6)
which means we no longer need our own helper.

Closes https://github.com/rfjakob/gocryptfs/issues/347
2019-01-20 12:59:59 +01:00
Sebastian Lackner
682e642cfa fusefrontend: Rework the Utimens handling on macOS.
For Linux, everything effectively stays the same. For both path-based and
fd-based Utimens() calls, we use unix.UtimesNanoAt(). To avoid introducing
a separate syscall wrapper for futimens() (as done in go-fuse, for example),
we instead use the /proc/self/fd - trick.

On macOS, this changes quite a lot:

* Path-based Utimens() calls were previously completely broken, since
  unix.UtimensNanoAt() ignores the passed file descriptor. Note that this
  cannot be fixed easily since there IS no appropriate syscall available on
  macOS prior to High Sierra (10.13). We emulate this case by using
  Fchdir() + setattrlist().

* Fd-based Utimens() calls were previously translated to f.GetAttr() (to
  fill any empty parameters) and syscall.Futimes(), which does not does
  support nanosecond precision. Both issues can be fixed by switching to
  fsetattrlist().

Fixes https://github.com/rfjakob/gocryptfs/issues/350
2019-01-16 20:55:20 +01:00
Jakob Unterwurzacher
20140e24ed tests: reduce noise on MacOS
This should get rid of

    Openat: O_NOFOLLOW missing: flags = 0x0
    Fchmodat: adding missing AT_SYMLINK_NOFOLLOW flag
    sys_common_test.go:203: chmod on symlink should have failed, but did not. New mode=0333
    UnmountErr: "[...]/057376762.mnt" was not found in MountInfo, cannot check for FD leak

and add some context to

    --- FAIL: TestUtimesNano (0.00s)
    matrix_test.go:628: no such file or directory

See https://github.com/rfjakob/gocryptfs/pull/343#issuecomment-453888006
for full test output
2019-01-14 22:11:15 +01:00
Jakob Unterwurzacher
6542ddd2f9 syscallcompat: fix FchmodatNofollow tests
FchmodatNofollow dropped the flags parameter.
2019-01-14 21:57:24 +01:00
Jakob Unterwurzacher
a7d59032d3 syscallcompat: rework Fchmodat to FchmodatNofollow
We never want Fchmodat to follow symlinks, so follow what
Qemu does, and call our function FchmodatNofollow.
2019-01-14 21:54:16 +01:00
Sebastian Lackner
a9d8eb49ef syscallcompat: Drop Fstatat emulation on macOS. 2019-01-14 21:27:28 +01:00
Sebastian Lackner
4134ff7570 syscallcompat: Drop Mkdirat emulation on macOS. 2019-01-14 21:27:28 +01:00
Sebastian Lackner
7b0d56fe98 syscallcompat: Drop Symlinkat emulation on macOS. 2019-01-14 21:27:28 +01:00
Sebastian Lackner
92110628ee syscallcompat: Drop Fchownat emulation on macOS. 2019-01-14 21:27:28 +01:00
Sebastian Lackner
0345cc0830 syscallcompat: Drop Fchmodat emulation on macOS.
On macOS the function has a flags argument, so we don't need the
/proc/self/fd trick used on Linux.
2019-01-14 21:27:28 +01:00
Sebastian Lackner
229a9da74b syscallcompat: Drop Unlinkat emulation on macOS. 2019-01-14 21:27:28 +01:00