Commit Graph

1953 Commits

Author SHA1 Message Date
Jakob Unterwurzacher 8bccd3b4bf Add contrib/findholes
Utility and libs to find hole/data segments using lseek.
2021-05-26 13:17:56 +02:00
Jakob Unterwurzacher 1b3c3b1347 syscallcompat: add GetdentsSpecial()
GetdentsSpecial calls then Getdents syscall,
with normal entries and "." / ".." split into two slices.
2021-05-26 13:17:56 +02:00
Jakob Unterwurzacher 7d72baca69 tests: add TestHaveDotdot
As discovered by xfstests generic/401 [1], during the move to
the v2 api we seem to have lost the "." and ".." directory
entries.

[1]: 4ef5b032bc/screenlog.0 (L520)
2021-05-26 13:17:56 +02:00
Jakob Unterwurzacher 1d2ac1e589 stupidgcm: prefer Go stdlib over OpenSSL on Apple M1
https://github.com/rfjakob/gocryptfs/issues/556
2021-05-26 09:20:22 +02:00
Jakob Unterwurzacher 09870bfac5 syscallcompat: also refactor MkdiratUser on GOOS=darwin
Breakage was:

+GOOS=darwin
+GOARCH=amd64
+go build -tags without_openssl
internal/fusefrontend/node_dir_ops.go:45:34: cannot use context (type *fuse.Context) as type *fuse.Caller in argument to syscallcompat.MkdiratUser
internal/fusefrontend/node_dir_ops.go:83:35: cannot use context (type *fuse.Context) as type *fuse.Caller in argument to syscallcompat.MkdiratUser
2021-05-22 22:01:46 +02:00
Jakob Unterwurzacher e1853e1011 syscallcompat: refactor MkdiratUser to take fuse.Context
Let's have MkdiratUser take fuse.Context like everybody
else.
2021-05-22 21:44:19 +02:00
Jakob Unterwurzacher cb4f9f9e29 syscallcompat: deduplicate OpenatUser/MknodatUser/SymlinkatUser/MkdiratUser
Turns out the whole euid switching logic can be shared when
wrapping the syscall in a closure.
2021-05-22 21:39:29 +02:00
Jakob Unterwurzacher 0650a512bb fsck: redirect go-fuse noise to syslog 2021-05-18 18:38:23 +02:00
Jakob Unterwurzacher 5da20da977 -speed: note that XChaCha20 is not selectable
This is unclear to users, as seen on
https://github.com/rfjakob/gocryptfs/issues/452#issuecomment-828836651
2021-05-18 09:53:23 +02:00
Jakob Unterwurzacher c33c7aaf0d Merge package-source.bash & package-static.bash scripts 2021-05-15 18:50:20 +02:00
Jakob Unterwurzacher 4488b5b93a Update README for v2.0-beta4 2021-05-15 18:27:41 +02:00
Jakob Unterwurzacher 3374afccc4 go.mod: switch to go-fuse acl branch
Until https://review.gerrithub.io/c/hanwen/go-fuse/+/516154
gets merged.

Commands used:

go mod edit -replace github.com/hanwen/go-fuse/v2=github.com/rfjakob/go-fuse/v2@acl
go mod download github.com/hanwen/go-fuse/v2
go mod tidy
2021-05-15 18:01:51 +02:00
Jakob Unterwurzacher 86d8336b43 Add -acl flag to enable ACL enforcement
With test to verify that it actually works this
time: Run "make root_test".

Depends-on: https://github.com/rfjakob/gocryptfs/issues/536
Fixes: https://github.com/rfjakob/gocryptfs/issues/536
2021-05-15 17:58:37 +02:00
Jakob Unterwurzacher a91ad29d36 fusefrontend: fix RENAME_NOREPLACE darwin build failure
Error was:

internal/fusefrontend/node.go:371:2: duplicate case syscallcompat.RENAME_NOREPLACE (value 0) in switch
	previous case at internal/fusefrontend/node.go:368:7

Rewrite using "if"s instead.
2021-05-15 17:20:43 +02:00
Jakob Unterwurzacher 1ba2e42234 fusefrontend: avoid duplicate const definition
RENAME_NOREPLACE is already defined in syscallcompat.
2021-05-08 20:42:42 +02:00
Jakob Unterwurzacher a267562d29 fusefrontend: reject broken RENAME_EXCHANGE and RENAME_WHITEOUT
Discovered by xfstests generic/013: or implementation for
RENAME_EXCHANGE and RENAME_WHITEOUT is incomplete. Reject the
flags so that the caller retries with regular rename.
2021-05-08 15:39:49 +02:00
Jakob Unterwurzacher 7466f12a92 README: fix broken markdown links 2021-05-07 20:58:42 +02:00
Jakob Unterwurzacher c57fd9adc9 Update README for v2.0-beta3 release 2021-04-24 12:26:54 +02:00
Jakob Unterwurzacher e678737967 go mod: switch back to mainline go-fuse
Change was merged:
a90e1f463c

Done using:

$ go mod edit -dropreplace github.com/hanwen/go-fuse/v2
$ go get github.com/hanwen/go-fuse/v2@master
go: downloading github.com/hanwen/go-fuse/v2 v2.1.1-0.20210423170155-a90e1f463c3f
go get: upgraded github.com/hanwen/go-fuse/v2 v2.0.4-0.20210125162859-8e0bbdb16cb7 => v2.1.1-0.20210423170155-a90e1f463c3f

Related: https://github.com/rfjakob/gocryptfs/issues/549
2021-04-24 12:23:17 +02:00
Jakob Unterwurzacher 5cb1e55714 Doc: update performance.txt 2021-04-07 07:21:20 +02:00
Jakob Unterwurzacher 4a07d6598c fusefrontend: make dirCache work for "node itself"
"node itself" can be converted to node + child by
ascending one level.

Performance gains are spectacular, as will be seen
in the next commit.
2021-04-07 07:18:35 +02:00
Jakob Unterwurzacher 770c4deb71 profiling/ls.bash: add -nosyslog
This makes sure we actually see the dirCache stats.
2021-04-05 18:20:59 +02:00
Jakob Unterwurzacher f73aee72f8 fusefrontend: print dirCache stats after unmount 2021-04-05 18:20:17 +02:00
Jakob Unterwurzacher 043f81dd01 fs: more idiomatic dirCache struct naming 2021-04-04 13:05:47 +02:00
Jakob Unterwurzacher dc8501f6b2 performance.txt: add dirfd caching results
Also, add v2.0-beta2-16-geaca820. I bisected the ls performance
regression to this commit.
2021-04-03 14:09:10 +02:00
Jakob Unterwurzacher 24d5d39300 fs: add initial dirfd caching
dirfd caching was temporarily removed when moving
to the v2api. Add it back to gain back some lost speed.
2021-04-03 13:08:28 +02:00
Jakob Unterwurzacher 6aae2aad97 tests: fusefronted: fix TestOpenBackingDir
This test only worked accidentially, and would
break once dirfd caching is added.

fs.Mkdir(..., "dir1/dir2", ...) is illegal
(child name cannot contain slashes).
2021-04-03 12:34:18 +02:00
Jakob Unterwurzacher 6b28c0be2c fusefronted: replace last rn.openBackingDir() calls
Use the n.prepareAtSyscall() wrapper instead. Prepares
for adding caching into n.prepareAtSyscall().
2021-04-03 11:19:04 +02:00
Jakob Unterwurzacher beab7004f2 tests: sharedstorage: wait 100ms longer for cache expiry
With 1.0 seconds we see failures on Travis, example:
https://travis-ci.org/github/rfjakob/gocryptfs/builds/765648739

With 1.1 seconds it seems to always work.
2021-04-02 20:22:26 +02:00
Jakob Unterwurzacher 688373c926 profiling: add ls.bash 2021-03-30 16:03:42 +02:00
Jakob Unterwurzacher 1d919d01fd go.mod: temporarily switch to go-fuse fork
Until https://review.gerrithub.io/c/hanwen/go-fuse/+/513646/2
gets merged.

Commands used:

go mod edit -replace github.com/hanwen/go-fuse/v2=github.com/rfjakob/go-fuse/v2@TestStaleHardlinks
go mod download github.com/hanwen/go-fuse/v2
go mod tidy
2021-03-30 15:31:17 +02:00
Jakob Unterwurzacher a2effaae39 tests: sharedstorage: add TestStaleHardlinks 2021-03-30 15:11:28 +02:00
Jakob Unterwurzacher e0981ea59b package-static.bash: add gocryptfs-xray to tarball
Requested here https://github.com/rfjakob/gocryptfs/issues/558
and here  https://github.com/rfjakob/gocryptfs/issues/496

Closes https://github.com/rfjakob/gocryptfs/issues/496
2021-03-27 15:30:01 +01:00
Jakob Unterwurzacher db91050f9e gocrypts-xray: add -version flag 2021-03-27 15:29:46 +01:00
Jakob Unterwurzacher b86d40de5f Delete obsolete script package.bash
I only use package-static.bash nowadays.
2021-03-27 15:03:27 +01:00
Jakob Unterwurzacher 7e18ee6b86 README: fix compile instructions for Go 1.13+
"go get -d" does not download to GOPATH/src anymore:
https://github.com/golang/go/issues/31529

Use explicit "git clone" to the current directory as suggested
in https://github.com/golang/go/issues/31529#issuecomment-484920157 .

Fixes https://github.com/rfjakob/gocryptfs/issues/553
2021-03-26 07:10:05 +01:00
Jakob Unterwurzacher d7d79aa81c Add tests/sharedstorage
Towards better test coverage of shared backing storage
mounts.

https://github.com/rfjakob/gocryptfs/issues/525
2021-03-21 11:10:18 +01:00
Jakob Unterwurzacher 6da2a69018 test_helpers: VerifySize: don't complain about ino mismatch
The inode number is not stable with `-sharedstorage`.
Ignore it.

Failure was like this:

--- FAIL: TestFallocate (0.02s)
    helpers.go:229: Stat vs Fstat mismatch:
        st= {59 11543 1 33188 1026 1026 0 0 0 4096 8 {1616315569 838232716} {1616315569 838232716} {1616315569 838232716} [0 0 0]}
        st2={59 11545 1 33188 1026 1026 0 0 0 4096 8 {1616315569 838232716} {1616315569 838232716} {1616315569 838232716} [0 0 0]}
2021-03-21 10:53:51 +01:00
Jakob Unterwurzacher 3b9a1b628b fusefronted: move Create() and Open() to new file
And deduplicate the code a little.
2021-03-21 09:31:05 +01:00
Jakob Unterwurzacher 47a4d33f24 fusefrontend: -sharedstorage: fix TestRmwRace failure
The Open() and Create() paths used different inode numbers,
which broke openfiletable locking against concurred readers.
2021-03-21 09:02:50 +01:00
Jakob Unterwurzacher 692a79461a tests: matrix: add -sharestorage to test matrix
Currently fails like this:

=== RUN   TestRmwRace
doRead 0: corrupt block #0: cipher: message authentication failed
-wpanic turns this warning into a panic: doRead 0: corrupt block #0: cipher: message authentication failed
panic: -wpanic turns this warning into a panic: doRead 0: corrupt block #0: cipher: message authentication failed

goroutine 1293 [running]:
log.(*Logger).Panic(0xc00011c230, 0xc0003b17c8, 0x1, 0x1)
	log/log.go:224 +0xac
github.com/rfjakob/gocryptfs/internal/tlog.(*toggledLogger).Printf(0xc00007a780, 0x55a821a766a1, 0x20, 0xc0003b19f0, 0x3, 0x3)
	github.com/rfjakob/gocryptfs/internal/tlog/log.go:78 +0x1ef
github.com/rfjakob/gocryptfs/internal/fusefrontend.(*File).doRead(0xc0001ff420, 0x0, 0x0, 0x0, 0x0, 0x1000, 0x0, 0x1, 0xc000880000, 0x1020)
	github.com/rfjakob/gocryptfs/internal/fusefrontend/file.go:201 +0x8c9
github.com/rfjakob/gocryptfs/internal/fusefrontend.(*File).doWrite(0xc0001ff420, 0xc000248428, 0x10, 0x30, 0xff0, 0x3, 0x18)
	github.com/rfjakob/gocryptfs/internal/fusefrontend/file.go:291 +0xc9e
github.com/rfjakob/gocryptfs/internal/fusefrontend.(*File).Write(0xc0001ff420, 0x55a821b306a0, 0xc000fbde90, 0xc000248428, 0x10, 0x30, 0xff0, 0x7f4a00000000, 0x0)
	github.com/rfjakob/gocryptfs/internal/fusefrontend/file.go:378 +0x25e
github.com/hanwen/go-fuse/v2/fs.(*rawBridge).Write(0xc000168140, 0xc000096000, 0xc0002483d8, 0xc000248428, 0x10, 0x30, 0x55a821ad40e0)
	github.com/hanwen/go-fuse/v2@v2.0.4-0.20210125162859-8e0bbdb16cb7/fs/bridge.go:819 +0x26d
github.com/hanwen/go-fuse/v2/fuse.doWrite(0xc000170160, 0xc000248240)
	github.com/hanwen/go-fuse/v2@v2.0.4-0.20210125162859-8e0bbdb16cb7/fuse/opcode.go:191 +0x6f
github.com/hanwen/go-fuse/v2/fuse.(*Server).handleRequest(0xc000170160, 0xc000248240, 0xc000000000)
	github.com/hanwen/go-fuse/v2@v2.0.4-0.20210125162859-8e0bbdb16cb7/fuse/server.go:472 +0x2be
github.com/hanwen/go-fuse/v2/fuse.(*Server).loop(0xc000170160, 0xc000cd4101)
	github.com/hanwen/go-fuse/v2@v2.0.4-0.20210125162859-8e0bbdb16cb7/fuse/server.go:445 +0x198
created by github.com/hanwen/go-fuse/v2/fuse.(*Server).readRequest
	github.com/hanwen/go-fuse/v2@v2.0.4-0.20210125162859-8e0bbdb16cb7/fuse/server.go:312 +0x41d
    matrix_test.go:354: Write failed
2021-03-20 18:27:34 +01:00
Jakob Unterwurzacher 255a71c917 tests: MountOrFatal creates mnt dir itself
Allows to drop a few Mkdir()s.
2021-03-20 10:33:34 +01:00
Jakob Unterwurzacher f6036c429a syscallcompat: getdents: link to #483
Give a user receiving the Getdents warning some background info.
2021-03-14 14:43:11 +01:00
Jakob Unterwurzacher 952d45ce84 tests: add TestDiskFull
Also fix incomplete uid restoration in TestSupplementaryGroups
and replace syscall.Setregid and friends with unix.Setregid and
friends.

This test is added to check if have problems handling a full disk.
The ticket https://github.com/rfjakob/gocryptfs/issues/550 states
that the full disk was not where the backing gocryptfs filesystem
was, but this has no effect on gocryptfs, so we test the harder
case.
2021-03-12 19:25:25 +01:00
Jakob Unterwurzacher d47bf9976f Makefile: root_test: don't run test when compile fails 2021-03-12 17:44:13 +01:00
Jakob Unterwurzacher e2dc52a965 v2api: -sharestorage: disable hard link tracking & add tests
Hard link tracking was not correctly disabled
since the migration to the go-fuse v2 api.

Add a test to ensure it stays off.

Fixes https://github.com/rfjakob/gocryptfs/issues/525
2021-03-07 17:22:29 +01:00
Jakob Unterwurzacher eaca820e87 fusefrontend: do not encrypt ACLs
Pass through system.posix_acl_access and system.posix_acl_default
unencrypted to fix "cp -a" problems.

"cp -a" uses "setxattr" even to set normal permissions, see
https://www.spinics.net/lists/linux-nfs/msg63986.html .

Fixes https://github.com/rfjakob/gocryptfs/issues/543
2021-02-07 20:01:16 +01:00
Jakob Unterwurzacher bb2484f152 tests/xattr: fix acl blob
The blob was truncated to 31 bytes.
2021-02-07 20:00:33 +01:00
Jakob Unterwurzacher 4b4a68eb21 README: warn about incomplete ACL support
https://github.com/rfjakob/gocryptfs/issues/543
https://github.com/rfjakob/gocryptfs/issues/542
https://github.com/rfjakob/gocryptfs/issues/453
2021-02-07 10:43:40 +01:00
Jakob Unterwurzacher 5d498f3a24 README: mention MacFUSE v4.x support
https://github.com/rfjakob/gocryptfs/issues/544
2021-02-06 16:47:27 +01:00