Commit Graph

1917 Commits

Author SHA1 Message Date
Jakob Unterwurzacher b23e21f61f fusefrontend: catch ReadAt integer overflow
Discovered by xfstests generic/564 .

Failure was:

generic/564	- output mismatch (see /opt/fuse-xfstests/results//generic/564.out.bad)
    --- tests/generic/564.out	2021-05-08 21:11:05.307395966 +0200
    +++ /opt/fuse-xfstests/results//generic/564.out.bad	2021-05-19 19:01:16.912888879 +0200
    @@ -31,7 +31,7 @@
     source range beyond 8TiB returns 0

     destination range beyond 8TiB returns EFBIG
    -copy_range: File too large
    +copy_range: Function not implemented
2021-06-02 18:20:05 +02:00
Jakob Unterwurzacher 04858ddd22 nametransform: check name validity on encryption
xfstests generic/523 discovered that we allowed to set
xattrs with "/" in the name, but did not allow to read
them later.

With this change we do not allow to set them in the first
place.
2021-06-02 14:29:48 +02:00
Jakob Unterwurzacher 242cdf966f go.mod: update go-fuse to get acl fixes
Done using:

go get github.com/hanwen/go-fuse/v2@master
go mod tidy
2021-05-30 09:34:03 +02:00
Jakob Unterwurzacher 198eb9797b Revert "go.mod: switch to go-fuse acl branch"
Change https://review.gerrithub.io/c/hanwen/go-fuse/+/516154
was merged upstream.

This reverts commit 3374afccc4.
2021-05-30 09:31:56 +02:00
Jakob Unterwurzacher 0e37fbe042 tests: TestFileHoleCopy: accept +-4kB
Failure looked like this:

--- FAIL: TestFileHoleCopy (3.73s)
    --- FAIL: TestFileHoleCopy/k81 (0.04s)
        file_holes_test.go:93: size changed: st0.Blocks=88 st2.Blocks=96
    file_holes_test.go:147: aborting further subtests

$ findholes TestFileHoleCopy.k81.1
         0 data
     36864 hole
     45056 data
     50434 hole
     50434 eof

$ findholes TestFileHoleCopy.k81.2
         0 data
     36864 hole
     45056 data
     50434 hole
     50434 eof

$ filefrag -v TestFileHoleCopy.k81.1
Filesystem type is: ef53
File size of TestFileHoleCopy.k81.1 is 50434 (13 blocks of 4096 bytes)
 ext:     logical_offset:        physical_offset: length:   expected: flags:
   0:        0..       2:   23702311..  23702313:      3:
   1:        3..       8:   20389855..  20389860:      6:   23702314:
   2:       11..      12:   23702314..  23702315:      2:   20389863: last,eof
TestFileHoleCopy.k81.1: 3 extents found

$ filefrag -v TestFileHoleCopy.k81.2
Filesystem type is: ef53
File size of TestFileHoleCopy.k81.2 is 50434 (13 blocks of 4096 bytes)
 ext:     logical_offset:        physical_offset: length:   expected: flags:
   0:        0..       2:   20389861..  20389863:      3:
   1:        3..       4:   23702316..  23702317:      2:   20389864:
   2:        5..       6:   20389864..  20389865:      2:   23702318:
   3:        7..       8:   23702318..  23702319:      2:   20389866:
   4:       11..      12:   23702320..  23702321:      2:             last,eof
TestFileHoleCopy.k81.2: 4 extents found
2021-05-29 16:56:20 +02:00
Jakob Unterwurzacher 18befda0e6 fusefrontend: list "." and ".." in dir entries
Fixes xfstests generic/401
2021-05-29 16:44:38 +02:00
Jakob Unterwurzacher 738a9e006a fusefrontend: rewrite Lseek SEEK_DATA / SEEK_HOLE
In response to the discussion of the xfstests mailing list [1],
I looked at the Lseek implementation, which was naive and
did not handle all cases correctly.

The new implementation aligns the returned values to 4096 bytes
as most callers expect.

A lot of tests are added to verify that we handle all
cases correctly now.

[1]: https://www.spinics.net/lists/fstests/msg16554.html
2021-05-29 16:05:36 +02:00
Jakob Unterwurzacher c1d7e38761 findholes: add --create, --verify
Also, change the logic for the segment walk to not
rely on the total size. cp does not use the total
size either, and we miss bugs by cheating!
2021-05-29 16:00:40 +02:00
Jakob Unterwurzacher 07164cbb3a contentenc: add PlainOffToCipherOff helper
Will be used for improving Lseek()
2021-05-26 18:28:59 +02:00
Jakob Unterwurzacher b4794bedec contentenc: fix CipherSizeToPlainSize non-monoticity
For an illegal cipherSize, pretend we have an additional
1-byte block.

See code comment for details.
2021-05-26 13:17:56 +02:00
Jakob Unterwurzacher bebd7ed81f contentenc: update comments
Also, replace one open-coded calculation with a
helper function.
2021-05-26 13:17:56 +02:00
Jakob Unterwurzacher 2a5ac3e9ba tests: contentenc: add TestSizeToSize
TestSizeToSize tests CipherSizeToPlainSize and PlainSizeToCipherSize.

Fails at the moment due to CipherSizeToPlainSize non-moniticity.
2021-05-26 13:17:56 +02:00
Jakob Unterwurzacher 9695f0e524 tests: add TestFileHoleCopy
Currently fails.
2021-05-26 13:17:56 +02:00
Jakob Unterwurzacher 629d0a5ad7 tests: re-enable TestInoReuseEvil
Problem in go-fuse has long been fixed.
2021-05-26 13:17:56 +02:00
Jakob Unterwurzacher 8bccd3b4bf Add contrib/findholes
Utility and libs to find hole/data segments using lseek.
2021-05-26 13:17:56 +02:00
Jakob Unterwurzacher 1b3c3b1347 syscallcompat: add GetdentsSpecial()
GetdentsSpecial calls then Getdents syscall,
with normal entries and "." / ".." split into two slices.
2021-05-26 13:17:56 +02:00
Jakob Unterwurzacher 7d72baca69 tests: add TestHaveDotdot
As discovered by xfstests generic/401 [1], during the move to
the v2 api we seem to have lost the "." and ".." directory
entries.

[1]: 4ef5b032bc/screenlog.0 (L520)
2021-05-26 13:17:56 +02:00
Jakob Unterwurzacher 1d2ac1e589 stupidgcm: prefer Go stdlib over OpenSSL on Apple M1
https://github.com/rfjakob/gocryptfs/issues/556
2021-05-26 09:20:22 +02:00
Jakob Unterwurzacher 09870bfac5 syscallcompat: also refactor MkdiratUser on GOOS=darwin
Breakage was:

+GOOS=darwin
+GOARCH=amd64
+go build -tags without_openssl
internal/fusefrontend/node_dir_ops.go:45:34: cannot use context (type *fuse.Context) as type *fuse.Caller in argument to syscallcompat.MkdiratUser
internal/fusefrontend/node_dir_ops.go:83:35: cannot use context (type *fuse.Context) as type *fuse.Caller in argument to syscallcompat.MkdiratUser
2021-05-22 22:01:46 +02:00
Jakob Unterwurzacher e1853e1011 syscallcompat: refactor MkdiratUser to take fuse.Context
Let's have MkdiratUser take fuse.Context like everybody
else.
2021-05-22 21:44:19 +02:00
Jakob Unterwurzacher cb4f9f9e29 syscallcompat: deduplicate OpenatUser/MknodatUser/SymlinkatUser/MkdiratUser
Turns out the whole euid switching logic can be shared when
wrapping the syscall in a closure.
2021-05-22 21:39:29 +02:00
Jakob Unterwurzacher 0650a512bb fsck: redirect go-fuse noise to syslog 2021-05-18 18:38:23 +02:00
Jakob Unterwurzacher 5da20da977 -speed: note that XChaCha20 is not selectable
This is unclear to users, as seen on
https://github.com/rfjakob/gocryptfs/issues/452#issuecomment-828836651
2021-05-18 09:53:23 +02:00
Jakob Unterwurzacher c33c7aaf0d Merge package-source.bash & package-static.bash scripts 2021-05-15 18:50:20 +02:00
Jakob Unterwurzacher 4488b5b93a Update README for v2.0-beta4 2021-05-15 18:27:41 +02:00
Jakob Unterwurzacher 3374afccc4 go.mod: switch to go-fuse acl branch
Until https://review.gerrithub.io/c/hanwen/go-fuse/+/516154
gets merged.

Commands used:

go mod edit -replace github.com/hanwen/go-fuse/v2=github.com/rfjakob/go-fuse/v2@acl
go mod download github.com/hanwen/go-fuse/v2
go mod tidy
2021-05-15 18:01:51 +02:00
Jakob Unterwurzacher 86d8336b43 Add -acl flag to enable ACL enforcement
With test to verify that it actually works this
time: Run "make root_test".

Depends-on: https://github.com/rfjakob/gocryptfs/issues/536
Fixes: https://github.com/rfjakob/gocryptfs/issues/536
2021-05-15 17:58:37 +02:00
Jakob Unterwurzacher a91ad29d36 fusefrontend: fix RENAME_NOREPLACE darwin build failure
Error was:

internal/fusefrontend/node.go:371:2: duplicate case syscallcompat.RENAME_NOREPLACE (value 0) in switch
	previous case at internal/fusefrontend/node.go:368:7

Rewrite using "if"s instead.
2021-05-15 17:20:43 +02:00
Jakob Unterwurzacher 1ba2e42234 fusefrontend: avoid duplicate const definition
RENAME_NOREPLACE is already defined in syscallcompat.
2021-05-08 20:42:42 +02:00
Jakob Unterwurzacher a267562d29 fusefrontend: reject broken RENAME_EXCHANGE and RENAME_WHITEOUT
Discovered by xfstests generic/013: or implementation for
RENAME_EXCHANGE and RENAME_WHITEOUT is incomplete. Reject the
flags so that the caller retries with regular rename.
2021-05-08 15:39:49 +02:00
Jakob Unterwurzacher 7466f12a92 README: fix broken markdown links 2021-05-07 20:58:42 +02:00
Jakob Unterwurzacher c57fd9adc9 Update README for v2.0-beta3 release 2021-04-24 12:26:54 +02:00
Jakob Unterwurzacher e678737967 go mod: switch back to mainline go-fuse
Change was merged:
a90e1f463c

Done using:

$ go mod edit -dropreplace github.com/hanwen/go-fuse/v2
$ go get github.com/hanwen/go-fuse/v2@master
go: downloading github.com/hanwen/go-fuse/v2 v2.1.1-0.20210423170155-a90e1f463c3f
go get: upgraded github.com/hanwen/go-fuse/v2 v2.0.4-0.20210125162859-8e0bbdb16cb7 => v2.1.1-0.20210423170155-a90e1f463c3f

Related: https://github.com/rfjakob/gocryptfs/issues/549
2021-04-24 12:23:17 +02:00
Jakob Unterwurzacher 5cb1e55714 Doc: update performance.txt 2021-04-07 07:21:20 +02:00
Jakob Unterwurzacher 4a07d6598c fusefrontend: make dirCache work for "node itself"
"node itself" can be converted to node + child by
ascending one level.

Performance gains are spectacular, as will be seen
in the next commit.
2021-04-07 07:18:35 +02:00
Jakob Unterwurzacher 770c4deb71 profiling/ls.bash: add -nosyslog
This makes sure we actually see the dirCache stats.
2021-04-05 18:20:59 +02:00
Jakob Unterwurzacher f73aee72f8 fusefrontend: print dirCache stats after unmount 2021-04-05 18:20:17 +02:00
Jakob Unterwurzacher 043f81dd01 fs: more idiomatic dirCache struct naming 2021-04-04 13:05:47 +02:00
Jakob Unterwurzacher dc8501f6b2 performance.txt: add dirfd caching results
Also, add v2.0-beta2-16-geaca820. I bisected the ls performance
regression to this commit.
2021-04-03 14:09:10 +02:00
Jakob Unterwurzacher 24d5d39300 fs: add initial dirfd caching
dirfd caching was temporarily removed when moving
to the v2api. Add it back to gain back some lost speed.
2021-04-03 13:08:28 +02:00
Jakob Unterwurzacher 6aae2aad97 tests: fusefronted: fix TestOpenBackingDir
This test only worked accidentially, and would
break once dirfd caching is added.

fs.Mkdir(..., "dir1/dir2", ...) is illegal
(child name cannot contain slashes).
2021-04-03 12:34:18 +02:00
Jakob Unterwurzacher 6b28c0be2c fusefronted: replace last rn.openBackingDir() calls
Use the n.prepareAtSyscall() wrapper instead. Prepares
for adding caching into n.prepareAtSyscall().
2021-04-03 11:19:04 +02:00
Jakob Unterwurzacher beab7004f2 tests: sharedstorage: wait 100ms longer for cache expiry
With 1.0 seconds we see failures on Travis, example:
https://travis-ci.org/github/rfjakob/gocryptfs/builds/765648739

With 1.1 seconds it seems to always work.
2021-04-02 20:22:26 +02:00
Jakob Unterwurzacher 688373c926 profiling: add ls.bash 2021-03-30 16:03:42 +02:00
Jakob Unterwurzacher 1d919d01fd go.mod: temporarily switch to go-fuse fork
Until https://review.gerrithub.io/c/hanwen/go-fuse/+/513646/2
gets merged.

Commands used:

go mod edit -replace github.com/hanwen/go-fuse/v2=github.com/rfjakob/go-fuse/v2@TestStaleHardlinks
go mod download github.com/hanwen/go-fuse/v2
go mod tidy
2021-03-30 15:31:17 +02:00
Jakob Unterwurzacher a2effaae39 tests: sharedstorage: add TestStaleHardlinks 2021-03-30 15:11:28 +02:00
Jakob Unterwurzacher e0981ea59b package-static.bash: add gocryptfs-xray to tarball
Requested here https://github.com/rfjakob/gocryptfs/issues/558
and here  https://github.com/rfjakob/gocryptfs/issues/496

Closes https://github.com/rfjakob/gocryptfs/issues/496
2021-03-27 15:30:01 +01:00
Jakob Unterwurzacher db91050f9e gocrypts-xray: add -version flag 2021-03-27 15:29:46 +01:00
Jakob Unterwurzacher b86d40de5f Delete obsolete script package.bash
I only use package-static.bash nowadays.
2021-03-27 15:03:27 +01:00
Jakob Unterwurzacher 7e18ee6b86 README: fix compile instructions for Go 1.13+
"go get -d" does not download to GOPATH/src anymore:
https://github.com/golang/go/issues/31529

Use explicit "git clone" to the current directory as suggested
in https://github.com/golang/go/issues/31529#issuecomment-484920157 .

Fixes https://github.com/rfjakob/gocryptfs/issues/553
2021-03-26 07:10:05 +01:00