Commit Graph

1779 Commits

Author SHA1 Message Date
Jakob Unterwurzacher
a08ef90811 crossbuild.bash: skip Apple Silicon build on old Go versions 2021-06-06 21:10:24 +02:00
Jakob Unterwurzacher
e44d4fcb96 crossbuild.bash: disable CGO
build-without-openssl.bash also disables CGO, so
this makes it more real-world-y.

But the real reason is that disabling CGO hopefully
fixes this travis ci build failure:

+GOOS=darwin
+GOARCH=arm64
+go build -tags without_openssl
/home/travis/.gimme/versions/go1.13.15.linux.amd64/pkg/tool/linux_amd64/link: running gcc failed: exit status 1
/usr/bin/ld: unrecognized option '-pagezero_size'
/usr/bin/ld: use the --help option for usage information
collect2: error: ld returned 1 exit status
The command "./crossbuild.bash" exited with 2.
2021-06-06 19:28:02 +02:00
Jakob Unterwurzacher
17f859d3c4 fusefronted: report plaintext size on symlink creation
gocryptfs 2.0 introduced the regression that the size
reported at symlink creation was the ciphertext size,
which is wrong.

Report the plaintext size.

Fixes https://github.com/rfjakob/gocryptfs/issues/574
2021-06-06 19:22:16 +02:00
Jakob Unterwurzacher
6910f86705 crossbuild.bash: also build for Apple M1 2021-06-05 15:07:18 +02:00
Jakob Unterwurzacher
e48f2377ec syscallcompat: drop obsolete wrappers
These are now available cross-platform in the unix
package.
2021-06-05 15:06:30 +02:00
Jakob Unterwurzacher
c0e7530216 Update README for v2.0 2021-06-05 14:47:57 +02:00
Jakob Unterwurzacher
5ed1a90c7e doc: add benchmark for v2.0, reformat table 2021-06-05 14:46:49 +02:00
Jakob Unterwurzacher
3a1f009c1a Add contrib/atomicrename
$ ./contrib/atomicrename/atomicrename -h
atomicrename creates 100 "src" files in the current directory, renames
them in random order over a single "dst" file while reading the "dst"
file concurrently in a loop.

Progress and errors are reported as they occour in addition to a summary
printed at the end. cifs and fuse filesystems are known to fail, local
filesystems and nfs seem ok.

See https://github.com/hanwen/go-fuse/issues/398 for background info.
2021-06-04 22:17:13 +02:00
Jakob Unterwurzacher
015cd066e1 fido2: quote argument strings in debug output
Tested using

  gocryptfs -init -debug -fido2 "hello world" cipherdir

Output before:

  callFidoCommand: executing "/usr/bin/fido2-cred" with args [fido2-cred -M -h -v hello world]

After:

  callFidoCommand: executing "/usr/bin/fido2-cred" with args ["fido2-cred" "-M" "-h" "-v" "hello world"]

Related: https://github.com/rfjakob/gocryptfs/issues/571
2021-06-03 22:03:21 +02:00
Jakob Unterwurzacher
8f2be5d93c fsck: mark temporary mount read-only
We don't write during fsck, but somebody else might try
to.
2021-06-02 19:32:20 +02:00
Jakob Unterwurzacher
307dfd7f71 fsck: clean up temporary mountpoint
We used to leave directories like

  /tmp/gocryptfs.fsck.104431245

behind. Let's clean up after ourselves.
2021-06-02 19:31:55 +02:00
Jakob Unterwurzacher
a38e5988ba fusefrontend: run acl Setxattr in user context
The result of setting an acl depends on who runs the
operation!

Fixes fuse-xfstests generic/375
(see https://github.com/rfjakob/fuse-xfstests/wiki/results_2021-05-19)
2021-06-02 19:10:36 +02:00
Jakob Unterwurzacher
b23e21f61f fusefrontend: catch ReadAt integer overflow
Discovered by xfstests generic/564 .

Failure was:

generic/564	- output mismatch (see /opt/fuse-xfstests/results//generic/564.out.bad)
    --- tests/generic/564.out	2021-05-08 21:11:05.307395966 +0200
    +++ /opt/fuse-xfstests/results//generic/564.out.bad	2021-05-19 19:01:16.912888879 +0200
    @@ -31,7 +31,7 @@
     source range beyond 8TiB returns 0

     destination range beyond 8TiB returns EFBIG
    -copy_range: File too large
    +copy_range: Function not implemented
2021-06-02 18:20:05 +02:00
Jakob Unterwurzacher
04858ddd22 nametransform: check name validity on encryption
xfstests generic/523 discovered that we allowed to set
xattrs with "/" in the name, but did not allow to read
them later.

With this change we do not allow to set them in the first
place.
2021-06-02 14:29:48 +02:00
Jakob Unterwurzacher
242cdf966f go.mod: update go-fuse to get acl fixes
Done using:

go get github.com/hanwen/go-fuse/v2@master
go mod tidy
2021-05-30 09:34:03 +02:00
Jakob Unterwurzacher
198eb9797b Revert "go.mod: switch to go-fuse acl branch"
Change https://review.gerrithub.io/c/hanwen/go-fuse/+/516154
was merged upstream.

This reverts commit 3374afccc4.
2021-05-30 09:31:56 +02:00
Jakob Unterwurzacher
0e37fbe042 tests: TestFileHoleCopy: accept +-4kB
Failure looked like this:

--- FAIL: TestFileHoleCopy (3.73s)
    --- FAIL: TestFileHoleCopy/k81 (0.04s)
        file_holes_test.go:93: size changed: st0.Blocks=88 st2.Blocks=96
    file_holes_test.go:147: aborting further subtests

$ findholes TestFileHoleCopy.k81.1
         0 data
     36864 hole
     45056 data
     50434 hole
     50434 eof

$ findholes TestFileHoleCopy.k81.2
         0 data
     36864 hole
     45056 data
     50434 hole
     50434 eof

$ filefrag -v TestFileHoleCopy.k81.1
Filesystem type is: ef53
File size of TestFileHoleCopy.k81.1 is 50434 (13 blocks of 4096 bytes)
 ext:     logical_offset:        physical_offset: length:   expected: flags:
   0:        0..       2:   23702311..  23702313:      3:
   1:        3..       8:   20389855..  20389860:      6:   23702314:
   2:       11..      12:   23702314..  23702315:      2:   20389863: last,eof
TestFileHoleCopy.k81.1: 3 extents found

$ filefrag -v TestFileHoleCopy.k81.2
Filesystem type is: ef53
File size of TestFileHoleCopy.k81.2 is 50434 (13 blocks of 4096 bytes)
 ext:     logical_offset:        physical_offset: length:   expected: flags:
   0:        0..       2:   20389861..  20389863:      3:
   1:        3..       4:   23702316..  23702317:      2:   20389864:
   2:        5..       6:   20389864..  20389865:      2:   23702318:
   3:        7..       8:   23702318..  23702319:      2:   20389866:
   4:       11..      12:   23702320..  23702321:      2:             last,eof
TestFileHoleCopy.k81.2: 4 extents found
2021-05-29 16:56:20 +02:00
Jakob Unterwurzacher
18befda0e6 fusefrontend: list "." and ".." in dir entries
Fixes xfstests generic/401
2021-05-29 16:44:38 +02:00
Jakob Unterwurzacher
738a9e006a fusefrontend: rewrite Lseek SEEK_DATA / SEEK_HOLE
In response to the discussion of the xfstests mailing list [1],
I looked at the Lseek implementation, which was naive and
did not handle all cases correctly.

The new implementation aligns the returned values to 4096 bytes
as most callers expect.

A lot of tests are added to verify that we handle all
cases correctly now.

[1]: https://www.spinics.net/lists/fstests/msg16554.html
2021-05-29 16:05:36 +02:00
Jakob Unterwurzacher
c1d7e38761 findholes: add --create, --verify
Also, change the logic for the segment walk to not
rely on the total size. cp does not use the total
size either, and we miss bugs by cheating!
2021-05-29 16:00:40 +02:00
Jakob Unterwurzacher
07164cbb3a contentenc: add PlainOffToCipherOff helper
Will be used for improving Lseek()
2021-05-26 18:28:59 +02:00
Jakob Unterwurzacher
b4794bedec contentenc: fix CipherSizeToPlainSize non-monoticity
For an illegal cipherSize, pretend we have an additional
1-byte block.

See code comment for details.
2021-05-26 13:17:56 +02:00
Jakob Unterwurzacher
bebd7ed81f contentenc: update comments
Also, replace one open-coded calculation with a
helper function.
2021-05-26 13:17:56 +02:00
Jakob Unterwurzacher
2a5ac3e9ba tests: contentenc: add TestSizeToSize
TestSizeToSize tests CipherSizeToPlainSize and PlainSizeToCipherSize.

Fails at the moment due to CipherSizeToPlainSize non-moniticity.
2021-05-26 13:17:56 +02:00
Jakob Unterwurzacher
9695f0e524 tests: add TestFileHoleCopy
Currently fails.
2021-05-26 13:17:56 +02:00
Jakob Unterwurzacher
629d0a5ad7 tests: re-enable TestInoReuseEvil
Problem in go-fuse has long been fixed.
2021-05-26 13:17:56 +02:00
Jakob Unterwurzacher
8bccd3b4bf Add contrib/findholes
Utility and libs to find hole/data segments using lseek.
2021-05-26 13:17:56 +02:00
Jakob Unterwurzacher
1b3c3b1347 syscallcompat: add GetdentsSpecial()
GetdentsSpecial calls then Getdents syscall,
with normal entries and "." / ".." split into two slices.
2021-05-26 13:17:56 +02:00
Jakob Unterwurzacher
7d72baca69 tests: add TestHaveDotdot
As discovered by xfstests generic/401 [1], during the move to
the v2 api we seem to have lost the "." and ".." directory
entries.

[1]: 4ef5b032bc/screenlog.0 (L520)
2021-05-26 13:17:56 +02:00
Jakob Unterwurzacher
1d2ac1e589 stupidgcm: prefer Go stdlib over OpenSSL on Apple M1
https://github.com/rfjakob/gocryptfs/issues/556
2021-05-26 09:20:22 +02:00
Jakob Unterwurzacher
09870bfac5 syscallcompat: also refactor MkdiratUser on GOOS=darwin
Breakage was:

+GOOS=darwin
+GOARCH=amd64
+go build -tags without_openssl
internal/fusefrontend/node_dir_ops.go:45:34: cannot use context (type *fuse.Context) as type *fuse.Caller in argument to syscallcompat.MkdiratUser
internal/fusefrontend/node_dir_ops.go:83:35: cannot use context (type *fuse.Context) as type *fuse.Caller in argument to syscallcompat.MkdiratUser
2021-05-22 22:01:46 +02:00
Jakob Unterwurzacher
e1853e1011 syscallcompat: refactor MkdiratUser to take fuse.Context
Let's have MkdiratUser take fuse.Context like everybody
else.
2021-05-22 21:44:19 +02:00
Jakob Unterwurzacher
cb4f9f9e29 syscallcompat: deduplicate OpenatUser/MknodatUser/SymlinkatUser/MkdiratUser
Turns out the whole euid switching logic can be shared when
wrapping the syscall in a closure.
2021-05-22 21:39:29 +02:00
Jakob Unterwurzacher
0650a512bb fsck: redirect go-fuse noise to syslog 2021-05-18 18:38:23 +02:00
Jakob Unterwurzacher
5da20da977 -speed: note that XChaCha20 is not selectable
This is unclear to users, as seen on
https://github.com/rfjakob/gocryptfs/issues/452#issuecomment-828836651
2021-05-18 09:53:23 +02:00
Jakob Unterwurzacher
c33c7aaf0d Merge package-source.bash & package-static.bash scripts 2021-05-15 18:50:20 +02:00
Jakob Unterwurzacher
4488b5b93a Update README for v2.0-beta4 2021-05-15 18:27:41 +02:00
Jakob Unterwurzacher
3374afccc4 go.mod: switch to go-fuse acl branch
Until https://review.gerrithub.io/c/hanwen/go-fuse/+/516154
gets merged.

Commands used:

go mod edit -replace github.com/hanwen/go-fuse/v2=github.com/rfjakob/go-fuse/v2@acl
go mod download github.com/hanwen/go-fuse/v2
go mod tidy
2021-05-15 18:01:51 +02:00
Jakob Unterwurzacher
86d8336b43 Add -acl flag to enable ACL enforcement
With test to verify that it actually works this
time: Run "make root_test".

Depends-on: https://github.com/rfjakob/gocryptfs/issues/536
Fixes: https://github.com/rfjakob/gocryptfs/issues/536
2021-05-15 17:58:37 +02:00
Jakob Unterwurzacher
a91ad29d36 fusefrontend: fix RENAME_NOREPLACE darwin build failure
Error was:

internal/fusefrontend/node.go:371:2: duplicate case syscallcompat.RENAME_NOREPLACE (value 0) in switch
	previous case at internal/fusefrontend/node.go:368:7

Rewrite using "if"s instead.
2021-05-15 17:20:43 +02:00
Jakob Unterwurzacher
1ba2e42234 fusefrontend: avoid duplicate const definition
RENAME_NOREPLACE is already defined in syscallcompat.
2021-05-08 20:42:42 +02:00
Jakob Unterwurzacher
a267562d29 fusefrontend: reject broken RENAME_EXCHANGE and RENAME_WHITEOUT
Discovered by xfstests generic/013: or implementation for
RENAME_EXCHANGE and RENAME_WHITEOUT is incomplete. Reject the
flags so that the caller retries with regular rename.
2021-05-08 15:39:49 +02:00
Jakob Unterwurzacher
7466f12a92 README: fix broken markdown links 2021-05-07 20:58:42 +02:00
Jakob Unterwurzacher
c57fd9adc9 Update README for v2.0-beta3 release 2021-04-24 12:26:54 +02:00
Jakob Unterwurzacher
e678737967 go mod: switch back to mainline go-fuse
Change was merged:
a90e1f463c

Done using:

$ go mod edit -dropreplace github.com/hanwen/go-fuse/v2
$ go get github.com/hanwen/go-fuse/v2@master
go: downloading github.com/hanwen/go-fuse/v2 v2.1.1-0.20210423170155-a90e1f463c3f
go get: upgraded github.com/hanwen/go-fuse/v2 v2.0.4-0.20210125162859-8e0bbdb16cb7 => v2.1.1-0.20210423170155-a90e1f463c3f

Related: https://github.com/rfjakob/gocryptfs/issues/549
2021-04-24 12:23:17 +02:00
Jakob Unterwurzacher
5cb1e55714 Doc: update performance.txt 2021-04-07 07:21:20 +02:00
Jakob Unterwurzacher
4a07d6598c fusefrontend: make dirCache work for "node itself"
"node itself" can be converted to node + child by
ascending one level.

Performance gains are spectacular, as will be seen
in the next commit.
2021-04-07 07:18:35 +02:00
Jakob Unterwurzacher
770c4deb71 profiling/ls.bash: add -nosyslog
This makes sure we actually see the dirCache stats.
2021-04-05 18:20:59 +02:00
Jakob Unterwurzacher
f73aee72f8 fusefrontend: print dirCache stats after unmount 2021-04-05 18:20:17 +02:00
Jakob Unterwurzacher
043f81dd01 fs: more idiomatic dirCache struct naming 2021-04-04 13:05:47 +02:00