Commit Graph

366 Commits

Author SHA1 Message Date
Jakob Unterwurzacher
2d386fc92e syscallcompat: move quirks logic here & fix darwin
We need to look at f_fstypename acc. to
https://stackoverflow.com/a/52299141/1380267 :

> As filesystem type numbers are now assigned at runtime in
> recent versions of MacOS, you must use f_fstypename to
> determine the type.

https://github.com/rfjakob/gocryptfs/issues/585
2021-08-11 20:23:35 +02:00
Jakob Unterwurzacher
1bc1db620b fusefrontend: -sharedstorage: present stable inode numbers
Use the Gen field (inode generation) to distinguish hard links
while passing the real inode numbers to userspace.

Fixes https://github.com/rfjakob/gocryptfs/issues/584
2021-07-31 13:24:25 +02:00
Jakob Unterwurzacher
eecbcbb090 tests: matrix: add TestPwd
https://github.com/rfjakob/gocryptfs/issues/584
2021-07-31 13:23:05 +02:00
Jakob Unterwurzacher
0ca302f12a fusefrontend: implement fsync on directories
Fixes https://github.com/rfjakob/gocryptfs/issues/587
2021-07-29 20:39:50 +02:00
Jakob Unterwurzacher
db81614cd6 canonical-benchmarks.bash: handle relative paths
Passing a relative path was broken because we cd'ed
away first.
2021-06-27 11:48:41 +02:00
Jakob Unterwurzacher
49507ea869 tests/fsck: delete obsolete script run_fsck.bash
Not called by anybody.
2021-06-26 19:27:58 +02:00
Jakob Unterwurzacher
ad3eeaedc5 tests, maxlen.bash: speed up TestMaxlen using QUICK=1
From >6 to <1 second.
2021-06-26 19:13:24 +02:00
Jakob Unterwurzacher
446c3d7e93 tests: matrix: show content detail on mismatch 2021-06-26 18:58:29 +02:00
Jakob Unterwurzacher
1f29542b39 tests: better error message on ctlsock query failure 2021-06-26 18:49:54 +02:00
Jakob Unterwurzacher
cbd5e8ba01 tests/default: add maxlen.bash test 2021-06-26 16:09:04 +02:00
Jakob Unterwurzacher
05b813f202 nametransform: rename BadNameFlag to BadnameSuffix 2021-06-21 12:12:44 +02:00
Jakob Unterwurzacher
2efef1e270 nametransform: delete NameTransformer interface
Useless layer of indirection.
2021-06-21 11:53:33 +02:00
Jakob Unterwurzacher
e244b51491 tests: cli: add TestZerokey
TestZerokey verifies that `gocryptfs -zerokey` uses the same options as
`gocryptfs -init`.
2021-06-21 11:48:16 +02:00
DerDonut
a611810ff4 Badname file content access
This proposal is the counterpart of the modifications from the `-badname`
parameter. It modifies the plain -> cipher mapping for filenames when using
`-badname` parameter. The new function `EncryptAndHashBadName` tries to find a
cipher filename for the given plain name with the following steps:

1. If `badname` is disabled or direct mapping is successful: Map directly
(default and current behaviour)

2. If a file with badname flag has a valid cipher file, this is returned
(=File just ends with the badname flag)

3. If a file with a badname flag exists where only the badname flag was added,
this is returned (=File cipher name could not be decrypted by function
`DecryptName` and just the badname flag was added)

4. Search for all files which cipher file name extists when cropping more and
more characters from the end. If only 1 file is found, return this

5. Return an error otherwise

This allows file access in the file browsers but most important it allows that
you rename files with undecryptable cipher names in the plain directories.
Renaming those files will then generate a proper cipher filename One
backdraft: When mounting the cipher dir with -badname parameter, you can never
create (or rename to) files whose file name ends with the badname file flag
(at the moment this is " GOCRYPTFS_BAD_NAME"). This will cause an error.

I modified the CLI test function to cover additional test cases. Test [Case
7](https://github.com/DerDonut/gocryptfs/blob/badnamecontent/tests/cli/cli_test.go#L712)
cannot be performed since the cli tests are executed in panic mode. The
testing is stopped on error. Since the function`DecryptName` produces internal
errors when hitting non-decryptable file names, this test was omitted.

This implementation is a proposal where I tried to change the minimum amount
of existing code. Another possibility would be instead of creating the new
function `EncryptAndHashBadName` to modify the signature of the existing
function `EncryptAndHashName(name string, iv []byte)` to
`EncryptAndHashName(name string, iv []byte, dirfd int)` and integrate the
functionality into this function directly. You may allow calling with dirfd=-1
or other invalid values an then performing the current functionality.
2021-06-20 18:09:21 +02:00
Jakob Unterwurzacher
9ac77d42d1 contrib/maxlen.bash: also test dir and path length
Move the script from tests to contrib as it may now
be useful to somebody else.

https://github.com/rfjakob/gocryptfs/issues/552
2021-06-20 18:09:21 +02:00
Jakob Unterwurzacher
17f859d3c4 fusefronted: report plaintext size on symlink creation
gocryptfs 2.0 introduced the regression that the size
reported at symlink creation was the ciphertext size,
which is wrong.

Report the plaintext size.

Fixes https://github.com/rfjakob/gocryptfs/issues/574
2021-06-06 19:22:16 +02:00
Jakob Unterwurzacher
04858ddd22 nametransform: check name validity on encryption
xfstests generic/523 discovered that we allowed to set
xattrs with "/" in the name, but did not allow to read
them later.

With this change we do not allow to set them in the first
place.
2021-06-02 14:29:48 +02:00
Jakob Unterwurzacher
0e37fbe042 tests: TestFileHoleCopy: accept +-4kB
Failure looked like this:

--- FAIL: TestFileHoleCopy (3.73s)
    --- FAIL: TestFileHoleCopy/k81 (0.04s)
        file_holes_test.go:93: size changed: st0.Blocks=88 st2.Blocks=96
    file_holes_test.go:147: aborting further subtests

$ findholes TestFileHoleCopy.k81.1
         0 data
     36864 hole
     45056 data
     50434 hole
     50434 eof

$ findholes TestFileHoleCopy.k81.2
         0 data
     36864 hole
     45056 data
     50434 hole
     50434 eof

$ filefrag -v TestFileHoleCopy.k81.1
Filesystem type is: ef53
File size of TestFileHoleCopy.k81.1 is 50434 (13 blocks of 4096 bytes)
 ext:     logical_offset:        physical_offset: length:   expected: flags:
   0:        0..       2:   23702311..  23702313:      3:
   1:        3..       8:   20389855..  20389860:      6:   23702314:
   2:       11..      12:   23702314..  23702315:      2:   20389863: last,eof
TestFileHoleCopy.k81.1: 3 extents found

$ filefrag -v TestFileHoleCopy.k81.2
Filesystem type is: ef53
File size of TestFileHoleCopy.k81.2 is 50434 (13 blocks of 4096 bytes)
 ext:     logical_offset:        physical_offset: length:   expected: flags:
   0:        0..       2:   20389861..  20389863:      3:
   1:        3..       4:   23702316..  23702317:      2:   20389864:
   2:        5..       6:   20389864..  20389865:      2:   23702318:
   3:        7..       8:   23702318..  23702319:      2:   20389866:
   4:       11..      12:   23702320..  23702321:      2:             last,eof
TestFileHoleCopy.k81.2: 4 extents found
2021-05-29 16:56:20 +02:00
Jakob Unterwurzacher
738a9e006a fusefrontend: rewrite Lseek SEEK_DATA / SEEK_HOLE
In response to the discussion of the xfstests mailing list [1],
I looked at the Lseek implementation, which was naive and
did not handle all cases correctly.

The new implementation aligns the returned values to 4096 bytes
as most callers expect.

A lot of tests are added to verify that we handle all
cases correctly now.

[1]: https://www.spinics.net/lists/fstests/msg16554.html
2021-05-29 16:05:36 +02:00
Jakob Unterwurzacher
9695f0e524 tests: add TestFileHoleCopy
Currently fails.
2021-05-26 13:17:56 +02:00
Jakob Unterwurzacher
629d0a5ad7 tests: re-enable TestInoReuseEvil
Problem in go-fuse has long been fixed.
2021-05-26 13:17:56 +02:00
Jakob Unterwurzacher
7d72baca69 tests: add TestHaveDotdot
As discovered by xfstests generic/401 [1], during the move to
the v2 api we seem to have lost the "." and ".." directory
entries.

[1]: 4ef5b032bc/screenlog.0 (L520)
2021-05-26 13:17:56 +02:00
Jakob Unterwurzacher
86d8336b43 Add -acl flag to enable ACL enforcement
With test to verify that it actually works this
time: Run "make root_test".

Depends-on: https://github.com/rfjakob/gocryptfs/issues/536
Fixes: https://github.com/rfjakob/gocryptfs/issues/536
2021-05-15 17:58:37 +02:00
Jakob Unterwurzacher
beab7004f2 tests: sharedstorage: wait 100ms longer for cache expiry
With 1.0 seconds we see failures on Travis, example:
https://travis-ci.org/github/rfjakob/gocryptfs/builds/765648739

With 1.1 seconds it seems to always work.
2021-04-02 20:22:26 +02:00
Jakob Unterwurzacher
a2effaae39 tests: sharedstorage: add TestStaleHardlinks 2021-03-30 15:11:28 +02:00
Jakob Unterwurzacher
d7d79aa81c Add tests/sharedstorage
Towards better test coverage of shared backing storage
mounts.

https://github.com/rfjakob/gocryptfs/issues/525
2021-03-21 11:10:18 +01:00
Jakob Unterwurzacher
6da2a69018 test_helpers: VerifySize: don't complain about ino mismatch
The inode number is not stable with `-sharedstorage`.
Ignore it.

Failure was like this:

--- FAIL: TestFallocate (0.02s)
    helpers.go:229: Stat vs Fstat mismatch:
        st= {59 11543 1 33188 1026 1026 0 0 0 4096 8 {1616315569 838232716} {1616315569 838232716} {1616315569 838232716} [0 0 0]}
        st2={59 11545 1 33188 1026 1026 0 0 0 4096 8 {1616315569 838232716} {1616315569 838232716} {1616315569 838232716} [0 0 0]}
2021-03-21 10:53:51 +01:00
Jakob Unterwurzacher
692a79461a tests: matrix: add -sharestorage to test matrix
Currently fails like this:

=== RUN   TestRmwRace
doRead 0: corrupt block #0: cipher: message authentication failed
-wpanic turns this warning into a panic: doRead 0: corrupt block #0: cipher: message authentication failed
panic: -wpanic turns this warning into a panic: doRead 0: corrupt block #0: cipher: message authentication failed

goroutine 1293 [running]:
log.(*Logger).Panic(0xc00011c230, 0xc0003b17c8, 0x1, 0x1)
	log/log.go:224 +0xac
github.com/rfjakob/gocryptfs/internal/tlog.(*toggledLogger).Printf(0xc00007a780, 0x55a821a766a1, 0x20, 0xc0003b19f0, 0x3, 0x3)
	github.com/rfjakob/gocryptfs/internal/tlog/log.go:78 +0x1ef
github.com/rfjakob/gocryptfs/internal/fusefrontend.(*File).doRead(0xc0001ff420, 0x0, 0x0, 0x0, 0x0, 0x1000, 0x0, 0x1, 0xc000880000, 0x1020)
	github.com/rfjakob/gocryptfs/internal/fusefrontend/file.go:201 +0x8c9
github.com/rfjakob/gocryptfs/internal/fusefrontend.(*File).doWrite(0xc0001ff420, 0xc000248428, 0x10, 0x30, 0xff0, 0x3, 0x18)
	github.com/rfjakob/gocryptfs/internal/fusefrontend/file.go:291 +0xc9e
github.com/rfjakob/gocryptfs/internal/fusefrontend.(*File).Write(0xc0001ff420, 0x55a821b306a0, 0xc000fbde90, 0xc000248428, 0x10, 0x30, 0xff0, 0x7f4a00000000, 0x0)
	github.com/rfjakob/gocryptfs/internal/fusefrontend/file.go:378 +0x25e
github.com/hanwen/go-fuse/v2/fs.(*rawBridge).Write(0xc000168140, 0xc000096000, 0xc0002483d8, 0xc000248428, 0x10, 0x30, 0x55a821ad40e0)
	github.com/hanwen/go-fuse/v2@v2.0.4-0.20210125162859-8e0bbdb16cb7/fs/bridge.go:819 +0x26d
github.com/hanwen/go-fuse/v2/fuse.doWrite(0xc000170160, 0xc000248240)
	github.com/hanwen/go-fuse/v2@v2.0.4-0.20210125162859-8e0bbdb16cb7/fuse/opcode.go:191 +0x6f
github.com/hanwen/go-fuse/v2/fuse.(*Server).handleRequest(0xc000170160, 0xc000248240, 0xc000000000)
	github.com/hanwen/go-fuse/v2@v2.0.4-0.20210125162859-8e0bbdb16cb7/fuse/server.go:472 +0x2be
github.com/hanwen/go-fuse/v2/fuse.(*Server).loop(0xc000170160, 0xc000cd4101)
	github.com/hanwen/go-fuse/v2@v2.0.4-0.20210125162859-8e0bbdb16cb7/fuse/server.go:445 +0x198
created by github.com/hanwen/go-fuse/v2/fuse.(*Server).readRequest
	github.com/hanwen/go-fuse/v2@v2.0.4-0.20210125162859-8e0bbdb16cb7/fuse/server.go:312 +0x41d
    matrix_test.go:354: Write failed
2021-03-20 18:27:34 +01:00
Jakob Unterwurzacher
255a71c917 tests: MountOrFatal creates mnt dir itself
Allows to drop a few Mkdir()s.
2021-03-20 10:33:34 +01:00
Jakob Unterwurzacher
952d45ce84 tests: add TestDiskFull
Also fix incomplete uid restoration in TestSupplementaryGroups
and replace syscall.Setregid and friends with unix.Setregid and
friends.

This test is added to check if have problems handling a full disk.
The ticket https://github.com/rfjakob/gocryptfs/issues/550 states
that the full disk was not where the backing gocryptfs filesystem
was, but this has no effect on gocryptfs, so we test the harder
case.
2021-03-12 19:25:25 +01:00
Jakob Unterwurzacher
e2dc52a965 v2api: -sharestorage: disable hard link tracking & add tests
Hard link tracking was not correctly disabled
since the migration to the go-fuse v2 api.

Add a test to ensure it stays off.

Fixes https://github.com/rfjakob/gocryptfs/issues/525
2021-03-07 17:22:29 +01:00
Jakob Unterwurzacher
eaca820e87 fusefrontend: do not encrypt ACLs
Pass through system.posix_acl_access and system.posix_acl_default
unencrypted to fix "cp -a" problems.

"cp -a" uses "setxattr" even to set normal permissions, see
https://www.spinics.net/lists/linux-nfs/msg63986.html .

Fixes https://github.com/rfjakob/gocryptfs/issues/543
2021-02-07 20:01:16 +01:00
Jakob Unterwurzacher
bb2484f152 tests/xattr: fix acl blob
The blob was truncated to 31 bytes.
2021-02-07 20:00:33 +01:00
Jakob Unterwurzacher
bed60101f4 nametransform: make gocryptfs.diriv and gocryptfs.xxx.name files world-readable
Make `gocryptfs.diriv` and `gocryptfs.xxx.name` files world-readable to make encrypted backups easier
when mounting via fstab.

Having the files follow chmod/chown of their parent does not seem
to be worth the hassle. The content of the diriv files is not
secret, and both diriv and name files are protected by the
perms of the parent dir.

Fixes https://github.com/rfjakob/gocryptfs/issues/539
2021-01-10 08:07:10 +01:00
Jakob Unterwurzacher
754c483870 tests: fsstress-gocryptfs.bash: log timestamp for each iteration 2020-10-04 00:12:46 +02:00
Jakob Unterwurzacher
66449bf56b tests: fsstress-gocryptfs.bash: add DEBUG option
Also add a wrapper script, fsstress.collect-crashes.sh, to collect
the debug output.

https://github.com/hanwen/go-fuse/issues/372
2020-10-04 00:12:25 +02:00
Jakob Unterwurzacher
3c5a80c27b fsstress-gocryptfs.bash: don't hang if TMPDIR ends in /
We would hang like this

   ./fsstress-loopback.bash
  Recompile go-fuse loopback: v2.0.3-7-g0b6cbc5
  Waiting for mount: xxxxxxxx^C

if TMPDIR has a trailing /. The reason is that the
paths in /proc/self/mounts are normalized, while
TMPDIR may not be.
2020-10-03 14:23:37 +02:00
Jakob Unterwurzacher
27c92f63be fsstress-gocryptfs.bash: print loopback version & mount path
To make the used go-fuse version clear in logs,
print it on startup, similar to what we do with gocryptfs.
2020-09-18 18:31:22 +02:00
Jakob Unterwurzacher
af7386713c stress_tests/fsstress-gocryptfs.bash: use rm -Rf for cleanup
Apparently, kernel 5.8 now allows unprivileged "mknod /tmp/x c 0 0":

	vfs: allow unprivileged whiteout creation
	https://git.kernel.org/pub/scm/linux/kernel/git/stable/linux.git/commit/?h=v5.8.9&id=a3c751a50fe6bbe50eb7622a14b18b361804ee0c

which is why rm throws a new prompt:

	rm: remove write-protected character special file '...'?

Use rm -Rf to suppress that.
2020-09-13 14:14:23 +02:00
Jakob Unterwurzacher
6a9c49e9cf tests/plaintextnames: add TestInoReuseEvil 2020-09-12 17:55:37 +02:00
Jakob Unterwurzacher
db61ec5115 tests: add TestInoReuse 2020-09-09 18:18:37 +02:00
Jakob Unterwurzacher
6577f1b146 test_helpers: print warning when not running on ext4
ext4 reuses inode numbers, tmpfs does not.
2020-09-09 11:47:17 +02:00
Jakob Unterwurzacher
7c0363dee5 test_helpers: mark MountOrFatal as a Helper() 2020-09-09 11:15:54 +02:00
Jakob Unterwurzacher
993b19c19c gocryptfs -init: fix wrong exit code on non-empty dir
Fixes https://github.com/rfjakob/gocryptfs/pull/503
2020-09-06 11:35:25 +02:00
Jakob Unterwurzacher
3806a8cc93 tests/reverse: implement (skipped) xattr test
v1api reverse mode did not have xattr support,
the v2api version may have at some point. Prep the
test already.
2020-08-30 16:36:10 +02:00
Jakob Unterwurzacher
d212b246c5 v2api/reverse: implement Lseek 2020-08-16 19:58:47 +02:00
Jakob Unterwurzacher
58a2726977 v2api/reverse: implement Statfs 2020-08-16 19:41:00 +02:00
Jakob Unterwurzacher
94e8fc12ea v2api/reverse: finish -exclude
Tests pass now.
2020-08-15 17:31:45 +02:00
Jakob Unterwurzacher
f270135c16 test_helper: VerifyExistence: don't panic
Instead bubble up the error to the testing object.
2020-08-15 15:39:08 +02:00
Jakob Unterwurzacher
9e0b07ec99 stress_tests: run pingpong.bash at nice level 19
Like extractloop.bash.
2020-07-26 20:01:30 +02:00