Commit Graph

363 Commits

Author SHA1 Message Date
Jakob Unterwurzacher
4fd95b718b fusefrontend: delete openBackingDir 2021-06-26 18:49:54 +02:00
Jakob Unterwurzacher
5306fc345b fusefrontend: convert last callers from openBackingDir to prepareAtSyscall 2021-06-26 18:49:54 +02:00
Jakob Unterwurzacher
45648e567a fusefrontend: ctlsock: get rid of unneccessary wrapper function 2021-06-26 18:49:54 +02:00
Jakob Unterwurzacher
f9f4bd214f fusefrontend: convert ctlsock from openBackingDir to prepareAtSyscall
openBackingDir will be removed.

Also, remove leftover debug printfs.
2021-06-26 18:49:54 +02:00
Jakob Unterwurzacher
ee59b5269b fusefrontend: convert openBackingDir tests to prepareAtSyscall
openBackingDir will be removed.
2021-06-26 16:28:30 +02:00
Jakob Unterwurzacher
84e702126a fusefrontend: implement recursive diriv caching
The new contrib/maxlen.bash showed that we have exponential
runtime with respect to directory depth.

The new recursive diriv caching is a lot smarter as it caches
intermediate lookups. maxlen.bash now completes in a few seconds.

xfstests results same as
2d158e4c82/screenlog.0 :

  Failures: generic/035 generic/062 generic/080 generic/093 generic/099 generic/215 generic/285 generic/319 generic/426 generic/444 generic/467 generic/477 generic/523
  Failed 13 of 580 tests

benchmark.bash results are identical:

  $ ./benchmark.bash
  Testing gocryptfs at /tmp/benchmark.bash.BdQ: gocryptfs v2.0.1-17-g6b09bc0; go-fuse v2.1.1-0.20210611132105-24a1dfe6b4f8; 2021-06-25 go1.16.5 linux/amd64
  /tmp/benchmark.bash.BdQ.mnt is a mountpoint
  WRITE: 262144000 bytes (262 MB, 250 MiB) copied, 0,4821 s, 544 MB/s
  READ:  262144000 bytes (262 MB, 250 MiB) copied, 0,266061 s, 985 MB/s
  UNTAR: 8,280
  MD5:   4,564
  LS:    1,745
  RM:    2,244
2021-06-25 13:56:53 +02:00
Jakob Unterwurzacher
2efef1e270 nametransform: delete NameTransformer interface
Useless layer of indirection.
2021-06-21 11:53:33 +02:00
Jakob Unterwurzacher
c5d8fa83ae nametransform: pass badname patterns via New
This means we can unexport the field.
2021-06-20 19:09:46 +02:00
DerDonut
a611810ff4 Badname file content access
This proposal is the counterpart of the modifications from the `-badname`
parameter. It modifies the plain -> cipher mapping for filenames when using
`-badname` parameter. The new function `EncryptAndHashBadName` tries to find a
cipher filename for the given plain name with the following steps:

1. If `badname` is disabled or direct mapping is successful: Map directly
(default and current behaviour)

2. If a file with badname flag has a valid cipher file, this is returned
(=File just ends with the badname flag)

3. If a file with a badname flag exists where only the badname flag was added,
this is returned (=File cipher name could not be decrypted by function
`DecryptName` and just the badname flag was added)

4. Search for all files which cipher file name extists when cropping more and
more characters from the end. If only 1 file is found, return this

5. Return an error otherwise

This allows file access in the file browsers but most important it allows that
you rename files with undecryptable cipher names in the plain directories.
Renaming those files will then generate a proper cipher filename One
backdraft: When mounting the cipher dir with -badname parameter, you can never
create (or rename to) files whose file name ends with the badname file flag
(at the moment this is " GOCRYPTFS_BAD_NAME"). This will cause an error.

I modified the CLI test function to cover additional test cases. Test [Case
7](https://github.com/DerDonut/gocryptfs/blob/badnamecontent/tests/cli/cli_test.go#L712)
cannot be performed since the cli tests are executed in panic mode. The
testing is stopped on error. Since the function`DecryptName` produces internal
errors when hitting non-decryptable file names, this test was omitted.

This implementation is a proposal where I tried to change the minimum amount
of existing code. Another possibility would be instead of creating the new
function `EncryptAndHashBadName` to modify the signature of the existing
function `EncryptAndHashName(name string, iv []byte)` to
`EncryptAndHashName(name string, iv []byte, dirfd int)` and integrate the
functionality into this function directly. You may allow calling with dirfd=-1
or other invalid values an then performing the current functionality.
2021-06-20 18:09:21 +02:00
Jakob Unterwurzacher
17f859d3c4 fusefronted: report plaintext size on symlink creation
gocryptfs 2.0 introduced the regression that the size
reported at symlink creation was the ciphertext size,
which is wrong.

Report the plaintext size.

Fixes https://github.com/rfjakob/gocryptfs/issues/574
2021-06-06 19:22:16 +02:00
Jakob Unterwurzacher
e48f2377ec syscallcompat: drop obsolete wrappers
These are now available cross-platform in the unix
package.
2021-06-05 15:06:30 +02:00
Jakob Unterwurzacher
a38e5988ba fusefrontend: run acl Setxattr in user context
The result of setting an acl depends on who runs the
operation!

Fixes fuse-xfstests generic/375
(see https://github.com/rfjakob/fuse-xfstests/wiki/results_2021-05-19)
2021-06-02 19:10:36 +02:00
Jakob Unterwurzacher
b23e21f61f fusefrontend: catch ReadAt integer overflow
Discovered by xfstests generic/564 .

Failure was:

generic/564	- output mismatch (see /opt/fuse-xfstests/results//generic/564.out.bad)
    --- tests/generic/564.out	2021-05-08 21:11:05.307395966 +0200
    +++ /opt/fuse-xfstests/results//generic/564.out.bad	2021-05-19 19:01:16.912888879 +0200
    @@ -31,7 +31,7 @@
     source range beyond 8TiB returns 0

     destination range beyond 8TiB returns EFBIG
    -copy_range: File too large
    +copy_range: Function not implemented
2021-06-02 18:20:05 +02:00
Jakob Unterwurzacher
04858ddd22 nametransform: check name validity on encryption
xfstests generic/523 discovered that we allowed to set
xattrs with "/" in the name, but did not allow to read
them later.

With this change we do not allow to set them in the first
place.
2021-06-02 14:29:48 +02:00
Jakob Unterwurzacher
18befda0e6 fusefrontend: list "." and ".." in dir entries
Fixes xfstests generic/401
2021-05-29 16:44:38 +02:00
Jakob Unterwurzacher
738a9e006a fusefrontend: rewrite Lseek SEEK_DATA / SEEK_HOLE
In response to the discussion of the xfstests mailing list [1],
I looked at the Lseek implementation, which was naive and
did not handle all cases correctly.

The new implementation aligns the returned values to 4096 bytes
as most callers expect.

A lot of tests are added to verify that we handle all
cases correctly now.

[1]: https://www.spinics.net/lists/fstests/msg16554.html
2021-05-29 16:05:36 +02:00
Jakob Unterwurzacher
e1853e1011 syscallcompat: refactor MkdiratUser to take fuse.Context
Let's have MkdiratUser take fuse.Context like everybody
else.
2021-05-22 21:44:19 +02:00
Jakob Unterwurzacher
a91ad29d36 fusefrontend: fix RENAME_NOREPLACE darwin build failure
Error was:

internal/fusefrontend/node.go:371:2: duplicate case syscallcompat.RENAME_NOREPLACE (value 0) in switch
	previous case at internal/fusefrontend/node.go:368:7

Rewrite using "if"s instead.
2021-05-15 17:20:43 +02:00
Jakob Unterwurzacher
1ba2e42234 fusefrontend: avoid duplicate const definition
RENAME_NOREPLACE is already defined in syscallcompat.
2021-05-08 20:42:42 +02:00
Jakob Unterwurzacher
a267562d29 fusefrontend: reject broken RENAME_EXCHANGE and RENAME_WHITEOUT
Discovered by xfstests generic/013: or implementation for
RENAME_EXCHANGE and RENAME_WHITEOUT is incomplete. Reject the
flags so that the caller retries with regular rename.
2021-05-08 15:39:49 +02:00
Jakob Unterwurzacher
4a07d6598c fusefrontend: make dirCache work for "node itself"
"node itself" can be converted to node + child by
ascending one level.

Performance gains are spectacular, as will be seen
in the next commit.
2021-04-07 07:18:35 +02:00
Jakob Unterwurzacher
f73aee72f8 fusefrontend: print dirCache stats after unmount 2021-04-05 18:20:17 +02:00
Jakob Unterwurzacher
043f81dd01 fs: more idiomatic dirCache struct naming 2021-04-04 13:05:47 +02:00
Jakob Unterwurzacher
24d5d39300 fs: add initial dirfd caching
dirfd caching was temporarily removed when moving
to the v2api. Add it back to gain back some lost speed.
2021-04-03 13:08:28 +02:00
Jakob Unterwurzacher
6aae2aad97 tests: fusefronted: fix TestOpenBackingDir
This test only worked accidentially, and would
break once dirfd caching is added.

fs.Mkdir(..., "dir1/dir2", ...) is illegal
(child name cannot contain slashes).
2021-04-03 12:34:18 +02:00
Jakob Unterwurzacher
6b28c0be2c fusefronted: replace last rn.openBackingDir() calls
Use the n.prepareAtSyscall() wrapper instead. Prepares
for adding caching into n.prepareAtSyscall().
2021-04-03 11:19:04 +02:00
Jakob Unterwurzacher
3b9a1b628b fusefronted: move Create() and Open() to new file
And deduplicate the code a little.
2021-03-21 09:31:05 +01:00
Jakob Unterwurzacher
47a4d33f24 fusefrontend: -sharedstorage: fix TestRmwRace failure
The Open() and Create() paths used different inode numbers,
which broke openfiletable locking against concurred readers.
2021-03-21 09:02:50 +01:00
Jakob Unterwurzacher
e2dc52a965 v2api: -sharestorage: disable hard link tracking & add tests
Hard link tracking was not correctly disabled
since the migration to the go-fuse v2 api.

Add a test to ensure it stays off.

Fixes https://github.com/rfjakob/gocryptfs/issues/525
2021-03-07 17:22:29 +01:00
Jakob Unterwurzacher
eaca820e87 fusefrontend: do not encrypt ACLs
Pass through system.posix_acl_access and system.posix_acl_default
unencrypted to fix "cp -a" problems.

"cp -a" uses "setxattr" even to set normal permissions, see
https://www.spinics.net/lists/linux-nfs/msg63986.html .

Fixes https://github.com/rfjakob/gocryptfs/issues/543
2021-02-07 20:01:16 +01:00
Jakob Unterwurzacher
bed60101f4 nametransform: make gocryptfs.diriv and gocryptfs.xxx.name files world-readable
Make `gocryptfs.diriv` and `gocryptfs.xxx.name` files world-readable to make encrypted backups easier
when mounting via fstab.

Having the files follow chmod/chown of their parent does not seem
to be worth the hassle. The content of the diriv files is not
secret, and both diriv and name files are protected by the
perms of the parent dir.

Fixes https://github.com/rfjakob/gocryptfs/issues/539
2021-01-10 08:07:10 +01:00
gmd20
c20c7992a0 main: add "-kernel_cache" flag
This option is similar to fuse(8) kernel_cache

Verified using vmtouch.

Without -kernel_cache:

$ dd if=/dev/zero of=foo bs=1M count=10 ; vmtouch -t foo ; vmtouch foo
10+0 records in
10+0 records out
10485760 bytes (10 MB, 10 MiB) copied, 0,0242321 s, 433 MB/s
           Files: 1
     Directories: 0
   Touched Pages: 2560 (10M)
         Elapsed: 0.011159 seconds
           Files: 1
     Directories: 0
  Resident Pages: 0/2560  0/10M  0%
         Elapsed: 0.000993 seconds

With -kernel_cache:

$ dd if=/dev/zero of=foo bs=1M count=10 ; vmtouch -t foo ; vmtouch foo
10+0 records in
10+0 records out
10485760 bytes (10 MB, 10 MiB) copied, 0,0244015 s, 430 MB/s
           Files: 1
     Directories: 0
   Touched Pages: 2560 (10M)
         Elapsed: 0.011564 seconds
           Files: 1
     Directories: 0
  Resident Pages: 2560/2560  10M/10M  100%
         Elapsed: 0.000369 seconds
2020-12-20 09:55:04 +01:00
Jakob Unterwurzacher
6697ffd6e2 fusefronted: reject GETXATTR "security.capability"
Unless we are mounted with -suid, we can reject
these requests, and gain back some lost speed.

Closes https://github.com/rfjakob/gocryptfs/issues/515
2020-10-18 21:07:30 +02:00
Jakob Unterwurzacher
ec3eaf0b87 syscallcompat: don't retry Close()
After Close() returns, the fd is dead, even if we
received EINTR. Don't retry, we could shoot down
an unrelated fd that received the same fd number.
2020-10-14 13:40:12 +02:00
Jakob Unterwurzacher
af4c1fb7a3 syscallcompat: retry ops on EINTR
Retry operations that have been shown to throw EINTR
errors on CIFS.

Todo: Solution for this pain in the back:

	warning: unix.Getdents returned errno 2 in the middle of data
	rm: cannot remove 'linux-3.0.old3/Documentation/ABI/removed': Input/output error

Progress towards fixing https://github.com/rfjakob/gocryptfs/issues/483 .
2020-10-14 00:35:16 +02:00
Jakob Unterwurzacher
ac687d5359 v2api: add Darwin xattr support 2020-09-09 11:17:19 +02:00
Jakob Unterwurzacher
d212b246c5 v2api/reverse: implement Lseek 2020-08-16 19:58:47 +02:00
Jakob Unterwurzacher
6f3cca2cdd v2api: clean up api TODOs 2020-08-16 13:52:27 +02:00
Jakob Unterwurzacher
ee5ab1cc29 v2api: rename "File2" to just "File"
Rename the symbols and the files.
2020-08-16 12:50:33 +02:00
Jakob Unterwurzacher
1867fdaef4 v2api: move helpers from node.go to node_helpers.go 2020-08-01 19:26:42 +02:00
Jakob Unterwurzacher
4572cd2103 v2api: fix missing size translation in Lookup 2020-07-26 19:49:26 +02:00
Jakob Unterwurzacher
777b95f82f v2api: delete (most) fusefrontend v1 files
All the functionality in these files has been reimplemented
for the v2 api. Drop the old files.
2020-07-26 18:35:12 +02:00
Jakob Unterwurzacher
9cd24d79a2 v2api: implement Lseek
This also fixes the last remaining tests/fsck failure.
2020-07-23 22:55:07 +02:00
Jakob Unterwurzacher
49fc3abcb4 v2api: Lookup: use newChild() helper 2020-07-19 13:15:18 +02:00
Jakob Unterwurzacher
6b7ff09373 v2api: Node: make Path() public
Helpful for fsck.
2020-07-18 23:40:54 +02:00
Jakob Unterwurzacher
7eae35e2d3 v2api: implement ctlsocksrv.Interface 2020-07-17 22:14:40 +02:00
Jakob Unterwurzacher
57d572dbc1 v2api: implement Getxattr, Setxattr, Removexattr, Listxattr
gocryptfs/tests/xattr passes.
2020-07-14 19:55:20 +02:00
Jakob Unterwurzacher
4a0966e79e v2api: Setattr: fill out structure 2020-07-12 21:17:52 +02:00
Jakob Unterwurzacher
ca5acae6f0 v2api: fix Mkfifo 2020-07-12 20:19:29 +02:00
Jakob Unterwurzacher
0fa824933c v2api: properly implement Node.Setattr
We used to always open a *File2 and letting the *File2
handle Setattr. This does not work it we cannot open the file!

Before:

    $ go test
    2020/07/12 20:14:57 writer: Write/Writev failed, err: 2=no such file or directory. opcode: INTERRUPT
    2020/07/12 20:14:57 writer: Write/Writev failed, err: 2=no such file or directory. opcode: INTERRUPT
    --- FAIL: TestLchown (0.00s)
        matrix_test.go:634: lchown /tmp/gocryptfs-test-parent-1026/014500839/default-plain/symlink: too many levels of symbolic links
    touch: setting times of '/tmp/gocryptfs-test-parent-1026/014500839/default-plain/utimesnano_symlink': Too many levels of symbolic links
    --- FAIL: TestUtimesNanoSymlink (0.00s)
        matrix_test.go:655: exit status 1
    --- FAIL: TestMkfifo (0.00s)
        matrix_test.go:755: file exists
    --- FAIL: TestMagicNames (0.00s)
        matrix_test.go:773: Testing n="gocryptfs.longname.QhUr5d9FHerwEs--muUs6_80cy6JRp89c1otLwp92Cs"
        matrix_test.go:773: Testing n="gocryptfs.diriv"
        matrix_test.go:815: open /tmp/gocryptfs-test-parent-1026/014500839/default-plain/linktarget: permission denied
    --- FAIL: TestChmod (0.00s)
        matrix_test.go:840: chmod 444 -> 000 failed: permission denied
        matrix_test.go:840: chmod 444 -> 111 failed: permission denied
        matrix_test.go:840: chmod 444 -> 123 failed: permission denied
        matrix_test.go:840: chmod 444 -> 321 failed: permission denied
    FAIL
    exit status 1
    FAIL	github.com/rfjakob/gocryptfs/tests/matrix	0.790s

After:

    $ go test
    --- FAIL: TestMkfifo (0.00s)
        matrix_test.go:755: file exists
    --- FAIL: TestMagicNames (0.00s)
        matrix_test.go:773: Testing n="gocryptfs.longname.QhUr5d9FHerwEs--muUs6_80cy6JRp89c1otLwp92Cs"
        matrix_test.go:773: Testing n="gocryptfs.diriv"
        matrix_test.go:815: open /tmp/gocryptfs-test-parent-1026/501766059/default-plain/linktarget: permission denied
    --- FAIL: TestChmod (0.00s)
        matrix_test.go:849: modeHave 0644 != modeWant 0
    FAIL
    exit status 1
    FAIL    github.com/rfjakob/gocryptfs/tests/matrix   0.787s
2020-07-12 20:17:15 +02:00