hardcoresushi
/
libgocryptfs
1
0
Fork 0
Code Issues Pull Requests Releases Activity
1,783 Commits 2 Branches 0 Tags 6 MiB
Commit Graph

740 Commits

Author SHA1 Message Date
Matéo Duparc 847d4fa781
Genesis patch 2021-06-11 15:52:46 +02:00
Jakob Unterwurzacher 17f859d3c4 fusefronted: report plaintext size on symlink creation 
gocryptfs 2.0 introduced the regression that the size
reported at symlink creation was the ciphertext size,
which is wrong.

Report the plaintext size.

Fixes https://github.com/rfjakob/gocryptfs/issues/574
 2021-06-06 19:22:16 +02:00
Jakob Unterwurzacher e48f2377ec syscallcompat: drop obsolete wrappers 
These are now available cross-platform in the unix
package.
 2021-06-05 15:06:30 +02:00
Jakob Unterwurzacher 015cd066e1 fido2: quote argument strings in debug output 
Tested using

  gocryptfs -init -debug -fido2 "hello world" cipherdir

Output before:

  callFidoCommand: executing "/usr/bin/fido2-cred" with args [fido2-cred -M -h -v hello world]

After:

  callFidoCommand: executing "/usr/bin/fido2-cred" with args ["fido2-cred" "-M" "-h" "-v" "hello world"]

Related: https://github.com/rfjakob/gocryptfs/issues/571
 2021-06-03 22:03:21 +02:00
Jakob Unterwurzacher a38e5988ba fusefrontend: run acl Setxattr in user context 
The result of setting an acl depends on who runs the
operation!

Fixes fuse-xfstests generic/375
(see https://github.com/rfjakob/fuse-xfstests/wiki/results_2021-05-19)
 2021-06-02 19:10:36 +02:00
Jakob Unterwurzacher b23e21f61f fusefrontend: catch ReadAt integer overflow 
Discovered by xfstests generic/564 .

Failure was:

generic/564	- output mismatch (see /opt/fuse-xfstests/results//generic/564.out.bad)
    --- tests/generic/564.out	2021-05-08 21:11:05.307395966 +0200
    +++ /opt/fuse-xfstests/results//generic/564.out.bad	2021-05-19 19:01:16.912888879 +0200
    @@ -31,7 +31,7 @@
     source range beyond 8TiB returns 0

     destination range beyond 8TiB returns EFBIG
    -copy_range: File too large
    +copy_range: Function not implemented
 2021-06-02 18:20:05 +02:00
Jakob Unterwurzacher 04858ddd22 nametransform: check name validity on encryption 
xfstests generic/523 discovered that we allowed to set
xattrs with "/" in the name, but did not allow to read
them later.

With this change we do not allow to set them in the first
place.
 2021-06-02 14:29:48 +02:00
Jakob Unterwurzacher 18befda0e6 fusefrontend: list "." and ".." in dir entries 
Fixes xfstests generic/401
 2021-05-29 16:44:38 +02:00
Jakob Unterwurzacher 738a9e006a fusefrontend: rewrite Lseek SEEK_DATA / SEEK_HOLE 
In response to the discussion of the xfstests mailing list [1],
I looked at the Lseek implementation, which was naive and
did not handle all cases correctly.

The new implementation aligns the returned values to 4096 bytes
as most callers expect.

A lot of tests are added to verify that we handle all
cases correctly now.

[1]: https://www.spinics.net/lists/fstests/msg16554.html
 2021-05-29 16:05:36 +02:00
Jakob Unterwurzacher 07164cbb3a contentenc: add PlainOffToCipherOff helper 
Will be used for improving Lseek()
 2021-05-26 18:28:59 +02:00
Jakob Unterwurzacher b4794bedec contentenc: fix CipherSizeToPlainSize non-monoticity 
For an illegal cipherSize, pretend we have an additional
1-byte block.

See code comment for details.
 2021-05-26 13:17:56 +02:00
Jakob Unterwurzacher bebd7ed81f contentenc: update comments 
Also, replace one open-coded calculation with a
helper function.
 2021-05-26 13:17:56 +02:00
Jakob Unterwurzacher 2a5ac3e9ba tests: contentenc: add TestSizeToSize 
TestSizeToSize tests CipherSizeToPlainSize and PlainSizeToCipherSize.

Fails at the moment due to CipherSizeToPlainSize non-moniticity.
 2021-05-26 13:17:56 +02:00
Jakob Unterwurzacher 1b3c3b1347 syscallcompat: add GetdentsSpecial() 
GetdentsSpecial calls then Getdents syscall,
with normal entries and "." / ".." split into two slices.
 2021-05-26 13:17:56 +02:00
Jakob Unterwurzacher 1d2ac1e589 stupidgcm: prefer Go stdlib over OpenSSL on Apple M1 
https://github.com/rfjakob/gocryptfs/issues/556
 2021-05-26 09:20:22 +02:00
Jakob Unterwurzacher 09870bfac5 syscallcompat: also refactor MkdiratUser on GOOS=darwin 
Breakage was:

+GOOS=darwin
+GOARCH=amd64
+go build -tags without_openssl
internal/fusefrontend/node_dir_ops.go:45:34: cannot use context (type *fuse.Context) as type *fuse.Caller in argument to syscallcompat.MkdiratUser
internal/fusefrontend/node_dir_ops.go:83:35: cannot use context (type *fuse.Context) as type *fuse.Caller in argument to syscallcompat.MkdiratUser
 2021-05-22 22:01:46 +02:00
Jakob Unterwurzacher e1853e1011 syscallcompat: refactor MkdiratUser to take fuse.Context 
Let's have MkdiratUser take fuse.Context like everybody
else.
 2021-05-22 21:44:19 +02:00
Jakob Unterwurzacher cb4f9f9e29 syscallcompat: deduplicate OpenatUser/MknodatUser/SymlinkatUser/MkdiratUser 
Turns out the whole euid switching logic can be shared when
wrapping the syscall in a closure.
 2021-05-22 21:39:29 +02:00
Jakob Unterwurzacher 0650a512bb fsck: redirect go-fuse noise to syslog 2021-05-18 18:38:23 +02:00
Jakob Unterwurzacher 5da20da977 -speed: note that XChaCha20 is not selectable 
This is unclear to users, as seen on
https://github.com/rfjakob/gocryptfs/issues/452#issuecomment-828836651
 2021-05-18 09:53:23 +02:00
Jakob Unterwurzacher a91ad29d36 fusefrontend: fix RENAME_NOREPLACE darwin build failure 
Error was:

internal/fusefrontend/node.go:371:2: duplicate case syscallcompat.RENAME_NOREPLACE (value 0) in switch
	previous case at internal/fusefrontend/node.go:368:7

Rewrite using "if"s instead.
 2021-05-15 17:20:43 +02:00
Jakob Unterwurzacher 1ba2e42234 fusefrontend: avoid duplicate const definition 
RENAME_NOREPLACE is already defined in syscallcompat.
 2021-05-08 20:42:42 +02:00
Jakob Unterwurzacher a267562d29 fusefrontend: reject broken RENAME_EXCHANGE and RENAME_WHITEOUT 
Discovered by xfstests generic/013: or implementation for
RENAME_EXCHANGE and RENAME_WHITEOUT is incomplete. Reject the
flags so that the caller retries with regular rename.
 2021-05-08 15:39:49 +02:00
Jakob Unterwurzacher 4a07d6598c fusefrontend: make dirCache work for "node itself" 
"node itself" can be converted to node + child by
ascending one level.

Performance gains are spectacular, as will be seen
in the next commit.
 2021-04-07 07:18:35 +02:00
Jakob Unterwurzacher f73aee72f8 fusefrontend: print dirCache stats after unmount 2021-04-05 18:20:17 +02:00
Jakob Unterwurzacher 043f81dd01 fs: more idiomatic dirCache struct naming 2021-04-04 13:05:47 +02:00
Jakob Unterwurzacher 24d5d39300 fs: add initial dirfd caching 
dirfd caching was temporarily removed when moving
to the v2api. Add it back to gain back some lost speed.
 2021-04-03 13:08:28 +02:00
Jakob Unterwurzacher 6aae2aad97 tests: fusefronted: fix TestOpenBackingDir 
This test only worked accidentially, and would
break once dirfd caching is added.

fs.Mkdir(..., "dir1/dir2", ...) is illegal
(child name cannot contain slashes).
 2021-04-03 12:34:18 +02:00
Jakob Unterwurzacher 6b28c0be2c fusefronted: replace last rn.openBackingDir() calls 
Use the n.prepareAtSyscall() wrapper instead. Prepares
for adding caching into n.prepareAtSyscall().
 2021-04-03 11:19:04 +02:00
Jakob Unterwurzacher 3b9a1b628b fusefronted: move Create() and Open() to new file 
And deduplicate the code a little.
 2021-03-21 09:31:05 +01:00
Jakob Unterwurzacher 47a4d33f24 fusefrontend: -sharedstorage: fix TestRmwRace failure 
The Open() and Create() paths used different inode numbers,
which broke openfiletable locking against concurred readers.
 2021-03-21 09:02:50 +01:00
Jakob Unterwurzacher f6036c429a syscallcompat: getdents: link to #483 
Give a user receiving the Getdents warning some background info.
 2021-03-14 14:43:11 +01:00
Jakob Unterwurzacher e2dc52a965 v2api: -sharestorage: disable hard link tracking & add tests 
Hard link tracking was not correctly disabled
since the migration to the go-fuse v2 api.

Add a test to ensure it stays off.

Fixes https://github.com/rfjakob/gocryptfs/issues/525
 2021-03-07 17:22:29 +01:00
Jakob Unterwurzacher eaca820e87 fusefrontend: do not encrypt ACLs 
Pass through system.posix_acl_access and system.posix_acl_default
unencrypted to fix "cp -a" problems.

"cp -a" uses "setxattr" even to set normal permissions, see
https://www.spinics.net/lists/linux-nfs/msg63986.html .

Fixes https://github.com/rfjakob/gocryptfs/issues/543
 2021-02-07 20:01:16 +01:00
Jakob Unterwurzacher 80a651a194 syscallcompat: MknodatUser: work around changed syscall.Setgroups semantics 
Since go1.16beta1 (commit d1b1145cace8b968307f9311ff611e4bb810710c ,
https://go-review.googlesource.com/c/go/+/210639 )
syscall.{Setgroups,Setregid,Setreuid} affects all threads, which
is exactly what we not want.

We now use unix.{Setgroups,Setregid,Setreuid} instead.

Workarounds https://github.com/golang/go/issues/1435 .
 2021-02-06 11:38:25 +01:00
Jakob Unterwurzacher bed60101f4 nametransform: make `gocryptfs.diriv` and `gocryptfs.xxx.name` files world-readable 
Make `gocryptfs.diriv` and `gocryptfs.xxx.name` files world-readable to make encrypted backups easier
when mounting via fstab.

Having the files follow chmod/chown of their parent does not seem
to be worth the hassle. The content of the diriv files is not
secret, and both diriv and name files are protected by the
perms of the parent dir.

Fixes https://github.com/rfjakob/gocryptfs/issues/539
 2021-01-10 08:07:10 +01:00
Jakob Unterwurzacher f3394ae286 nametransform: move permission constants to perms.go 
Prep for https://github.com/rfjakob/gocryptfs/issues/539
 2021-01-10 07:27:04 +01:00
gmd20 c20c7992a0 main: add "-kernel_cache" flag 
This option is similar to fuse(8) kernel_cache

Verified using vmtouch.

Without -kernel_cache:

$ dd if=/dev/zero of=foo bs=1M count=10 ; vmtouch -t foo ; vmtouch foo
10+0 records in
10+0 records out
10485760 bytes (10 MB, 10 MiB) copied, 0,0242321 s, 433 MB/s
           Files: 1
     Directories: 0
   Touched Pages: 2560 (10M)
         Elapsed: 0.011159 seconds
           Files: 1
     Directories: 0
  Resident Pages: 0/2560  0/10M  0%
         Elapsed: 0.000993 seconds

With -kernel_cache:

$ dd if=/dev/zero of=foo bs=1M count=10 ; vmtouch -t foo ; vmtouch foo
10+0 records in
10+0 records out
10485760 bytes (10 MB, 10 MiB) copied, 0,0244015 s, 430 MB/s
           Files: 1
     Directories: 0
   Touched Pages: 2560 (10M)
         Elapsed: 0.011564 seconds
           Files: 1
     Directories: 0
  Resident Pages: 2560/2560  10M/10M  100%
         Elapsed: 0.000369 seconds
 2020-12-20 09:55:04 +01:00
Jakob Unterwurzacher 1c1692c4d9 fusefrontend_reverse: fix GETATTR panic 
We don't implement Getattr(), so don't try to call it.

Reported at https://github.com/rfjakob/gocryptfs/issues/519#issuecomment-718790790 :

15:22:53.414101 rx 3344: READ n2565 {Fh 7 [42143744 +131072)  L 0 RDONLY,0x8000}
15:22:53.414274 rx 3342: READ n2565 {Fh 7 [42012672 +131072)  L 0 RDONLY,0x8000}
15:22:53.414787 rx 3346: READ n2565 {Fh 7 [42274816 +131072)  L 0 RDONLY,0x8000}
15:22:53.414806 tx 3336:     OK,  131072b data "\xcb\xd3<\"!-\xa7\xc4"...
15:22:53.414874 rx 3348: GETATTR n1446 {Fh 0}
panic: interface conversion: *fusefrontend_reverse.File is not fs.FileGetattrer: missing method Getattr

goroutine 451 [running]:
github.com/rfjakob/gocryptfs/internal/fusefrontend_reverse.(*Node).Getattr(0xc00034c880, 0x5620579784c0, 0xc000593e60, 0x562057939800, 0xc000218050, 0xc0000fc108, 0x0)
        github.com/rfjakob/gocryptfs/internal/fusefrontend_reverse/node.go:69 +0x273
github.com/hanwen/go-fuse/v2/fs.(*rawBridge).getattr(0xc00011e000, 0x5620579784c0, 0xc000593e60, 0xc00034c880, 0x562057939800, 0xc000218050, 0xc0000fc108, 0xbfded1ef58ba7b13)
        github.com/hanwen/go-fuse/v2@v2.0.4-0.20200908172753-0b6cbc515082/fs/bridge.go:500 +0x2d4
github.com/hanwen/go-fuse/v2/fs.(*rawBridge).GetAttr(0xc00011e000, 0xc0000e0000, 0xc0000fc198, 0xc0000fc108, 0x0)
        github.com/hanwen/go-fuse/v2@v2.0.4-0.20200908172753-0b6cbc515082/fs/bridge.go:488 +0x11c
github.com/hanwen/go-fuse/v2/fuse.doGetAttr(0xc000120000, 0xc0000fc000)
        github.com/hanwen/go-fuse/v2@v2.0.4-0.20200908172753-0b6cbc515082/fuse/opcode.go:287 +0x6f
github.com/hanwen/go-fuse/v2/fuse.(*Server).handleRequest(0xc000120000, 0xc0000fc000, 0xc000000000)
        github.com/hanwen/go-fuse/v2@v2.0.4-0.20200908172753-0b6cbc515082/fuse/server.go:472 +0x2c1
github.com/hanwen/go-fuse/v2/fuse.(*Server).loop(0xc000120000, 0xc000288001)
        github.com/hanwen/go-fuse/v2@v2.0.4-0.20200908172753-0b6cbc515082/fuse/server.go:445 +0x18c
created by github.com/hanwen/go-fuse/v2/fuse.(*Server).readRequest
        github.com/hanwen/go-fuse/v2@v2.0.4-0.20200908172753-0b6cbc515082/fuse/server.go:312 +0x419
 2020-11-10 19:37:49 +01:00
Jakob Unterwurzacher 832e58cad4 Drop two more generated files 
These were committed by mistake.
 2020-10-19 19:27:47 +02:00
Jakob Unterwurzacher 165bf6c849 Drop generated files 
These were committed by mistake.
 2020-10-19 19:25:47 +02:00
Jakob Unterwurzacher 6697ffd6e2 fusefronted: reject GETXATTR "security.capability" 
Unless we are mounted with -suid, we can reject
these requests, and gain back some lost speed.

Closes https://github.com/rfjakob/gocryptfs/issues/515
 2020-10-18 21:07:30 +02:00
Jakob Unterwurzacher c943ed32aa syscallcompat: add getxattr fastpaths 
The allocations from Lgetxattr were #1 in the tar extract
allocation profile (caused by security.capability lookups).
No more!

$ benchstat old.txt new.txt
name         old time/op  new time/op  delta
Lgetxattr-4  15.2µs ± 0%   1.8µs ± 0%   ~     (p=1.000 n=1+1)

$ ./benchmark.bash
Testing gocryptfs at /tmp/benchmark.bash.H8p: gocryptfs v2.0-beta1-4-g95ea738-dirty; go-fuse v2.0.4-0.20200908172753-0b6cbc515082 => github.com/rfjakob/go-fuse/v2 v2.0.4-0.20201015204057-88b12c99f8af; 2020-10-18 go1.15.3 linux/amd64
/tmp/benchmark.bash.H8p.mnt is a mountpoint
WRITE: 262144000 bytes (262 MB, 250 MiB) copied, 0,520109 s, 504 MB/s
READ:  262144000 bytes (262 MB, 250 MiB) copied, 0,255672 s, 1,0 GB/s
UNTAR: 30,238
MD5:   12,721
LS:    10,038
RM:    16,536
 2020-10-18 00:25:42 +02:00
Jakob Unterwurzacher 83a324a46b syscallcompat: add Lgetxattr benchmark 2020-10-16 20:04:22 +02:00
Jakob Unterwurzacher fcb28e4ff3 v2pai: delete fusefrontend_reverse_v1api 
Served its mission a copy-paste source but can now be deleted.
 2020-10-15 23:18:21 +02:00
Jakob Unterwurzacher ec3eaf0b87 syscallcompat: don't retry Close() 
After Close() returns, the fd is dead, even if we
received EINTR. Don't retry, we could shoot down
an unrelated fd that received the same fd number.
 2020-10-14 13:40:12 +02:00
Jakob Unterwurzacher af4c1fb7a3 syscallcompat: retry ops on EINTR 
Retry operations that have been shown to throw EINTR
errors on CIFS.

Todo: Solution for this pain in the back:

	warning: unix.Getdents returned errno 2 in the middle of data
	rm: cannot remove 'linux-3.0.old3/Documentation/ABI/removed': Input/output error

Progress towards fixing https://github.com/rfjakob/gocryptfs/issues/483 .
 2020-10-14 00:35:16 +02:00
Jakob Unterwurzacher 803fdf410b syscallcompat: Openat: retry on EINTR 
Towards fixing https://github.com/rfjakob/gocryptfs/issues/507
 2020-10-11 01:31:09 +02:00
Pavol Rusnak 1e624a4cc3 Add support for FIDO2 tokens 2020-09-12 18:06:54 +02:00
Jakob Unterwurzacher ac687d5359 v2api: add Darwin xattr support 2020-09-09 11:17:19 +02:00