hardcoresushi/libgocryptfs
1
0
Fork 0
Code Issues Pull Requests Releases Activity
1,715 Commits 2 Branches 0 Tags 6 MiB
3b9a1b628b
Commit Graph

1715 Commits

This Branch
This Branch
All Branches
Author SHA1 Message Date
Jakob Unterwurzacher
3b9a1b628b fusefronted: move Create() and Open() to new file 
And deduplicate the code a little.
 2021-03-21 09:31:05 +01:00
Jakob Unterwurzacher
47a4d33f24 fusefrontend: -sharedstorage: fix TestRmwRace failure 
The Open() and Create() paths used different inode numbers,
which broke openfiletable locking against concurred readers.
 2021-03-21 09:02:50 +01:00
Jakob Unterwurzacher
692a79461a tests: matrix: add -sharestorage to test matrix 
Currently fails like this:

=== RUN   TestRmwRace
doRead 0: corrupt block #0: cipher: message authentication failed
-wpanic turns this warning into a panic: doRead 0: corrupt block #0: cipher: message authentication failed
panic: -wpanic turns this warning into a panic: doRead 0: corrupt block #0: cipher: message authentication failed

goroutine 1293 [running]:
log.(*Logger).Panic(0xc00011c230, 0xc0003b17c8, 0x1, 0x1)
	log/log.go:224 +0xac
github.com/rfjakob/gocryptfs/internal/tlog.(*toggledLogger).Printf(0xc00007a780, 0x55a821a766a1, 0x20, 0xc0003b19f0, 0x3, 0x3)
	github.com/rfjakob/gocryptfs/internal/tlog/log.go:78 +0x1ef
github.com/rfjakob/gocryptfs/internal/fusefrontend.(*File).doRead(0xc0001ff420, 0x0, 0x0, 0x0, 0x0, 0x1000, 0x0, 0x1, 0xc000880000, 0x1020)
	github.com/rfjakob/gocryptfs/internal/fusefrontend/file.go:201 +0x8c9
github.com/rfjakob/gocryptfs/internal/fusefrontend.(*File).doWrite(0xc0001ff420, 0xc000248428, 0x10, 0x30, 0xff0, 0x3, 0x18)
	github.com/rfjakob/gocryptfs/internal/fusefrontend/file.go:291 +0xc9e
github.com/rfjakob/gocryptfs/internal/fusefrontend.(*File).Write(0xc0001ff420, 0x55a821b306a0, 0xc000fbde90, 0xc000248428, 0x10, 0x30, 0xff0, 0x7f4a00000000, 0x0)
	github.com/rfjakob/gocryptfs/internal/fusefrontend/file.go:378 +0x25e
github.com/hanwen/go-fuse/v2/fs.(*rawBridge).Write(0xc000168140, 0xc000096000, 0xc0002483d8, 0xc000248428, 0x10, 0x30, 0x55a821ad40e0)
	github.com/hanwen/go-fuse/v2@v2.0.4-0.20210125162859-8e0bbdb16cb7/fs/bridge.go:819 +0x26d
github.com/hanwen/go-fuse/v2/fuse.doWrite(0xc000170160, 0xc000248240)
	github.com/hanwen/go-fuse/v2@v2.0.4-0.20210125162859-8e0bbdb16cb7/fuse/opcode.go:191 +0x6f
github.com/hanwen/go-fuse/v2/fuse.(*Server).handleRequest(0xc000170160, 0xc000248240, 0xc000000000)
	github.com/hanwen/go-fuse/v2@v2.0.4-0.20210125162859-8e0bbdb16cb7/fuse/server.go:472 +0x2be
github.com/hanwen/go-fuse/v2/fuse.(*Server).loop(0xc000170160, 0xc000cd4101)
	github.com/hanwen/go-fuse/v2@v2.0.4-0.20210125162859-8e0bbdb16cb7/fuse/server.go:445 +0x198
created by github.com/hanwen/go-fuse/v2/fuse.(*Server).readRequest
	github.com/hanwen/go-fuse/v2@v2.0.4-0.20210125162859-8e0bbdb16cb7/fuse/server.go:312 +0x41d
    matrix_test.go:354: Write failed
 2021-03-20 18:27:34 +01:00
Jakob Unterwurzacher
255a71c917 tests: MountOrFatal creates mnt dir itself 
Allows to drop a few Mkdir()s.
 2021-03-20 10:33:34 +01:00
Jakob Unterwurzacher
f6036c429a syscallcompat: getdents: link to #483 
Give a user receiving the Getdents warning some background info.
 2021-03-14 14:43:11 +01:00
Jakob Unterwurzacher
952d45ce84 tests: add TestDiskFull 
Also fix incomplete uid restoration in TestSupplementaryGroups
and replace syscall.Setregid and friends with unix.Setregid and
friends.

This test is added to check if have problems handling a full disk.
The ticket https://github.com/rfjakob/gocryptfs/issues/550 states
that the full disk was not where the backing gocryptfs filesystem
was, but this has no effect on gocryptfs, so we test the harder
case.
 2021-03-12 19:25:25 +01:00
Jakob Unterwurzacher
d47bf9976f Makefile: root_test: don't run test when compile fails 2021-03-12 17:44:13 +01:00
Jakob Unterwurzacher
e2dc52a965 v2api: -sharestorage: disable hard link tracking & add tests 
Hard link tracking was not correctly disabled
since the migration to the go-fuse v2 api.

Add a test to ensure it stays off.

Fixes https://github.com/rfjakob/gocryptfs/issues/525
 2021-03-07 17:22:29 +01:00
Jakob Unterwurzacher
eaca820e87 fusefrontend: do not encrypt ACLs 
Pass through system.posix_acl_access and system.posix_acl_default
unencrypted to fix "cp -a" problems.

"cp -a" uses "setxattr" even to set normal permissions, see
https://www.spinics.net/lists/linux-nfs/msg63986.html .

Fixes https://github.com/rfjakob/gocryptfs/issues/543
 2021-02-07 20:01:16 +01:00
Jakob Unterwurzacher
bb2484f152 tests/xattr: fix acl blob 
The blob was truncated to 31 bytes.
 2021-02-07 20:00:33 +01:00
Jakob Unterwurzacher
4b4a68eb21 README: warn about incomplete ACL support 
https://github.com/rfjakob/gocryptfs/issues/543
https://github.com/rfjakob/gocryptfs/issues/542
https://github.com/rfjakob/gocryptfs/issues/453
 2021-02-07 10:43:40 +01:00
Jakob Unterwurzacher
5d498f3a24 README: mention MacFUSE v4.x support 
https://github.com/rfjakob/gocryptfs/issues/544
 2021-02-06 16:47:27 +01:00
Jakob Unterwurzacher
80a651a194 syscallcompat: MknodatUser: work around changed syscall.Setgroups semantics 
Since go1.16beta1 (commit d1b1145cace8b968307f9311ff611e4bb810710c ,
https://go-review.googlesource.com/c/go/+/210639 )
syscall.{Setgroups,Setregid,Setreuid} affects all threads, which
is exactly what we not want.

We now use unix.{Setgroups,Setregid,Setreuid} instead.

Workarounds https://github.com/golang/go/issues/1435 .
 2021-02-06 11:38:25 +01:00
Jakob Unterwurzacher
3ab1bcf1c5 README: note Trezor removal 
Adding support for Trezor was listed, but removal was not,
causing confusion.

Closes https://github.com/rfjakob/gocryptfs/pull/534
 2021-01-31 09:46:43 +01:00
Jakob Unterwurzacher
ac2e5baa13 go.mod: update go-fuse to latest master 
We need

	09a3c38171
	fuse: support the new macFUSE mount protocol

so mounting on MacFuse 4.x works.

Updated using

	go get github.com/hanwen/go-fuse/v2@master

.

Fixes https://github.com/rfjakob/gocryptfs/issues/524
 2021-01-31 08:36:57 +01:00
Jakob Unterwurzacher
bed60101f4 nametransform: make gocryptfs.diriv and gocryptfs.xxx.name files world-readable 
Make `gocryptfs.diriv` and `gocryptfs.xxx.name` files world-readable to make encrypted backups easier
when mounting via fstab.

Having the files follow chmod/chown of their parent does not seem
to be worth the hassle. The content of the diriv files is not
secret, and both diriv and name files are protected by the
perms of the parent dir.

Fixes https://github.com/rfjakob/gocryptfs/issues/539
 2021-01-10 08:07:10 +01:00
Jakob Unterwurzacher
6b492fdcb8 MANPAGE: add sub-headers to EXAMPLES 
Makes linking to them easier.
 2021-01-10 08:01:08 +01:00
Jakob Unterwurzacher
f3394ae286 nametransform: move permission constants to perms.go 
Prep for https://github.com/rfjakob/gocryptfs/issues/539
 2021-01-10 07:27:04 +01:00
Jakob Unterwurzacher
de108d3fc0 -idle: don't lazy-unmount 
When a process has its working dir inside the mount,
the only way we notice is that we get EBUSY when trying
to unmount.

We used to lazy-unmount in this case, but this means
pulling the rug from under the process.

For example, bash will start throwing

  cd: error retrieving current directory: getcwd: cannot access parent directories: No such file or directory

messages.

Fixes https://github.com/rfjakob/gocryptfs/issues/533
 2021-01-02 18:19:41 +01:00
gmd20
c20c7992a0 main: add "-kernel_cache" flag 
This option is similar to fuse(8) kernel_cache

Verified using vmtouch.

Without -kernel_cache:

$ dd if=/dev/zero of=foo bs=1M count=10 ; vmtouch -t foo ; vmtouch foo
10+0 records in
10+0 records out
10485760 bytes (10 MB, 10 MiB) copied, 0,0242321 s, 433 MB/s
           Files: 1
     Directories: 0
   Touched Pages: 2560 (10M)
         Elapsed: 0.011159 seconds
           Files: 1
     Directories: 0
  Resident Pages: 0/2560  0/10M  0%
         Elapsed: 0.000993 seconds

With -kernel_cache:

$ dd if=/dev/zero of=foo bs=1M count=10 ; vmtouch -t foo ; vmtouch foo
10+0 records in
10+0 records out
10485760 bytes (10 MB, 10 MiB) copied, 0,0244015 s, 430 MB/s
           Files: 1
     Directories: 0
   Touched Pages: 2560 (10M)
         Elapsed: 0.011564 seconds
           Files: 1
     Directories: 0
  Resident Pages: 2560/2560  10M/10M  100%
         Elapsed: 0.000369 seconds
 2020-12-20 09:55:04 +01:00
Jakob Unterwurzacher
14dac373c2 MANPAGE: add fstab example 
https://github.com/rfjakob/gocryptfs/issues/497
 2020-11-14 15:27:40 +01:00
Jakob Unterwurzacher
8470cc38df MANPAGE: add -info example output 2020-11-14 15:08:57 +01:00
Jakob Unterwurzacher
6bb42f79fd MANPAGE: split up OPTIONS into action flags, init, mount, common 
Fixes https://github.com/rfjakob/gocryptfs/issues/517
 2020-11-14 15:00:47 +01:00
Jakob Unterwurzacher
0b2562fdba MANPAGE: clarify -fg and syslog interaction 
The flag -fg does NOT imply -nosyslog. Syslog redirection is
active when -notifypid is passed.
 2020-11-14 14:29:04 +01:00
Jakob Unterwurzacher
5d872b7106 Update Changelog for v2.0-beta2 2020-11-14 13:21:26 +01:00
Jakob Unterwurzacher
1c1692c4d9 fusefrontend_reverse: fix GETATTR panic 
We don't implement Getattr(), so don't try to call it.

Reported at https://github.com/rfjakob/gocryptfs/issues/519#issuecomment-718790790 :

15:22:53.414101 rx 3344: READ n2565 {Fh 7 [42143744 +131072)  L 0 RDONLY,0x8000}
15:22:53.414274 rx 3342: READ n2565 {Fh 7 [42012672 +131072)  L 0 RDONLY,0x8000}
15:22:53.414787 rx 3346: READ n2565 {Fh 7 [42274816 +131072)  L 0 RDONLY,0x8000}
15:22:53.414806 tx 3336:     OK,  131072b data "\xcb\xd3<\"!-\xa7\xc4"...
15:22:53.414874 rx 3348: GETATTR n1446 {Fh 0}
panic: interface conversion: *fusefrontend_reverse.File is not fs.FileGetattrer: missing method Getattr

goroutine 451 [running]:
github.com/rfjakob/gocryptfs/internal/fusefrontend_reverse.(*Node).Getattr(0xc00034c880, 0x5620579784c0, 0xc000593e60, 0x562057939800, 0xc000218050, 0xc0000fc108, 0x0)
        github.com/rfjakob/gocryptfs/internal/fusefrontend_reverse/node.go:69 +0x273
github.com/hanwen/go-fuse/v2/fs.(*rawBridge).getattr(0xc00011e000, 0x5620579784c0, 0xc000593e60, 0xc00034c880, 0x562057939800, 0xc000218050, 0xc0000fc108, 0xbfded1ef58ba7b13)
        github.com/hanwen/go-fuse/v2@v2.0.4-0.20200908172753-0b6cbc515082/fs/bridge.go:500 +0x2d4
github.com/hanwen/go-fuse/v2/fs.(*rawBridge).GetAttr(0xc00011e000, 0xc0000e0000, 0xc0000fc198, 0xc0000fc108, 0x0)
        github.com/hanwen/go-fuse/v2@v2.0.4-0.20200908172753-0b6cbc515082/fs/bridge.go:488 +0x11c
github.com/hanwen/go-fuse/v2/fuse.doGetAttr(0xc000120000, 0xc0000fc000)
        github.com/hanwen/go-fuse/v2@v2.0.4-0.20200908172753-0b6cbc515082/fuse/opcode.go:287 +0x6f
github.com/hanwen/go-fuse/v2/fuse.(*Server).handleRequest(0xc000120000, 0xc0000fc000, 0xc000000000)
        github.com/hanwen/go-fuse/v2@v2.0.4-0.20200908172753-0b6cbc515082/fuse/server.go:472 +0x2c1
github.com/hanwen/go-fuse/v2/fuse.(*Server).loop(0xc000120000, 0xc000288001)
        github.com/hanwen/go-fuse/v2@v2.0.4-0.20200908172753-0b6cbc515082/fuse/server.go:445 +0x18c
created by github.com/hanwen/go-fuse/v2/fuse.(*Server).readRequest
        github.com/hanwen/go-fuse/v2@v2.0.4-0.20200908172753-0b6cbc515082/fuse/server.go:312 +0x419
 2020-11-10 19:37:49 +01:00
Jakob Unterwurzacher
4872a4a8aa performance.txt: add v2.0-beta1-9 results 2020-11-10 19:27:30 +01:00
Jakob Unterwurzacher
6bdc8c6ee8 go.mod: switch back to mainline go-fuse 
https://review.gerrithub.io/c/hanwen/go-fuse/+/503508
was merged as
1d0096e579
so we don't need our fork anymore.
 2020-11-10 19:24:48 +01:00
Jakob Unterwurzacher
029e44d405 README: add link to gocryptfs-inspect 
This is *very* hard to find.

https://github.com/rfjakob/gocryptfs/issues/152
 2020-10-22 21:47:03 +02:00
Jakob Unterwurzacher
832e58cad4 Drop two more generated files 
These were committed by mistake.
 2020-10-19 19:27:47 +02:00
Jakob Unterwurzacher
165bf6c849 Drop generated files 
These were committed by mistake.
 2020-10-19 19:25:47 +02:00
Jakob Unterwurzacher
6697ffd6e2 fusefronted: reject GETXATTR "security.capability" 
Unless we are mounted with -suid, we can reject
these requests, and gain back some lost speed.

Closes https://github.com/rfjakob/gocryptfs/issues/515
 2020-10-18 21:07:30 +02:00
Jakob Unterwurzacher
c943ed32aa syscallcompat: add getxattr fastpaths 
The allocations from Lgetxattr were #1 in the tar extract
allocation profile (caused by security.capability lookups).
No more!

$ benchstat old.txt new.txt
name         old time/op  new time/op  delta
Lgetxattr-4  15.2µs ± 0%   1.8µs ± 0%   ~     (p=1.000 n=1+1)

$ ./benchmark.bash
Testing gocryptfs at /tmp/benchmark.bash.H8p: gocryptfs v2.0-beta1-4-g95ea738-dirty; go-fuse v2.0.4-0.20200908172753-0b6cbc515082 => github.com/rfjakob/go-fuse/v2 v2.0.4-0.20201015204057-88b12c99f8af; 2020-10-18 go1.15.3 linux/amd64
/tmp/benchmark.bash.H8p.mnt is a mountpoint
WRITE: 262144000 bytes (262 MB, 250 MiB) copied, 0,520109 s, 504 MB/s
READ:  262144000 bytes (262 MB, 250 MiB) copied, 0,255672 s, 1,0 GB/s
UNTAR: 30,238
MD5:   12,721
LS:    10,038
RM:    16,536
 2020-10-18 00:25:42 +02:00
Jakob Unterwurzacher
95ea7383f9 fsck: make sure we unmount in all cases 2020-10-17 23:03:58 +02:00
Jakob Unterwurzacher
83a324a46b syscallcompat: add Lgetxattr benchmark 2020-10-16 20:04:22 +02:00
Jakob Unterwurzacher
d656574d08 performance.txt: add gocryptfs v1.8.0 results 2020-10-16 20:03:59 +02:00
Jakob Unterwurzacher
f99050b78a Update performance.txt with gocryptfs v2.0-beta1 result 
As expected, we are slow. Fd caching will be implemented later.
 2020-10-16 19:13:32 +02:00
Jakob Unterwurzacher
95775bf52c Update README for v2.0-beta1 2020-10-15 23:23:07 +02:00
Jakob Unterwurzacher
fcb28e4ff3 v2pai: delete fusefrontend_reverse_v1api 
Served its mission a copy-paste source but can now be deleted.
 2020-10-15 23:18:21 +02:00
Jakob Unterwurzacher
f03f56844b go.mod: temporarily switch to patched go-fuse fork 
Use our fork until https://review.gerrithub.io/c/hanwen/go-fuse/+/503508
gets merged.

Command used was:

	go mod edit -replace github.com/hanwen/go-fuse/v2=github.com/rfjakob/go-fuse/v2@knodes_upstreaming_v3
 2020-10-15 22:45:06 +02:00
Jakob Unterwurzacher
a2f48eb838 contrib: cleanup-tmp-mounts: also clean ext4-ramdisk 2020-10-15 17:49:17 +02:00
Jakob Unterwurzacher
871f305f90 main: show microseconds in go-fuse logs 
Suppress the date but show microseconds in fuse debug logs
(-fusedebug) and go-fuse warnings.
 2020-10-14 15:37:37 +02:00
Jakob Unterwurzacher
ec3eaf0b87 syscallcompat: don't retry Close() 
After Close() returns, the fd is dead, even if we
received EINTR. Don't retry, we could shoot down
an unrelated fd that received the same fd number.
 2020-10-14 13:40:12 +02:00
Jakob Unterwurzacher
af4c1fb7a3 syscallcompat: retry ops on EINTR 
Retry operations that have been shown to throw EINTR
errors on CIFS.

Todo: Solution for this pain in the back:

	warning: unix.Getdents returned errno 2 in the middle of data
	rm: cannot remove 'linux-3.0.old3/Documentation/ABI/removed': Input/output error

Progress towards fixing https://github.com/rfjakob/gocryptfs/issues/483 .
 2020-10-14 00:35:16 +02:00
Jakob Unterwurzacher
803fdf410b syscallcompat: Openat: retry on EINTR 
Towards fixing https://github.com/rfjakob/gocryptfs/issues/507
 2020-10-11 01:31:09 +02:00
Jakob Unterwurzacher
fe340477b2 build.bash: show "go mod edit -replace" in version string 
If you do something like this,

  go mod edit -replace github.com/hanwen/go-fuse/v2=/home/jakob/go/src/github.com/hanwen/go-fuse

the version string of the resulting binary should reflect that.

Before:

  gocryptfs v1.8.0-135-g352b547-dirty.gofuse_v2api; go-fuse v2.0.4-0.20200908172753-0b6cbc515082; 2020-10-03 go1.15.2 linux/amd64

After:

  gocryptfs v1.8.0-135-g352b547-dirty.gofuse_v2api; go-fuse v2.0.4-0.20200908172753-0b6cbc515082 => /home/jakob/go/src/github.com/hanwen/go-fuse; 2020-10-03 go1.15.2 linux/amd64
 2020-10-04 00:12:46 +02:00
Jakob Unterwurzacher
754c483870 tests: fsstress-gocryptfs.bash: log timestamp for each iteration 2020-10-04 00:12:46 +02:00
Jakob Unterwurzacher
66449bf56b tests: fsstress-gocryptfs.bash: add DEBUG option 
Also add a wrapper script, fsstress.collect-crashes.sh, to collect
the debug output.

https://github.com/hanwen/go-fuse/issues/372
 2020-10-04 00:12:25 +02:00
Jakob Unterwurzacher
3c5a80c27b fsstress-gocryptfs.bash: don't hang if TMPDIR ends in / 
We would hang like this

   ./fsstress-loopback.bash
  Recompile go-fuse loopback: v2.0.3-7-g0b6cbc5
  Waiting for mount: xxxxxxxx^C

if TMPDIR has a trailing /. The reason is that the
paths in /proc/self/mounts are normalized, while
TMPDIR may not be.
 2020-10-03 14:23:37 +02:00
Pavol Rusnak
068692ac31 go mod tidy 2020-09-21 14:15:21 +02:00