Adding flags allows to use inomap in reverse mode,
replacing the clunky inoBaseDirIV/inoBaseNameFile
logic that causes problems with high underlying
inode numbers ( https://github.com/rfjakob/gocryptfs/issues/457 )
Microbenchmarks (values below) show that the "SingleDev"
case is now much slower due to an extra map lookup,
but this has no visible effects in ./test.bash results,
so there was no time spent optimizing the case further.
$ go test -bench=.
goos: linux
goarch: amd64
pkg: github.com/rfjakob/gocryptfs/internal/inomap
BenchmarkTranslateSingleDev-4 18757510 61.5 ns/op
BenchmarkTranslateManyDevs-4 18061515 64.5 ns/op
PASS
ok github.com/rfjakob/gocryptfs/internal/inomap 2.467s
Error from Travis CI was:
+GOOS=darwin
+GOARCH=amd64
+go build -tags without_openssl
# github.com/rfjakob/gocryptfs/internal/fusefrontend
internal/fusefrontend/fs.go:88:45: cannot use st.Dev (type int32) as type uint64 in argument to openfiletable.NewInumMap
Add uint64 to fix it.
This adds support for gitignore-like wildcards and exclude patters in
reverse mode. It (somewhat) fixes#273: no regexp support, but the
syntax should be powerful enough to satisfy most needs.
Also, since adding a lot of --exclude options can be tedious, it adds
the --exclude-from option to read patterns from a file (or files).
For Linux, everything effectively stays the same. For both path-based and
fd-based Utimens() calls, we use unix.UtimesNanoAt(). To avoid introducing
a separate syscall wrapper for futimens() (as done in go-fuse, for example),
we instead use the /proc/self/fd - trick.
On macOS, this changes quite a lot:
* Path-based Utimens() calls were previously completely broken, since
unix.UtimensNanoAt() ignores the passed file descriptor. Note that this
cannot be fixed easily since there IS no appropriate syscall available on
macOS prior to High Sierra (10.13). We emulate this case by using
Fchdir() + setattrlist().
* Fd-based Utimens() calls were previously translated to f.GetAttr() (to
fill any empty parameters) and syscall.Futimes(), which does not does
support nanosecond precision. Both issues can be fixed by switching to
fsetattrlist().
Fixes https://github.com/rfjakob/gocryptfs/issues/350
The only call forwarded to loopbackFileSystem was Statfs,
which is trivial to implement.
Implement it and drop loopbackFileSystem, as having it carries the
risk that a coding error bypasses the usual encryption/decryption
chain.
Instead of manually adjusting the user after creating the symlink,
adjust effective permissions and let the kernel deal with it.
Related to https://github.com/rfjakob/gocryptfs/issues/338.
Instead of manually adjusting the user and mode after creating the
device file, adjust effective permissions and let the kernel deal
with it.
Related to https://github.com/rfjakob/gocryptfs/issues/338.
The current code has a risk of race-conditions, since we pass a path
containing "/" to Fchownat. We could fix this by opening a file descriptor,
however, this does not seem worth the effort. We also don't chown *.name files.
Reported by @slackner at https://github.com/rfjakob/gocryptfs/issues/327 :
Possible race-conditions between file creation and Fchownat
* Assume a system contains a gocryptfs mount as root user
with -allow_other
* As a regular user create a new file with mode containing
the SUID flag and write access for other users
* Before gocryptfs executes the Fchownat call, try to open
the file again, write some exploit code to it, and try to run it.
For a short time, the file is owned by root and has the SUID flag, so
this is pretty dangerous.
We alread have this warning in Open(), but xfstests generic/488
causes "too many open files" via Create. Add the same message so
the user sees what is going on.
This function is in all fastpaths, will get a cache, and needs
its own file.
renamed: internal/fusefrontend/names.go -> internal/fusefrontend/openbackingdir.go
renamed: internal/fusefrontend/names_test.go -> internal/fusefrontend/openbackingdir_test.go
Use openBackingDir() and Fstatat().
High performance impact, though part of it should be
mitigated by adding DirIV caching to the new code paths.
$ ./benchmark.bash
Testing gocryptfs at /tmp/benchmark.bash.Eou: gocryptfs v1.6-37-ge3914b3-dirty; go-fuse v20170619-66-g6df8ddc; 2018-10-14 go1.11
WRITE: 262144000 bytes (262 MB, 250 MiB) copied, 1.2289 s, 213 MB/s
READ: 262144000 bytes (262 MB, 250 MiB) copied, 1.02616 s, 255 MB/s
UNTAR: 24.490
MD5: 13.120
LS: 3.368
RM: 9.232
Even though filesystem notifications aren't implemented for FUSE, I decided to
try my hand at implementing the autounmount feature (#128). I based it on the
EncFS autounmount code, which records filesystem accesses and checks every X
seconds whether it's idled long enough to unmount.
I've tested the feature locally, but I haven't added any tests for this flag.
I also haven't worked with Go before. So please let me know if there's
anything that should be done differently.
One particular concern: I worked from the assumption that the open files table
is unique per-filesystem. If that's not true, I'll need to add an open file
count and associated lock to the Filesystem type instead.
https://github.com/rfjakob/gocryptfs/pull/265
Error was:
# github.com/rfjakob/gocryptfs/internal/fusefrontend
internal/fusefrontend/fs.go:179: cannot use perms | 256 (type uint16) as type uint32 in argument to syscall.Fchmod
internal/fusefrontend/fs.go:185: cannot use perms (type uint16) as type uint32 in argument to syscall.Fchmod
Rename openBackingPath to openBackingDir and use OpenDirNofollow
to be safe against symlink races. Note that openBackingDir is
not used in several important code paths like Create().
But it is used in Unlink, and the performance impact in the RM benchmark
to be acceptable:
Before
$ ./benchmark.bash
Testing gocryptfs at /tmp/benchmark.bash.bYO: gocryptfs v1.6-12-g930c37e-dirty; go-fuse v20170619-49-gb11e293; 2018-09-08 go1.10.3
WRITE: 262144000 bytes (262 MB, 250 MiB) copied, 1.07979 s, 243 MB/s
READ: 262144000 bytes (262 MB, 250 MiB) copied, 0.882413 s, 297 MB/s
UNTAR: 16.703
MD5: 7.606
LS: 1.349
RM: 3.237
After
$ ./benchmark.bash
Testing gocryptfs at /tmp/benchmark.bash.jK3: gocryptfs v1.6-13-g84d6faf-dirty; go-fuse v20170619-49-gb11e293; 2018-09-08 go1.10.3
WRITE: 262144000 bytes (262 MB, 250 MiB) copied, 1.06261 s, 247 MB/s
READ: 262144000 bytes (262 MB, 250 MiB) copied, 0.947228 s, 277 MB/s
UNTAR: 17.197
MD5: 7.540
LS: 1.364
RM: 3.410
O_DIRECT accesses must be aligned in both offset and length. Due to our
crypto header, alignment will be off, even if userspace makes aligned
accesses. Running xfstests generic/013 on ext4 used to trigger lots of
EINVAL errors due to missing alignment. Just fall back to buffered IO.
