Commit Graph

186 Commits

Author SHA1 Message Date
Jakob Unterwurzacher
4cdf6b9af9 fusefronted: log more details on WriteAt failures
Also log inode number, fd number, offset and length.

Maybe help debugging https://github.com/rfjakob/gocryptfs/issues/269 .
2018-10-17 22:18:07 +02:00
Jesse Dunietz
87d3ed9187 Add option for autounmount
Even though filesystem notifications aren't implemented for FUSE, I decided to
try my hand at implementing the autounmount feature (#128). I based it on the
EncFS autounmount code, which records filesystem accesses and checks every X
seconds whether it's idled long enough to unmount.

I've tested the feature locally, but I haven't added any tests for this flag.
I also haven't worked with Go before. So please let me know if there's
anything that should be done differently.

One particular concern: I worked from the assumption that the open files table
is unique per-filesystem. If that's not true, I'll need to add an open file
count and associated lock to the Filesystem type instead.

https://github.com/rfjakob/gocryptfs/pull/265
2018-10-11 20:16:45 +02:00
Jakob Unterwurzacher
e4f1a32a9a fusefrontend: Fix uint16 build failure on Darwin
Error was:

  # github.com/rfjakob/gocryptfs/internal/fusefrontend
  internal/fusefrontend/fs.go:179: cannot use perms | 256 (type uint16) as type uint32 in argument to syscall.Fchmod
  internal/fusefrontend/fs.go:185: cannot use perms (type uint16) as type uint32 in argument to syscall.Fchmod
2018-09-23 12:17:59 +02:00
Jakob Unterwurzacher
a1fb456618 fusefrontend: make Rename() symlink-safe
Use Openat() and the openBackingDir() helper so we
never follow symlinks.
2018-09-23 12:17:59 +02:00
Jakob Unterwurzacher
897bb8924f fusefrontend: make Create() symlink-safe
Use Openat() and the openBackingDir() helper so we
never follow symlinks.
2018-09-23 12:17:59 +02:00
Jakob Unterwurzacher
63762b33af fusefrontend: Open(): fix dirfd leak
Close was missing.
2018-09-23 12:17:59 +02:00
Jakob Unterwurzacher
bead82c9fb fusefrontend: add named parameters to openBackingDir
Named parameters make using the function easier.
2018-09-23 12:17:59 +02:00
Jakob Unterwurzacher
c270b21efc fusefrontend: get rid of os.File* wrapping
Directly use int file descriptors for the dirfd
and get rid of one level of indirection.
2018-09-23 12:17:26 +02:00
Jakob Unterwurzacher
22fba4ac3e fusefrontent: make Open() symlink-safe 2018-09-23 12:17:26 +02:00
Jakob Unterwurzacher
e8d8ae54d3 fusefrontend: use OpenDirNofollow in openBackingDir
Rename openBackingPath to openBackingDir and use OpenDirNofollow
to be safe against symlink races. Note that openBackingDir is
not used in several important code paths like Create().

But it is used in Unlink, and the performance impact in the RM benchmark
to be acceptable:

Before

	$ ./benchmark.bash
	Testing gocryptfs at /tmp/benchmark.bash.bYO: gocryptfs v1.6-12-g930c37e-dirty; go-fuse v20170619-49-gb11e293; 2018-09-08 go1.10.3
	WRITE: 262144000 bytes (262 MB, 250 MiB) copied, 1.07979 s, 243 MB/s
	READ:  262144000 bytes (262 MB, 250 MiB) copied, 0.882413 s, 297 MB/s
	UNTAR: 16.703
	MD5:   7.606
	LS:    1.349
	RM:    3.237

After

	$ ./benchmark.bash
	Testing gocryptfs at /tmp/benchmark.bash.jK3: gocryptfs v1.6-13-g84d6faf-dirty; go-fuse v20170619-49-gb11e293; 2018-09-08 go1.10.3
	WRITE: 262144000 bytes (262 MB, 250 MiB) copied, 1.06261 s, 247 MB/s
	READ:  262144000 bytes (262 MB, 250 MiB) copied, 0.947228 s, 277 MB/s
	UNTAR: 17.197
	MD5:   7.540
	LS:    1.364
	RM:    3.410
2018-09-08 19:27:33 +02:00
Jakob Unterwurzacher
dbd400d930 fusefrontend: truncateGrowFile: pass zeroPad error to caller
Errors from zeroPad were ignored until now, as discovered
using xfstests generic/083.
2018-08-15 17:25:22 +02:00
Jakob Unterwurzacher
ec2fdc19cf reverse mode: add --exclude option
https://github.com/rfjakob/gocryptfs/issues/235
2018-08-11 23:26:49 +02:00
Jakob Unterwurzacher
f316f1b2df fusefronted: disallow writes running concurrently with reads
As uncovered by xfstests generic/465, concurrent reads and writes
could lead to this,

  doRead 3015532: corrupt block #1039: stupidgcm: message authentication failed,

as the read could pick up a block that has not yet been completely written -
write() is not atomic!

Now writes take ContentLock exclusively, while reads take it shared,
meaning that multiple reads can run in parallel with each other, but
not with a write.

This also simplifies the file header locking.
2018-07-22 22:29:22 +02:00
Jakob Unterwurzacher
c70df522d2 fusefrontend: doWrite: delete file header if first write fails
xfstests generic/083 fills the filesystem almost completely while
running fsstress in parallel. In fsck, these would show up:

  readFileID 2580: incomplete file, got 18 instead of 19 bytes

This could happen when writing the file header works, but writing
the actual data fails.

Now we kill the header again by truncating the file to zero.
2018-07-15 15:12:55 +02:00
Jakob Unterwurzacher
55bb22bad6 fusefrontend: doWrite: no need to take HeaderLock.RLock()
Other writers are blocked by ContentLock already.
2018-07-15 12:40:23 +02:00
Jakob Unterwurzacher
95b93db35f fusefrontend: log prealloc failures at Info level
If the underlying filesystem is full, it is normal get ENOSPC here.
Log at Info level instead of Warning.

Fixes xfstests generic/015 and generic/027, which complained about
the extra output.
2018-07-14 15:18:27 +02:00
Jakob Unterwurzacher
53f7e1a0f0 macos: fix O_DIRECT build failure
O_DIRECT has no direct equivalent on MacOS
(check out https://github.com/libuv/libuv/issues/1600 for details).

Just define it to zero there.
2018-07-04 09:04:00 +02:00
Jakob Unterwurzacher
893e41149e fusefrontend: disallow O_DIRECT and fall back to buffered IO
O_DIRECT accesses must be aligned in both offset and length. Due to our
crypto header, alignment will be off, even if userspace makes aligned
accesses. Running xfstests generic/013 on ext4 used to trigger lots of
EINVAL errors due to missing alignment. Just fall back to buffered IO.
2018-07-02 23:54:37 +02:00
Jakob Unterwurzacher
c51fc9e07d fusefronted: downgrade fallocate message severity
The message causes output mismatches in xfstests generic/112.
Downgrade the severity to Info so it gets disabled when using "-q".
2018-07-02 23:03:43 +02:00
Jakob Unterwurzacher
01a078e7c0 Fix golint warnings 2018-07-01 22:00:06 +02:00
Jakob Unterwurzacher
e951043084 fusefrontend: add File.SeekData() function
This function will enable "gocryptfs -fsck" to handle
sparse files efficiently.
2018-07-01 20:56:22 +02:00
Jakob Unterwurzacher
a2af1fb5da fusefrontend: export "File" type
"gocryptfs -fsck" will need access to helper functions,
and to get that, it will need to cast a gofuse.File to a
fusefrontend.File. Make fusefrontend.File exported to make
this work.
2018-07-01 20:56:22 +02:00
Jakob Unterwurzacher
1a18d8e609 fsck: rename "CorruptItems" channel to "MitigatedCorruptions"
Make it clear that this channel is only used to report corruptions
that are transparently mitigated and do not return an error to
the user.
2018-07-01 20:56:22 +02:00
Jakob Unterwurzacher
1bab400fca Fix three golint warnings
We are clean again.

Warnings were:

internal/fusefrontend/fs.go:443:14: should omit type string from declaration
of var cTarget; it will be inferred from the right-hand side
internal/fusefrontend/xattr.go:26:1: comment on exported method FS.GetXAttr
should be of the form "GetXAttr ..."
internal/syscallcompat/sys_common.go:9:7: exported const PATH_MAX should have
comment or be unexported
2018-06-19 20:16:21 +02:00
Jakob Unterwurzacher
bfa50517e9 xattr: return EOPNOTSUPP instead of ENODATA in GetXattr
Reading system.posix_acl_access and system.posix_acl_default
should return EOPNOTSUPP to inform user-space that we do not
support ACLs.

xftestest essientially does

	chacl -l | grep "Operation not supported"

to determine if the filesystem supports ACLs, and used to
wrongly believe that gocryptfs does.
2018-06-12 23:05:53 +02:00
Jakob Unterwurzacher
ae02ca1ded xattr: use LGet/LSet etc
Support has been merged into the xattr package
( https://github.com/pkg/xattr/pull/29 ), use it.
2018-05-27 20:09:48 +02:00
Jakob Unterwurzacher
7b00681807 xattr: return EOPNOTSUPP for unsupported attributes
mv is unhappy when we return EPERM when it tries to set
system.posix_acl_access:

   mv: preserving permissions for ‘b/x’: Operation not permitted

Now we return EOPNOTSUPP like tmpfs does and mv seems happy.
2018-05-15 23:00:47 +02:00
Bolshevik
a41ec2028c xattr: optimize storage, store as binary instead of bae64
Values a binary-safe, there is no need to base64-encode them.

Old, base64-encoded values are supported transparently
on reading. Writing xattr values now always writes them binary.
2018-05-10 23:25:49 +02:00
Bolshevik
5ccc06d5cb xattr: added passing of a "flags" parameter
Pass the "flags" parameter to the lower layer syscall.
This makes Apple applications being able to successfully save data.
2018-05-07 21:45:40 +02:00
Jakob Unterwurzacher
fe3890688a fusefrontend: xattr: return ENOSYS on unsupported flags
We previously returned EPERM to prevent the kernel from
blacklisting our xattr support once we get an unsupported
flag, but this causes lots of trouble on MacOS:
Cannot save files from GUI apps, see
https://github.com/rfjakob/gocryptfs/issues/229

Returning ENOSYS triggers the dotfiles fallback on MacOS
and fixes the issue.
2018-05-01 23:30:53 +02:00
bolshevik
12b32aa06c Improved xattr handling on non-linux systems (#227)
* Fixed xattr filtering for MacOS. "system." and "user." prefixes are only relevant for Linux.
* Small cleanup and additional tests.
2018-04-17 20:33:04 +02:00
Jakob Unterwurzacher
4e5783591f fsck: report skipped corrupt files
OpenDir and ListXAttr skip over corrupt entries,
readFileID treats files the are too small as empty.
This improves usability in the face of corruption,
but hides the problem in a log message instead of
putting it in the return code.

Create a channel to report these corruptions to fsck
so it can report them to the user.

Also update the manpage and the changelog with the -fsck option.

Closes https://github.com/rfjakob/gocryptfs/issues/191
2018-04-03 21:24:48 +02:00
Jakob Unterwurzacher
4407ca3a4d fusefrontend: xattr: return ENODATA for security.* and system.*
"ls -l" queries security.selinux, system.posix_acl_access, system.posix_acl_default
and throws error messages if it gets something else than ENODATA.
2018-04-02 18:59:14 +02:00
Jakob Unterwurzacher
b6c8960b01 fsck: clean up log output
Make sure we get only 1 warning output per
problem.

Also, add new corruption types to broken_fs_v1.4.
2018-04-02 18:32:30 +02:00
Jakob Unterwurzacher
fb06c65ee9 fusefronted: reject oversized Read and Write requests
This should not happen via FUSE as the kernel caps the size,
but with fsck we have the first user that calls Read directly.
For symmetry, check it for Write as well.
2018-04-01 21:21:55 +02:00
Jakob Unterwurzacher
1a3d04ab87 Switch from private copy to pkg/xattr
Now that https://github.com/pkg/xattr/pull/24
has been merged there is no reason to keep
our private copy.

Switch to the upstream version.
2018-03-28 19:19:58 +02:00
Jakob Unterwurzacher
db778aae7d fusefrontend: handle empty xattrs efficiently
We handle empty files by storing an actual empty file
on disk. Handle xattrs similarily and encrypt the
empty value to the empty value.
2018-03-25 21:06:10 +02:00
Jakob Unterwurzacher
1ed3d51df1 fusefrontend: add xattr support
At the moment, only for reverse mode.

https://github.com/rfjakob/gocryptfs/issues/217
2018-03-25 21:06:10 +02:00
Jakob Unterwurzacher
3d54fc3a3a fusefrontend: create helpers for symlink encryption
These will be reused by the upcoming xattr support.
2018-03-24 21:40:11 +01:00
Jakob Unterwurzacher
b96e3ee271 tlog: stop embedding log.Logger to prevent mistakes
A few places have called tlog.Warn.Print, which directly
calls into log.Logger due to embedding, losing all features
of tlog.

Stop embedding log.Logger to make sure the internal functions
cannot be called accidentially and fix (several!) instances
that did.
2018-02-28 09:02:18 +01:00
Jakob Unterwurzacher
719693ec5d fusefrontend[_reverse]: move crypto init up to caller
Both fusefrontend and fusefrontend_reverse were doing
essentially the same thing, move it into main's
initFuseFrontend.

A side-effect is that we have a reference to cryptocore
in main, which will help with wiping the keys on exit
(https://github.com/rfjakob/gocryptfs/issues/211).
2018-02-18 11:21:58 +01:00
Jakob Unterwurzacher
ea51837361 fusefrontend: drop unused haveGetdents warning
We don't actually print that warning anymore.
2018-01-25 09:06:06 +01:00
Jakob Unterwurzacher
8951eb2472 fusefronted: add PlaintextNames special-cases for Create & Rename
gocryptfs.longname.XXX files were considered magic in PlaintextNames
mode, which was wrong.

Fix that and add tests.

Fixes https://github.com/rfjakob/gocryptfs/issues/174
2018-01-17 00:25:36 +01:00
Sebastian Lackner
a85dbcab38 fusefrontend: Use Linkat syscall to implement Link 2017-12-25 15:07:37 +01:00
Sebastian Lackner
a24342f656 fusefrontend: Handle PlaintextNames mode in Link
In PlaintextNames mode the "gocryptfs.longname." prefix does not have any
special meaning.

https://github.com/rfjakob/gocryptfs/issues/174
2017-12-25 15:07:37 +01:00
Jakob Unterwurzacher
70bcf58a9b syscallcompat: convert Getdents to fd input, add emulation
Now that we have Fstatat we can use it in Getdents to
get rid of the path name.

Also, add an emulated version of getdents for MacOS. This allows
to drop the !HaveGetdents special cases from fusefrontend.

Modify the getdents test to test both native getdents and the emulated
version.
2017-12-03 19:33:26 +01:00
Sebastian Lackner
9bcde0c09e fusefrontend: Improve documentation of mkdirWithIv and WriteDirIV
As requested in https://github.com/rfjakob/gocryptfs/pull/179
2017-12-01 09:41:52 +01:00
Jakob Unterwurzacher
e97c23e083 syscallcompat: check that we get NOFOLLOW wherever possible
...and fix the instances where the AT_SYMLINK_NOFOLLOW /
O_NOFOLLOW / O_EXCL flag was missing.
2017-11-30 19:40:53 +01:00
Sebastian Lackner
614745ee57 fusefrontend: allow_other: close race between mkdir and chown
Fixes the same problem as described in 72b975867a,
except for directories instead of device nodes.
2017-11-29 13:28:04 +01:00
Sebastian Lackner
67bcbe81e8 fusefrontend: Use Fchownat to implement Chown 2017-11-29 13:05:46 +01:00