Commit Graph

347 Commits

Author SHA1 Message Date
Sebastian Lackner
fcaca5fc94 fusefrontend: -allow_other: set file mode after chown in Mkdir().
Make sure that the directory belongs to the correct owner before users
can access it. For directories with SUID/SGID mode, there is a risk of
race-conditions when files are created before the correct owner is set.
They will then inherit the wrong user and/or group.

See https://github.com/rfjakob/gocryptfs/issues/327 for more details.
2019-01-09 20:48:00 +01:00
Jakob Unterwurzacher
b22cc03c75 fusefrontend: -allow_other: set file mode *after* chown in Create()
Reported by @slackner at https://github.com/rfjakob/gocryptfs/issues/327 :

Possible race-conditions between file creation and Fchownat

* Assume a system contains a gocryptfs mount as root user
  with -allow_other
* As a regular user create a new file with mode containing
  the SUID flag and write access for other users
* Before gocryptfs executes the Fchownat call, try to open
  the file again, write some exploit code to it, and try to run it.

For a short time, the file is owned by root and has the SUID flag, so
this is pretty dangerous.
2019-01-08 21:50:10 +01:00
Sebastian Lackner
57a52d6aef fusefrontend: Clarify access mode check related to O_WRONLY handling.
Use O_ACCMODE mask in openWriteOnlyFile for improved readability.
2019-01-07 21:25:23 +01:00
Sebastian Lackner
2332462e78 fusefrontend: Filter O_CREAT in mangleOpenFlags. 2019-01-07 21:19:13 +01:00
Sebastian Lackner
aa2fa24c42 fusefrontend: Check result of Fchmod syscall.
Fixes https://github.com/rfjakob/gocryptfs/issues/328
2019-01-06 21:00:17 +01:00
Sebastian Lackner
40d2427fd7 fusefrontend: Fix computation of cipherSz in Allocate FUSE call.
Do not use PlainSizeToCipherSize() since this adds the 18 bytes file header.

Partially fixes https://github.com/rfjakob/gocryptfs/issues/311
2019-01-06 20:56:59 +01:00
Sebastian Lackner
8310dd95be fusefrontend: Properly convert plaintext <-> ciphertext offsets in SeekData().
Fixes https://github.com/rfjakob/gocryptfs/issues/304
2019-01-06 12:21:54 +01:00
Sebastian Lackner
7e05e809b7 main: Run 'ensure fds' code early during the program startup.
The files are apparently processed in alphabetic order, so cli_args.go is
processed before main.go. In order to run before the go-fuse imports, put
the 'ensure fds' code in a separate package. Debug messages are omitted
to avoid additional imports (that might contain other code messing up our
file descriptors).
2019-01-05 16:12:16 +01:00
Sebastian Lackner
5055f39bd5 fusefrontend: Allow to set/remove xattr on directory without read permission.
Setting/removing extended attributes on directories was partially fixed with
commit eff35e60b6. However, on most file systems
it is also possible to do these operations without read access (see tests).

Since we cannot open a write-access fd to a directory, we have to use the
/proc/self/fd trick (already used for ListXAttr) for the other operations aswell.
For simplicity, let's separate the Linux and Darwin code again (basically revert
commit f320b76fd1), and always use the
/proc/self/fd trick on Linux. On Darwin we use the best-effort approach with
openBackingFile() as a fallback.

More discussion about the available options is available in
https://github.com/rfjakob/gocryptfs/issues/308.
2019-01-05 12:34:40 +01:00
Sebastian Lackner
f17721c364 A few more spelling fixes. 2019-01-05 12:27:55 +01:00
Jakob Unterwurzacher
65eded4a22 tests: bump maxCacheFds to 3
As the dirCache now has 3 entries, the tests should accept
up to 3 extra fds without declaring an fd leak.
2019-01-04 23:50:01 +01:00
Jakob Unterwurzacher
c32066c5b0 fusefrontend: fix fd leak in dirCache
The missing break meant that we may find a second
hit in the cache, Dup() a second fd, and leak the first
one.

Thanks @slackner for finding this.
2019-01-04 23:35:48 +01:00
Jakob Unterwurzacher
3473a84963 fusefrontend: print warning when Create() runs out of file descriptors
We alread have this warning in Open(), but xfstests generic/488
causes "too many open files" via Create. Add the same message so
the user sees what is going on.
2019-01-04 23:15:04 +01:00
Jakob Unterwurzacher
eff35e60b6 fusefrontend: fix setting xattrs on directories
Directories cannot be opened read-write. Retry with RDONLY.
2019-01-04 22:22:24 +01:00
Jakob Unterwurzacher
3365cfc02b fusefrontend: disable dirCache stats printing
This was inadvertedly kept enabled after benchmarking.
2019-01-04 22:07:02 +01:00
Jakob Unterwurzacher
c0a7a14cde fusefrontend: upgrade to three-entry dirCache
3 entries should work well for up to three parallel users.
It works well for extractloop.bash (two parallel tar extracts).
2019-01-04 21:45:03 +01:00
Sebastian Lackner
acf7e52022 fusefrontend: Allow to create sparse file of size 4096.
When the old size is zero, there are no existing blocks to merge the
new data with. Directly use Ftruncate if the size is block-aligned.

Fixes https://github.com/rfjakob/gocryptfs/issues/305
2019-01-04 01:38:47 +01:00
Sebastian Lackner
ab169443fd A few more spelling fixes.
Found with the 'codespell' utility.
2019-01-04 01:23:44 +01:00
Sebastian Lackner
a1ba4b6576 Omit syscall.O_RDONLY flag when passing O_PATH.
When O_PATH is specified in flags, flag bits other than O_CLOEXEC, O_DIRECTORY,
and O_NOFOLLOW are ignored.
2019-01-03 18:24:05 +01:00
Sebastian Lackner
0414ef2572 fusefrontend: Use appropriate flags in decryptPathAt. 2019-01-03 18:24:05 +01:00
Sebastian Lackner
c0640ff3ef fusefrontend: Open directory with syscall.O_DIRECTORY in OpenDir. 2019-01-03 18:24:05 +01:00
Sebastian Lackner
078a431493 fusefrontend: Open directory with syscall.O_DIRECTORY in Rmdir. 2019-01-03 18:24:05 +01:00
Sebastian Lackner
885a341df6 fusefrontend: Do not Clear cache at end of Rmdir function.
We already do 'defer fs.dirCache.Clear()', so this is no longer required.
2019-01-03 18:24:05 +01:00
Sebastian Lackner
d86f9914ac fusefrontend: Remove debug code.
This code was accidentially added in 4f66d66755.
2019-01-03 18:24:05 +01:00
Jakob Unterwurzacher
fcdb4bec09 fusefronted: dirCache: fix bug handling ""
Bug looked like this:

  $ ls -l .
  total 0
  drwxrwxr-x. 2 jakob jakob 60 Jan  3 15:42 foo
  -rw-rw-r--. 1 jakob jakob  0 Jan  3 15:46 x

  $ ls -l .
  ls: cannot access '.': No such file or directory

(only happened when "" was in the dirCache)
2019-01-03 15:59:54 +01:00
Jakob Unterwurzacher
4f66d66755 fusefrontend: add dirCache 2019-01-03 15:31:13 +01:00
Jakob Unterwurzacher
f6dad8d0fa nametransform: simplify WriteDirIV to WriteDirIVAt
Un-spaghettify the function and let the callers open
the directory.
2019-01-03 15:31:13 +01:00
Jakob Unterwurzacher
0fd7637624 fusefrontend: use O_RDONLY in the ListXAttr fallback path
Copy-paste error.

https://github.com/rfjakob/gocryptfs/issues/308
2019-01-02 22:20:44 +01:00
Jakob Unterwurzacher
352f3147c5 fusefrontend: move openBackingDir into its own file
This function is in all fastpaths, will get a cache, and needs
its own file.
renamed:    internal/fusefrontend/names.go -> internal/fusefrontend/openbackingdir.go
renamed:    internal/fusefrontend/names_test.go -> internal/fusefrontend/openbackingdir_test.go
2019-01-02 21:52:52 +01:00
Jakob Unterwurzacher
2b12bba274 fusefronted: make EncryptPath symlink-safe
Finally allows us to delete EncryptPathDirIV.
2019-01-02 21:45:40 +01:00
Jakob Unterwurzacher
b214be5e3f fusefrontend: xattr: fix operations on files without read permissions
* listxattr is fixed via the /proc/self/fd trick
* setxattr,removexattr are fixed by opening the file O_WRONLY

Fixes https://github.com/rfjakob/gocryptfs/issues/308
2019-01-02 20:48:46 +01:00
Jakob Unterwurzacher
bb7f919674 fusefrontend: don't downgrade type needlessly 2019-01-02 20:45:08 +01:00
Jakob Unterwurzacher
f320b76fd1 fusefrontend: use Fsetxattr/Fgetxattr/etc on all platforms
Darwin now also has these functions, use them. Simplifies
the code and makes it symlink-safe on Darwin as well.
2019-01-02 16:58:48 +01:00
Jakob Unterwurzacher
5aa1755cbc fusefrontend: openBackingDir: fix fd leak in error path
Reported by @slackner at
932efbd459 (r31813373)
thanks!
2019-01-02 00:14:12 +01:00
Jakob Unterwurzacher
cd0ec342b9 fusefrontend: fix fd leak in error path 2019-01-01 20:49:56 +01:00
Jakob Unterwurzacher
77c3df48ef fusefrontend: fix fd leak in Access()
Thanks @slackner!

Fixes https://github.com/rfjakob/gocryptfs/issues/306
2019-01-01 20:10:17 +01:00
Jakob Unterwurzacher
60e7a0ca9f fusefrontend: xattr: fix hang on FIFOs
An Open() a fifo blocks until it is opened for writing.
This meant that xattr operations on FIFOs would block.
Pass O_NONBLOCK to fix that, and add a test.
2019-01-01 16:24:25 +01:00
Jakob Unterwurzacher
1d5500c3db fusefrontend: only compile getBackingPath() on Darwin
This function is NOT symlink-safe. Darwin needs it because it lacks
fgetxattr(2) and friends.
2019-01-01 16:24:25 +01:00
Jakob Unterwurzacher
c3adf9729d fusefrontend: make ListXAttr symlink-safe on Linux
Uses /proc/self/fd.
2019-01-01 16:24:25 +01:00
Jakob Unterwurzacher
d3ae87fa2b fusefrontend: make RemoveXAttr() symlink-safe
Uses /proc/self/fd on Linux.
2019-01-01 16:24:25 +01:00
Jakob Unterwurzacher
810d2a8b47 fusefrontend: make SetXAttr() symlink-safe on Linux
Uses the /proc/self/fd trick.
2019-01-01 16:24:25 +01:00
Jakob Unterwurzacher
2286372603 fusefrontend: make GetXAttr() symlink-safe on Linux
Uses the /proc/self/fd trick, which does not work
on Darwin.
2019-01-01 16:24:25 +01:00
Jakob Unterwurzacher
a355670ca2 fusefrontend: make Utimens symlink-safe
unix.UtimesNanoAt now also exists on Darwin, yay!
2019-01-01 16:24:25 +01:00
Jakob Unterwurzacher
abbdaa8ea4 fusefrontend: fix compile failure on Darwin
Failure was:

 + GOOS=darwin
 + GOARCH=amd64
 + go build -tags without_openssl
 # github.com/rfjakob/gocryptfs/internal/fusefrontend
 internal/fusefrontend/fs_dir.go:159:60: cannot use origMode | 448 (type uint16) as type uint32 in argument to syscallcompat.Fchmodat
 internal/fusefrontend/fs_dir.go:170:33: cannot use origMode (type uint16) as type uint32 in argument to syscallcompat.Fchmodat
2019-01-01 16:24:25 +01:00
Jakob Unterwurzacher
d4b7f42c3b fusefrontend: mark Truncate, Unlink, Symlink symlink-safe
No changes needed.
2019-01-01 16:24:25 +01:00
Jakob Unterwurzacher
436f918c21 fusefrontend: make Rmdir symlink-safe
Now uses Unlinkat.
2019-01-01 16:24:25 +01:00
Jakob Unterwurzacher
2de3851abd nametransform: rename WriteLongName() -> WriteLongNameAt()
And also rename DeleteLongName() -> DeleteLongNameAt(). The
naming follow the names open the openat() etc syscalls.
2019-01-01 16:24:25 +01:00
Jakob Unterwurzacher
4fae240153 fusefrontend: make Readlink() symlink-safe
Now symlink-safe through Readlinkat().
2019-01-01 16:24:25 +01:00
Jakob Unterwurzacher
21f1f858b9 fusefrontend: make OpenDir() symlink-safe
Interestingly, little or no performance impact:

$ ./benchmark.bash
Testing gocryptfs at /tmp/benchmark.bash.39W: gocryptfs v1.6-42-g30c2349-dirty; go-fuse v20170619-66-g6df8ddc; 2018-11-04 go1.11
Downloading linux-3.0.tar.gz
/tmp/linux-3.0.tar.gz                  100%[=========================================================================>]  92.20M  2.93MB/s    in 31s
2018-11-04 21:44:44 URL:https://cdn.kernel.org/pub/linux/kernel/v3.0/linux-3.0.tar.gz [96675825/96675825] -> "/tmp/linux-3.0.tar.gz" [1]
WRITE: 262144000 bytes (262 MB, 250 MiB) copied, 1.1808 s, 222 MB/s
READ:  262144000 bytes (262 MB, 250 MiB) copied, 0.866438 s, 303 MB/s
UNTAR: 24.745
MD5:   12.050
LS:    3.525
RM:    9.544

Note: kernel has been updated:

$ uname -a
Linux brikett 4.18.16-200.fc28.x86_64 #1 SMP Sat Oct 20 23:53:47 UTC 2018 x86_64 x86_64 x86_64 GNU/Linux
2019-01-01 16:24:25 +01:00
Jakob Unterwurzacher
de3a2c1895 fusefrontend: mark a few more functions as symlink-safe / unsafe 2019-01-01 16:24:25 +01:00
Jakob Unterwurzacher
8586a83825 fusefrontend: use openBackingDir in ctlsock interface
Instead of calling syscall.Open() ourselves, rely on
openBackingDir().
2019-01-01 16:24:20 +01:00
Jakob Unterwurzacher
0c1ceed1fa fusefrontend: make GetAttr() symlink-safe
Use openBackingDir() and Fstatat().

High performance impact, though part of it should be
mitigated by adding DirIV caching to the new code paths.

$ ./benchmark.bash
Testing gocryptfs at /tmp/benchmark.bash.Eou: gocryptfs v1.6-37-ge3914b3-dirty; go-fuse v20170619-66-g6df8ddc; 2018-10-14 go1.11
WRITE: 262144000 bytes (262 MB, 250 MiB) copied, 1.2289 s, 213 MB/s
READ:  262144000 bytes (262 MB, 250 MiB) copied, 1.02616 s, 255 MB/s
UNTAR: 24.490
MD5:   13.120
LS:    3.368
RM:    9.232
2019-01-01 16:24:09 +01:00
Jakob Unterwurzacher
932efbd459 fusefrontend: make openBackingDir() symlink-safe
openBackingDir() used encryptPath(), which is not symlink-safe
itself. Drop encryptPath() and implement our own directory walk.

Adds three seconds to untar and two seconds to rm:

$ ./benchmark.bash
Testing gocryptfs at /tmp/benchmark.bash.MzG: gocryptfs v1.6-36-g8fb3c2f-dirty; go-fuse v20170619-66-g6df8ddc; 2018-10-14 go1.11
WRITE: 262144000 bytes (262 MB, 250 MiB) copied, 1.25078 s, 210 MB/s
READ:  262144000 bytes (262 MB, 250 MiB) copied, 1.0318 s, 254 MB/s
UNTAR: 20.941
MD5:   11.568
LS:    1.638
RM:    5.337
2019-01-01 16:24:09 +01:00
Jakob Unterwurzacher
0e2e7c13cf fusefrontend: mark symlink-safe FUSE calls
Document which FUSE calls are already symlink-safe in
the function comment.
2019-01-01 16:24:09 +01:00
Jakob Unterwurzacher
c09bf1f228 fusefrontend: make DecryptPath() symlink-safe
DecryptPath is now symlink-safe through the use of *at()
functions.
2019-01-01 16:24:09 +01:00
Jakob Unterwurzacher
ed6ed513d7 fusefrontend: make Access() symlink-safe.
Make Access() symlink-safe through use of faccessat.
2019-01-01 16:24:09 +01:00
Sebastian Lackner
5713154468 fusefrontend: Fix debug message in doWrite() method. 2019-01-01 16:12:42 +01:00
Sebastian Lackner
9ed60678e5 fusefrontend: Fix order of arguments in debug message for Read() FUSE call. 2019-01-01 16:12:05 +01:00
Sebastian Lackner
4c2ff26457 fusefrontend: Remove unnecessary check in doRead function.
The same condition is already checked a few lines above, and 'err' is not
changed inbetween.
2018-12-27 15:18:03 +01:00
Sebastian Lackner
1ced0b192e fusefrontend: Don't treat Fchownat error as failure in Mkdir.
The directory was already created, so return success even if Fchownat fails.
The same error handling is already used if fs.args.PlaintextNames is false.
2018-12-27 15:16:00 +01:00
Sebastian Lackner
5918884926 fusefrontend: Check the correct 'err' variable. 2018-12-27 15:11:23 +01:00
Jakob Unterwurzacher
4cdf6b9af9 fusefronted: log more details on WriteAt failures
Also log inode number, fd number, offset and length.

Maybe help debugging https://github.com/rfjakob/gocryptfs/issues/269 .
2018-10-17 22:18:07 +02:00
Jesse Dunietz
87d3ed9187 Add option for autounmount
Even though filesystem notifications aren't implemented for FUSE, I decided to
try my hand at implementing the autounmount feature (#128). I based it on the
EncFS autounmount code, which records filesystem accesses and checks every X
seconds whether it's idled long enough to unmount.

I've tested the feature locally, but I haven't added any tests for this flag.
I also haven't worked with Go before. So please let me know if there's
anything that should be done differently.

One particular concern: I worked from the assumption that the open files table
is unique per-filesystem. If that's not true, I'll need to add an open file
count and associated lock to the Filesystem type instead.

https://github.com/rfjakob/gocryptfs/pull/265
2018-10-11 20:16:45 +02:00
Jakob Unterwurzacher
e4f1a32a9a fusefrontend: Fix uint16 build failure on Darwin
Error was:

  # github.com/rfjakob/gocryptfs/internal/fusefrontend
  internal/fusefrontend/fs.go:179: cannot use perms | 256 (type uint16) as type uint32 in argument to syscall.Fchmod
  internal/fusefrontend/fs.go:185: cannot use perms (type uint16) as type uint32 in argument to syscall.Fchmod
2018-09-23 12:17:59 +02:00
Jakob Unterwurzacher
a1fb456618 fusefrontend: make Rename() symlink-safe
Use Openat() and the openBackingDir() helper so we
never follow symlinks.
2018-09-23 12:17:59 +02:00
Jakob Unterwurzacher
897bb8924f fusefrontend: make Create() symlink-safe
Use Openat() and the openBackingDir() helper so we
never follow symlinks.
2018-09-23 12:17:59 +02:00
Jakob Unterwurzacher
63762b33af fusefrontend: Open(): fix dirfd leak
Close was missing.
2018-09-23 12:17:59 +02:00
Jakob Unterwurzacher
bead82c9fb fusefrontend: add named parameters to openBackingDir
Named parameters make using the function easier.
2018-09-23 12:17:59 +02:00
Jakob Unterwurzacher
c270b21efc fusefrontend: get rid of os.File* wrapping
Directly use int file descriptors for the dirfd
and get rid of one level of indirection.
2018-09-23 12:17:26 +02:00
Jakob Unterwurzacher
22fba4ac3e fusefrontent: make Open() symlink-safe 2018-09-23 12:17:26 +02:00
Jakob Unterwurzacher
e8d8ae54d3 fusefrontend: use OpenDirNofollow in openBackingDir
Rename openBackingPath to openBackingDir and use OpenDirNofollow
to be safe against symlink races. Note that openBackingDir is
not used in several important code paths like Create().

But it is used in Unlink, and the performance impact in the RM benchmark
to be acceptable:

Before

	$ ./benchmark.bash
	Testing gocryptfs at /tmp/benchmark.bash.bYO: gocryptfs v1.6-12-g930c37e-dirty; go-fuse v20170619-49-gb11e293; 2018-09-08 go1.10.3
	WRITE: 262144000 bytes (262 MB, 250 MiB) copied, 1.07979 s, 243 MB/s
	READ:  262144000 bytes (262 MB, 250 MiB) copied, 0.882413 s, 297 MB/s
	UNTAR: 16.703
	MD5:   7.606
	LS:    1.349
	RM:    3.237

After

	$ ./benchmark.bash
	Testing gocryptfs at /tmp/benchmark.bash.jK3: gocryptfs v1.6-13-g84d6faf-dirty; go-fuse v20170619-49-gb11e293; 2018-09-08 go1.10.3
	WRITE: 262144000 bytes (262 MB, 250 MiB) copied, 1.06261 s, 247 MB/s
	READ:  262144000 bytes (262 MB, 250 MiB) copied, 0.947228 s, 277 MB/s
	UNTAR: 17.197
	MD5:   7.540
	LS:    1.364
	RM:    3.410
2018-09-08 19:27:33 +02:00
Jakob Unterwurzacher
dbd400d930 fusefrontend: truncateGrowFile: pass zeroPad error to caller
Errors from zeroPad were ignored until now, as discovered
using xfstests generic/083.
2018-08-15 17:25:22 +02:00
Jakob Unterwurzacher
ec2fdc19cf reverse mode: add --exclude option
https://github.com/rfjakob/gocryptfs/issues/235
2018-08-11 23:26:49 +02:00
Jakob Unterwurzacher
f316f1b2df fusefronted: disallow writes running concurrently with reads
As uncovered by xfstests generic/465, concurrent reads and writes
could lead to this,

  doRead 3015532: corrupt block #1039: stupidgcm: message authentication failed,

as the read could pick up a block that has not yet been completely written -
write() is not atomic!

Now writes take ContentLock exclusively, while reads take it shared,
meaning that multiple reads can run in parallel with each other, but
not with a write.

This also simplifies the file header locking.
2018-07-22 22:29:22 +02:00
Jakob Unterwurzacher
c70df522d2 fusefrontend: doWrite: delete file header if first write fails
xfstests generic/083 fills the filesystem almost completely while
running fsstress in parallel. In fsck, these would show up:

  readFileID 2580: incomplete file, got 18 instead of 19 bytes

This could happen when writing the file header works, but writing
the actual data fails.

Now we kill the header again by truncating the file to zero.
2018-07-15 15:12:55 +02:00
Jakob Unterwurzacher
55bb22bad6 fusefrontend: doWrite: no need to take HeaderLock.RLock()
Other writers are blocked by ContentLock already.
2018-07-15 12:40:23 +02:00
Jakob Unterwurzacher
95b93db35f fusefrontend: log prealloc failures at Info level
If the underlying filesystem is full, it is normal get ENOSPC here.
Log at Info level instead of Warning.

Fixes xfstests generic/015 and generic/027, which complained about
the extra output.
2018-07-14 15:18:27 +02:00
Jakob Unterwurzacher
53f7e1a0f0 macos: fix O_DIRECT build failure
O_DIRECT has no direct equivalent on MacOS
(check out https://github.com/libuv/libuv/issues/1600 for details).

Just define it to zero there.
2018-07-04 09:04:00 +02:00
Jakob Unterwurzacher
893e41149e fusefrontend: disallow O_DIRECT and fall back to buffered IO
O_DIRECT accesses must be aligned in both offset and length. Due to our
crypto header, alignment will be off, even if userspace makes aligned
accesses. Running xfstests generic/013 on ext4 used to trigger lots of
EINVAL errors due to missing alignment. Just fall back to buffered IO.
2018-07-02 23:54:37 +02:00
Jakob Unterwurzacher
c51fc9e07d fusefronted: downgrade fallocate message severity
The message causes output mismatches in xfstests generic/112.
Downgrade the severity to Info so it gets disabled when using "-q".
2018-07-02 23:03:43 +02:00
Jakob Unterwurzacher
01a078e7c0 Fix golint warnings 2018-07-01 22:00:06 +02:00
Jakob Unterwurzacher
e951043084 fusefrontend: add File.SeekData() function
This function will enable "gocryptfs -fsck" to handle
sparse files efficiently.
2018-07-01 20:56:22 +02:00
Jakob Unterwurzacher
a2af1fb5da fusefrontend: export "File" type
"gocryptfs -fsck" will need access to helper functions,
and to get that, it will need to cast a gofuse.File to a
fusefrontend.File. Make fusefrontend.File exported to make
this work.
2018-07-01 20:56:22 +02:00
Jakob Unterwurzacher
1a18d8e609 fsck: rename "CorruptItems" channel to "MitigatedCorruptions"
Make it clear that this channel is only used to report corruptions
that are transparently mitigated and do not return an error to
the user.
2018-07-01 20:56:22 +02:00
Jakob Unterwurzacher
1bab400fca Fix three golint warnings
We are clean again.

Warnings were:

internal/fusefrontend/fs.go:443:14: should omit type string from declaration
of var cTarget; it will be inferred from the right-hand side
internal/fusefrontend/xattr.go:26:1: comment on exported method FS.GetXAttr
should be of the form "GetXAttr ..."
internal/syscallcompat/sys_common.go:9:7: exported const PATH_MAX should have
comment or be unexported
2018-06-19 20:16:21 +02:00
Jakob Unterwurzacher
bfa50517e9 xattr: return EOPNOTSUPP instead of ENODATA in GetXattr
Reading system.posix_acl_access and system.posix_acl_default
should return EOPNOTSUPP to inform user-space that we do not
support ACLs.

xftestest essientially does

	chacl -l | grep "Operation not supported"

to determine if the filesystem supports ACLs, and used to
wrongly believe that gocryptfs does.
2018-06-12 23:05:53 +02:00
Jakob Unterwurzacher
ae02ca1ded xattr: use LGet/LSet etc
Support has been merged into the xattr package
( https://github.com/pkg/xattr/pull/29 ), use it.
2018-05-27 20:09:48 +02:00
Jakob Unterwurzacher
7b00681807 xattr: return EOPNOTSUPP for unsupported attributes
mv is unhappy when we return EPERM when it tries to set
system.posix_acl_access:

   mv: preserving permissions for ‘b/x’: Operation not permitted

Now we return EOPNOTSUPP like tmpfs does and mv seems happy.
2018-05-15 23:00:47 +02:00
Bolshevik
a41ec2028c xattr: optimize storage, store as binary instead of bae64
Values a binary-safe, there is no need to base64-encode them.

Old, base64-encoded values are supported transparently
on reading. Writing xattr values now always writes them binary.
2018-05-10 23:25:49 +02:00
Bolshevik
5ccc06d5cb xattr: added passing of a "flags" parameter
Pass the "flags" parameter to the lower layer syscall.
This makes Apple applications being able to successfully save data.
2018-05-07 21:45:40 +02:00
Jakob Unterwurzacher
fe3890688a fusefrontend: xattr: return ENOSYS on unsupported flags
We previously returned EPERM to prevent the kernel from
blacklisting our xattr support once we get an unsupported
flag, but this causes lots of trouble on MacOS:
Cannot save files from GUI apps, see
https://github.com/rfjakob/gocryptfs/issues/229

Returning ENOSYS triggers the dotfiles fallback on MacOS
and fixes the issue.
2018-05-01 23:30:53 +02:00
bolshevik
12b32aa06c Improved xattr handling on non-linux systems (#227)
* Fixed xattr filtering for MacOS. "system." and "user." prefixes are only relevant for Linux.
* Small cleanup and additional tests.
2018-04-17 20:33:04 +02:00
Jakob Unterwurzacher
4e5783591f fsck: report skipped corrupt files
OpenDir and ListXAttr skip over corrupt entries,
readFileID treats files the are too small as empty.
This improves usability in the face of corruption,
but hides the problem in a log message instead of
putting it in the return code.

Create a channel to report these corruptions to fsck
so it can report them to the user.

Also update the manpage and the changelog with the -fsck option.

Closes https://github.com/rfjakob/gocryptfs/issues/191
2018-04-03 21:24:48 +02:00
Jakob Unterwurzacher
4407ca3a4d fusefrontend: xattr: return ENODATA for security.* and system.*
"ls -l" queries security.selinux, system.posix_acl_access, system.posix_acl_default
and throws error messages if it gets something else than ENODATA.
2018-04-02 18:59:14 +02:00
Jakob Unterwurzacher
b6c8960b01 fsck: clean up log output
Make sure we get only 1 warning output per
problem.

Also, add new corruption types to broken_fs_v1.4.
2018-04-02 18:32:30 +02:00
Jakob Unterwurzacher
fb06c65ee9 fusefronted: reject oversized Read and Write requests
This should not happen via FUSE as the kernel caps the size,
but with fsck we have the first user that calls Read directly.
For symmetry, check it for Write as well.
2018-04-01 21:21:55 +02:00
Jakob Unterwurzacher
1a3d04ab87 Switch from private copy to pkg/xattr
Now that https://github.com/pkg/xattr/pull/24
has been merged there is no reason to keep
our private copy.

Switch to the upstream version.
2018-03-28 19:19:58 +02:00
Jakob Unterwurzacher
db778aae7d fusefrontend: handle empty xattrs efficiently
We handle empty files by storing an actual empty file
on disk. Handle xattrs similarily and encrypt the
empty value to the empty value.
2018-03-25 21:06:10 +02:00
Jakob Unterwurzacher
1ed3d51df1 fusefrontend: add xattr support
At the moment, only for reverse mode.

https://github.com/rfjakob/gocryptfs/issues/217
2018-03-25 21:06:10 +02:00
Jakob Unterwurzacher
3d54fc3a3a fusefrontend: create helpers for symlink encryption
These will be reused by the upcoming xattr support.
2018-03-24 21:40:11 +01:00
Jakob Unterwurzacher
b96e3ee271 tlog: stop embedding log.Logger to prevent mistakes
A few places have called tlog.Warn.Print, which directly
calls into log.Logger due to embedding, losing all features
of tlog.

Stop embedding log.Logger to make sure the internal functions
cannot be called accidentially and fix (several!) instances
that did.
2018-02-28 09:02:18 +01:00
Jakob Unterwurzacher
719693ec5d fusefrontend[_reverse]: move crypto init up to caller
Both fusefrontend and fusefrontend_reverse were doing
essentially the same thing, move it into main's
initFuseFrontend.

A side-effect is that we have a reference to cryptocore
in main, which will help with wiping the keys on exit
(https://github.com/rfjakob/gocryptfs/issues/211).
2018-02-18 11:21:58 +01:00
Jakob Unterwurzacher
ea51837361 fusefrontend: drop unused haveGetdents warning
We don't actually print that warning anymore.
2018-01-25 09:06:06 +01:00
Jakob Unterwurzacher
8951eb2472 fusefronted: add PlaintextNames special-cases for Create & Rename
gocryptfs.longname.XXX files were considered magic in PlaintextNames
mode, which was wrong.

Fix that and add tests.

Fixes https://github.com/rfjakob/gocryptfs/issues/174
2018-01-17 00:25:36 +01:00
Sebastian Lackner
a85dbcab38 fusefrontend: Use Linkat syscall to implement Link 2017-12-25 15:07:37 +01:00
Sebastian Lackner
a24342f656 fusefrontend: Handle PlaintextNames mode in Link
In PlaintextNames mode the "gocryptfs.longname." prefix does not have any
special meaning.

https://github.com/rfjakob/gocryptfs/issues/174
2017-12-25 15:07:37 +01:00
Jakob Unterwurzacher
70bcf58a9b syscallcompat: convert Getdents to fd input, add emulation
Now that we have Fstatat we can use it in Getdents to
get rid of the path name.

Also, add an emulated version of getdents for MacOS. This allows
to drop the !HaveGetdents special cases from fusefrontend.

Modify the getdents test to test both native getdents and the emulated
version.
2017-12-03 19:33:26 +01:00
Sebastian Lackner
9bcde0c09e fusefrontend: Improve documentation of mkdirWithIv and WriteDirIV
As requested in https://github.com/rfjakob/gocryptfs/pull/179
2017-12-01 09:41:52 +01:00
Jakob Unterwurzacher
e97c23e083 syscallcompat: check that we get NOFOLLOW wherever possible
...and fix the instances where the AT_SYMLINK_NOFOLLOW /
O_NOFOLLOW / O_EXCL flag was missing.
2017-11-30 19:40:53 +01:00
Sebastian Lackner
614745ee57 fusefrontend: allow_other: close race between mkdir and chown
Fixes the same problem as described in 72b975867a,
except for directories instead of device nodes.
2017-11-29 13:28:04 +01:00
Sebastian Lackner
67bcbe81e8 fusefrontend: Use Fchownat to implement Chown 2017-11-29 13:05:46 +01:00
Sebastian Lackner
0162392a28 fusefrontend: Use Fchmodat to implement Chmod 2017-11-29 12:55:41 +01:00
Sebastian Lackner
0f44c617d0 syscallcompat: Introduce unlinkat syscall with flags argument 2017-11-29 12:41:23 +01:00
Sebastian Lackner
5d44a31b41 fusefrontend: Use openBackingPath in Unlink and simplify code 2017-11-28 09:28:06 +01:00
Sebastian Lackner
2591900b69 fusefrontend: Handle PlaintextNames mode in Unlink
In PlaintextNames mode the "gocryptfs.longname." prefix does not have any
special meaning. We should not attempt to delete any .name files.

Partially fixes https://github.com/rfjakob/gocryptfs/issues/174
2017-11-28 09:28:06 +01:00
Sebastian Lackner
eba49402e4 fusefrontend: Introduce a openBackingPath helper and use it to simplify Mknod and Symlink 2017-11-28 09:28:06 +01:00
Sebastian Lackner
ad2720e0f9 fusefrontend: allow_other: close race between symlink and chown
Fixes the same problem as described in 72b975867a,
except for symlinks instead of device nodes.
2017-11-28 09:28:06 +01:00
Sebastian Lackner
5a56810603 fusefrontend: Use the Symlinkat syscall for longname handling 2017-11-28 09:28:06 +01:00
Sebastian Lackner
295c4c2b85 fusefrontend: Set owner after symlink creation in PlaintextNames mode
This is already done in regular mode, but was missing when PlaintextNames mode
is enabled. As a result, symlinks created by non-root users were still owned
by root afterwards.

Fixes https://github.com/rfjakob/gocryptfs/issues/176
2017-11-28 09:28:06 +01:00
Sebastian Lackner
3f68b0c09a fusefrontend: Handle PlaintextNames mode in Mknod
In PlaintextNames mode the "gocryptfs.longname." prefix does not have any
special meaning. We should not attempt to read the directory IV or to
create special .name files.

Partially fixes https://github.com/rfjakob/gocryptfs/issues/174
2017-11-28 09:28:06 +01:00
Jakob Unterwurzacher
72b975867a fusefronted: allow_other: close race between mknod and chown
If the user manages to replace the directory with
a symlink at just the right time, we could be tricked
into chown'ing the wrong file.

This change fixes the race by using fchownat, which
unfortunately is not available on darwin, hence a compat
wrapper is added.

Scenario, as described by @slackner at
https://github.com/rfjakob/gocryptfs/issues/177 :

1. Create a forward mount point with `plaintextnames` enabled
2. Mount as root user with `allow_other`
3. For testing purposes create a file `/tmp/file_owned_by_root`
   which is owned by the root user
4. As a regular user run inside of the GoCryptFS mount:

```
mkdir tempdir
mknod tempdir/file_owned_by_root p &
mv tempdir tempdir2
ln -s /tmp tempdir
```

When the steps are done fast enough and in the right order
(run in a loop!), the device file will be created in
`tempdir`, but the `lchown` will be executed by following
the symlink. As a result, the ownership of the file located
at `/tmp/file_owned_by_root` will be changed.
2017-11-27 21:04:45 +01:00
Sebastian Lackner
95870e841e fusefrontend: Skip gocryptfs.diriv handling when directory was deleted successfully
Fixes https://github.com/rfjakob/gocryptfs/issues/171

Steps to reproduce:

* Create a regular forward mount point
* Create a new directory in the mount point
* Manually delete the gocryptfs.diriv file from the corresponding ciphertext
  directory
* Attempt to delete the directory with 'rmdir <dirname>'

Although the code explicitly checks for empty directories, it will still attempt
to move the non-existent gocryptfs.diriv file and fails with:

    rmdir: failed to remove '<dirname>': No such file or directory
2017-11-25 16:20:00 +01:00
Sebastian Lackner
9f56b33e0c fusefrontend: Fix longname handling for renames with existing target
Fixes https://github.com/rfjakob/gocryptfs/issues/170

Steps to reproduce the problem:

* Create a regular forward mount point
* Create a file with a shortname and one with a long filename
* Try to run 'mv <shortname> <longname>'

This should actually work and replace the existing file, but instead it
fails with:

    mv: cannot move '<shortname>' to '<longname>': File exists

The problem is the creation of the .name file. If the target already exists
we can safely ignore the EEXIST error and just keep the existing .name file.
2017-11-25 16:19:09 +01:00
Jakob Unterwurzacher
268e0484e2 Revert most of "fusefrontend: clamp oversized reads"
We cannot return less data than requested to the kernel!

From https://libfuse.github.io/doxygen/structfuse__operations.html:

  Read should return exactly the number of bytes
  requested except on EOF or error, otherwise the
  rest of the data will be substituted with
  zeroes.

Reverts commit 3009ec9852 minus
the formatting improvements we want to keep.

Fixes https://github.com/rfjakob/gocryptfs/issues/147
Reopens https://github.com/rfjakob/gocryptfs/issues/145
2017-10-21 17:43:21 +02:00
Jakob Unterwurzacher
3009ec9852 fusefrontend: clamp oversized reads
Our byte cache pools are sized acc. to MAX_KERNEL_WRITE, but the
running kernel may have a higher limit set. Clamp to what we can
handle.

Fixes a panic on a Synology NAS reported at
https://github.com/rfjakob/gocryptfs/issues/145
2017-10-17 21:48:29 +02:00
Jakob Unterwurzacher
604b0779d4 macos: automatically remove .DS_Store on Rmdir
MacOS sprinkles .DS_Store files everywhere. This is hard to avoid for
users, so handle it transparently in Rmdir().

Mitigates https://github.com/rfjakob/gocryptfs/issues/140
2017-09-05 22:47:15 +02:00
Jakob Unterwurzacher
6f3b65d924 fusefrontend: reorder logic in Rmdir to get rid of one indentation level
Handle the errors first so that the normal code path is not indented.

This should not cause any behavoir changes.
2017-09-05 22:10:08 +02:00
Jakob Unterwurzacher
3a5a783b54 macos: don't throw IO errors because of .DS_Store files
MacOS creates lots of these files, and if the directory is otherwise
empty, we would throw an IO error to the unsuspecting user.

With this patch, we log a warning, but otherwise pretend we did not
see it.

Mitigates https://github.com/rfjakob/gocryptfs/issues/140
2017-09-05 21:47:05 +02:00
Jakob Unterwurzacher
538cae610c syscallcompat: Getdents: warn once if we get DT_UNKNOWN
...and if Getdents is not available at all.

Due to this warning I now know that SSHFS always returns DT_UNKNOWN:

    gocryptfs[8129]: Getdents: convertDType: received DT_UNKNOWN, falling back to Lstat

This behavoir is confirmed at http://ahefner.livejournal.com/16875.html:

    "With sshfs, I finally found that obscure case. The dtype is always set to DT_UNKNOWN [...]"
2017-09-03 15:05:54 +02:00
Jakob Unterwurzacher
276567eb13 fusefrontend: use DirIVCache in OpenDir()
Previously, OpenDir() did not use the cache at all, missing
an opportunity to speed up repeated directory reads.
2017-09-03 13:59:53 +02:00
Jakob Unterwurzacher
989b880989 fusefrontend: use Getdents if available
Getdents avoids calling Lstat on each file.
2017-08-15 19:04:02 +02:00
Jakob Unterwurzacher
0c520845f3 main: purge masterkey from memory as soon as possible
Remove the "Masterkey" field from fusefrontend.Args because it
should not be stored longer than neccessary. Instead pass the
masterkey as a separate argument to the filesystem initializers.

Then overwrite it with zeros immediately so we don't have
to wait for garbage collection.

Note that the crypto implementation still stores at least a
masterkey-derived value, so this change makes it harder, but not
impossible, to extract the encryption keys from memory.

Suggested at https://github.com/rfjakob/gocryptfs/issues/137
2017-08-11 19:02:26 +02:00
Jakob Unterwurzacher
3062de6187 fusefronted: enable writing to write-only files
Due to RMW, we always need read permissions on the backing file. This is a
problem if the file permissions do not allow reading (i.e. 0200 permissions).
This patch works around that problem by chmod'ing the file, obtaining a fd,
and chmod'ing it back.

Test included.

Issue reported at: https://github.com/rfjakob/gocryptfs/issues/125
2017-07-11 23:19:58 +02:00
Jakob Unterwurzacher
52ab0462a4 fusefrontend: doRead: skip decryption for an empty read
Previously we ran through the decryption steps even for an empty
ciphertext slice. The functions handle it correctly, but returning
early skips all the extra calls.

Speeds up the tar extract benchmark by about 4%.
2017-07-02 16:02:13 +02:00
Jakob Unterwurzacher
12c0101a23 contentenc: add PReqPool and use it in DecryptBlocks
This gets us a massive speed boost in streaming reads.
2017-06-30 23:30:57 +02:00
Jakob Unterwurzacher
b2a23e94d1 fusefrontend: doRead: use CReqPool for ciphertext buffer
Easily saves lots of allocations.
2017-06-30 23:15:31 +02:00
Jakob Unterwurzacher
06398e82d9 fusefrontend: Read: use provided buffer
This will allow us to return internal buffers to a pool.
2017-06-30 23:11:38 +02:00
Jakob Unterwurzacher
80676c685f contentenc: add safer "bPool" pool variant; add pBlockPool
bPool verifies the lengths of slices going in and out.

Also, add a plaintext block pool - pBlockPool - and use
it for decryption.
2017-06-29 23:44:32 +02:00
Jakob Unterwurzacher
3c6fe98eb1 contentenc: use sync.Pool memory pools for encryption
We use two levels of buffers:

1) 4kiB+overhead for each ciphertext block
2) 128kiB+overhead for each FUSE write (32 ciphertext blocks)

This commit adds a sync.Pool for both levels.

The memory-efficiency for small writes could be improved,
as we now always use a 128kiB buffer.
2017-06-20 21:22:00 +02:00
Charles Duffy
da1bd74246 Fix missing Owner coercion for already-open files (#117) 2017-06-09 22:04:56 +02:00
Jakob Unterwurzacher
a24faa3ba5 fusefrontend: write: consolidate and move encryption to contentenc
Collect all the plaintext and pass everything to contentenc in
one call.

This will allow easier parallization of the encryption.

https://github.com/rfjakob/gocryptfs/issues/116
2017-06-01 22:19:27 +02:00
Charles Duffy
cf1ded5236 Implement force_owner option to display ownership as a specific user. 2017-06-01 00:26:17 +02:00
Jakob Unterwurzacher
e827763f2e nametransform: harden name decryption against invalid input
This fixes a few issues I have found reviewing the code:

1) Limit the amount of data ReadLongName() will read. Previously,
you could send gocryptfs into out-of-memory by symlinking
gocryptfs.diriv to /dev/zero.

2) Handle the empty input case in unPad16() by returning an
error. Previously, it would panic with an out-of-bounds array
read. It is unclear to me if this could actually be triggered.

3) Reject empty names after base64-decoding in DecryptName().
An empty name crashes emeCipher.Decrypt().
It is unclear to me if B64.DecodeString() can actually return
a non-error empty result, but let's guard against it anyway.
2017-05-23 21:26:38 +02:00
Jakob Unterwurzacher
ad7942f434 fusefrontend: implement path decryption via ctlsock
Closes https://github.com/rfjakob/gocryptfs/issues/84 .
2017-05-07 21:01:39 +02:00
Jakob Unterwurzacher
26881538e1 nametranform, fusefrontend: better errors on invalid names
nametransform.DecryptName() now always returns syscall.EBADMSG if
the name was invalid.

fusefrontend.OpenDir error messages have been normalized.
2017-05-07 20:58:27 +02:00
Jakob Unterwurzacher
c52e1abc58 fusefrontend: log "too many open files" errors
This usually indicates that the open file limit for gocryptfs is
too low. We should report this to the user.
2017-05-03 23:46:52 +02:00
Jakob Unterwurzacher
fb3cc6ea40 openfiletable: rename WriteLock to ContentLock
...and IDLock to HeaderLock. This matches what the locks actually
protect.
2017-05-01 21:57:18 +02:00
Jakob Unterwurzacher
f322ee87e3 fusefrontend: rely on nodefs.defaultFile for no-op functions
Now that we embed nodefs.NewDefaultFile(), we can drop our own
no-ops.
2017-05-01 19:12:37 +02:00
Jakob Unterwurzacher
9ab11aa4d7 fusefrontend: drop writeOnly flag
We do not have to track the writeOnly status because the kernel
will not forward read requests on a write-only FD to us anyway.

I have verified this behavoir manually on a 4.10.8 kernel and also
added a testcase.
2017-05-01 17:49:37 +02:00
Jakob Unterwurzacher
514f515dd7 fusefronted, openfiletable: move the open file table to its own package
The open file table code needs some room to grow for the upcoming
FD multiplexing implementation.
2017-05-01 17:26:50 +02:00