Commit Graph

124 Commits (d6e75be3769d9750eba70fa5fbef918a5a4bf8d7)

Author SHA1 Message Date
Sebastian Lackner 42bf6d1c68 syscallcompat: Drop Renameat emulation on macOS. 2019-01-14 21:27:28 +01:00
Sebastian Lackner da557702d7 syscallcompat: Drop Openat emulation on macOS. 2019-01-14 21:27:28 +01:00
Sebastian Lackner d7be766851 syscallcompat: Use pthread_setugid_np() to implement *User() functions on macOS.
Fixes -allow_other mode on macOS.
2019-01-14 21:27:28 +01:00
Jakob Unterwurzacher 711ef81bfb macos: filter SUID and SGID bits in OpenatUser, MknodatUser, MkdiratUser
When gocryptfs runs as root, we don't want to allow people to create
SUID root files.
2019-01-13 14:05:03 +01:00
Sebastian Lackner efc280330c fusefrontend: -allow_other: Use SymlinkatUser in Symlink FUSE call.
Instead of manually adjusting the user after creating the symlink,
adjust effective permissions and let the kernel deal with it.

Related to
2019-01-12 21:22:58 +01:00
Sebastian Lackner 1fbe7798cf fusefrontend: -allow_other: Use MknodatUser in Mknod FUSE call.
Instead of manually adjusting the user and mode after creating the
device file, adjust effective permissions and let the kernel deal
with it.

Related to
2019-01-12 21:20:16 +01:00
Sebastian Lackner a525e33eaa fusefrontend: -allow_other: Use MkdiratUser in Mkdir FUSE call.
Revert commit fcaca5fc94.

Instead of manually adjusting the user and mode after creating the
directory, adjust effective permissions and let the kernel deal with it.

Related to
2019-01-12 21:20:07 +01:00
Sebastian Lackner 03b9d65cce fusefrontend: -allow_other: Use OpenatUser in Create FUSE call.
Revert commit b22cc03c75.

Instead of manually adjusting the user and mode after creating the
file, adjust effective permissions and let the kernel deal with it.

Related to
2019-01-12 20:54:39 +01:00
Sebastian Lackner 4170ef00f3 syscallcompat: Implement workaround for Fchmodat with AT_SYMLINK_NOFOLLOW.
2019-01-07 23:07:53 +01:00
Jakob Unterwurzacher 8253c55386 tests: add Fchmodat test
Test that we handle symlinks correctly.
2019-01-07 23:07:53 +01:00
Sebastian Lackner 5055f39bd5 fusefrontend: Allow to set/remove xattr on directory without read permission.
Setting/removing extended attributes on directories was partially fixed with
commit eff35e60b6. However, on most file systems
it is also possible to do these operations without read access (see tests).

Since we cannot open a write-access fd to a directory, we have to use the
/proc/self/fd trick (already used for ListXAttr) for the other operations aswell.
For simplicity, let's separate the Linux and Darwin code again (basically revert
commit f320b76fd1), and always use the
/proc/self/fd trick on Linux. On Darwin we use the best-effort approach with
openBackingFile() as a fallback.

More discussion about the available options is available in
2019-01-05 12:34:40 +01:00
Sebastian Lackner 927b3ce4cf syscallcompat: Use O_PATH to open base directory.
Also remove some unnecessary flags: When O_PATH is specified in flags, flag
bits other than O_CLOEXEC, O_DIRECTORY, and O_NOFOLLOW are ignored.
2019-01-03 18:24:05 +01:00
Jakob Unterwurzacher 4f66d66755 fusefrontend: add dirCache 2019-01-03 15:31:13 +01:00
Jakob Unterwurzacher b214be5e3f fusefrontend: xattr: fix operations on files without read permissions
* listxattr is fixed via the /proc/self/fd trick
* setxattr,removexattr are fixed by opening the file O_WRONLY

2019-01-02 20:48:46 +01:00
Jakob Unterwurzacher 7995a8358e syscallcompat: add Fgetxattr / Fsetxattr wrappers
These take care of buffer sizing and parsing.
2019-01-02 16:56:23 +01:00
Jakob Unterwurzacher 4fae240153 fusefrontend: make Readlink() symlink-safe
Now symlink-safe through Readlinkat().
2019-01-01 16:24:25 +01:00
Jakob Unterwurzacher e665df7179 syscallcompat: downgrade DT_UNKNOWN message level on XFS
Old XFS filesystems always return DT_UNKNOWN. Downgrade the message
to "info" level if we are on XFS.

Using the "warning" level means that users on old XFS filesystems
cannot run the test suite as it intentionally aborts on any

2018-11-17 17:44:21 +01:00
Jakob Unterwurzacher 57a5a8791f tests: syscallcompat: allow failure for symlinks > 1000
MacOS and old XFS versions do not support very long symlinks,
but let's not make the tests fail because of that.
2018-10-11 19:45:47 +02:00
Jakob Unterwurzacher 737a2f2012 syscallcompat: untangle Openat flag check
Check for O_NWFOLLOW and O_EXCL separately to
make the logic clearer.
2018-09-22 19:38:47 +02:00
Jakob Unterwurzacher 930c37e03d syscallcompat: use O_PATH in OpenDirNofollow
This fixes the "0100 directory" problem in reverse mode,
and should be slightly faster.
2018-09-08 18:06:33 +02:00
Jakob Unterwurzacher 9ec9d0c49c syscallcompat: untangle OpenNofollow and rename to OpenDirNofollow
The function used to do two things:

1) Walk the directory tree in a manner safe from symlink attacks
2) Open the final component in the mode requested by the caller

This change drops (2), which was only used once, and lets the caller
handle it. This simplifies the function and makes it fit for reuse in
forward mode in openBackingPath(), and for using O_PATH on Linux.
2018-09-08 17:41:17 +02:00
Jakob Unterwurzacher 658cc4aebb syscallcompat: drop Fchmodat flags
These were silently ignored until now (!) but
are rejected by Go 1.11 stdlib.

Drop the flags so the tests work again, until
we figure out a better solution.
2018-08-26 13:04:01 +02:00
Jakob Unterwurzacher c70df522d2 fusefrontend: doWrite: delete file header if first write fails
xfstests generic/083 fills the filesystem almost completely while
running fsstress in parallel. In fsck, these would show up:

  readFileID 2580: incomplete file, got 18 instead of 19 bytes

This could happen when writing the file header works, but writing
the actual data fails.

Now we kill the header again by truncating the file to zero.
2018-07-15 15:12:55 +02:00
Jakob Unterwurzacher 53f7e1a0f0 macos: fix O_DIRECT build failure
O_DIRECT has no direct equivalent on MacOS
(check out for details).

Just define it to zero there.
2018-07-04 09:04:00 +02:00
Jakob Unterwurzacher 1bab400fca Fix three golint warnings
We are clean again.

Warnings were:

internal/fusefrontend/fs.go:443:14: should omit type string from declaration
of var cTarget; it will be inferred from the right-hand side
internal/fusefrontend/xattr.go:26:1: comment on exported method FS.GetXAttr
should be of the form "GetXAttr ..."
internal/syscallcompat/sys_common.go:9:7: exported const PATH_MAX should have
comment or be unexported
2018-06-19 20:16:21 +02:00
Jakob Unterwurzacher bde7ba57b0 darwin does not have PATH_MAX
Define our own, with the value from Linux.
2018-06-08 00:47:48 +02:00
Jakob Unterwurzacher bcc8378a2c Fix the easy golint warnings
Reported by
2018-04-08 20:26:25 +02:00
Jakob Unterwurzacher 4732e33a9a macos: tests: fix deleting of scratch dir
macos rm does not understand --one-file-system,
and it cannot handle unreadable directories.
2018-03-05 23:40:08 +01:00
Jakob Unterwurzacher 7db5395c53 macos: fix second TestEmulateSymlinkat test failure 2018-03-05 21:20:07 +01:00
Jakob Unterwurzacher c5243fc79e MacOS: don't test symlinks longer than 1000 bytes
The limit is much lower than on Linux.
2018-02-28 20:40:08 +01:00
Jakob Unterwurzacher 90f2fea7fb MacOS: fix TestEmulateSymlinkat test failure
On MacOS, symlinks don't have their own permissions,
so don't check for them.
2018-02-28 20:19:31 +01:00
Jakob Unterwurzacher 8151222ada gccgo: skip emulateGetdents on linux
The test is known to fail on gccgo
(, but
getdents emulation is not used on linux, so let's skip
the test and ignore the failure.
2018-02-04 21:14:12 +01:00
Felix Lechner 5b986288cf Fix assignment error in Unix2syscall by converting Timespec into Nsec (#203)
$ go.gcc build
internal/syscallcompat/unix2syscall_linux.go:32:13: error: incompatible types in assignment (cannot use type int64 as type syscall.Timespec_sec_t)
  s.Atim.Sec = u.Atim.Sec
2018-02-03 13:42:49 +01:00
Jakob Unterwurzacher 9f8d0d8e57 gccgo: replace syscall.NAME_MAX with unix.NAME_MAX
For some reason the syscall.NAME_MAX constant does not exist
on gccgo, and it does not hurt us to use unix.NAME_MAX instead.
2018-02-01 23:50:11 +01:00
Jakob Unterwurzacher 26ba8103bf syscallcompat: switch from syscall.Getdents to unix.Getdents
On mips64le, syscall.Getdents() and struct syscall.Dirent do
not fit together, causing our Getdents implementation to
return garbage (
and ).

Switch to unix.Getdents which does not have this problem -
the next Go release with the syscall package fixes is too
far away, and will take time to trickle into distros.
2018-01-31 18:59:10 +01:00
Jakob Unterwurzacher f3838c09d8 syscallcompat: hardcode maxReclen = 280 for all architectures
Due to padding between entries, it is 280 even on 32-bit architectures.
See for details.
2018-01-25 22:22:13 +01:00
Jakob Unterwurzacher b318572312 syscallcompat: fix reversed warning output
We used to print somewhat strange messages:

	Getdents: corrupt entry #1: Reclen=276 > 280. Returning EBADR

Reported at
2018-01-25 21:42:15 +01:00
Jakob Unterwurzacher de878a3346 syscallcompat: explain why we don't use syscall.ParseDirent()
syscall.ParseDirent only returns the NAMES, we want
2018-01-25 08:43:30 +01:00
Sebastian Lackner a85dbcab38 fusefrontend: Use Linkat syscall to implement Link 2017-12-25 15:07:37 +01:00
Jakob Unterwurzacher 2ceef01afe syscallcompat: add Faccessat
Add faccessat(2) with a hack for symlink, because the
kernel does not actually looks at the passed flags.

From man 2 faccessat:

   C library/kernel differences
       The  raw faccessat() system call takes only the first three argu‐
       ments.  The AT_EACCESS and AT_SYMLINK_NOFOLLOW flags are actually
       implemented  within  the  glibc wrapper function for faccessat().
2017-12-07 00:05:28 +01:00
Jakob Unterwurzacher f97494e89b syscallcompat: add Readlinkat
We need readlinkat to implement Readlink
2017-12-06 21:07:24 +01:00
Jakob Unterwurzacher 6beb45e5b7 syscallcompat: add Darwin version of unix2syscall
The "Atim" field is called "Atimespec" on Darwin,
same for Mtim and Ctim.
2017-12-06 00:18:38 +01:00
Jakob Unterwurzacher a3bdc2bf2b fusefrontend_reverse: secure GetAttr against symlink races using the OpenNofollow helper & Fstatat.

Also introduce a helper to convert from unix.Stat_t to

Tracking ticket:
2017-12-06 00:06:31 +01:00
Jakob Unterwurzacher 03bf604fc0 syscallcompat: OpenNofollow: use O_DIRECTORY flag
...when opening intermedia directories to give us an
extra layer of safety.

From the FreeBSD man page:

     This flag can be used to prevent applications with elevated
     privileges from opening files which are even unsafe to open with O_RDONLY,
     such as device nodes.
2017-12-05 23:31:07 +01:00
Jakob Unterwurzacher e604ce6dea syscallcompat: OpenNofollow: fix relPath="" case
Sometimes want to open baseDir itself. This case
was broken, fix it.
2017-12-05 23:08:55 +01:00
Jakob Unterwurzacher 70bcf58a9b syscallcompat: convert Getdents to fd input, add emulation
Now that we have Fstatat we can use it in Getdents to
get rid of the path name.

Also, add an emulated version of getdents for MacOS. This allows
to drop the !HaveGetdents special cases from fusefrontend.

Modify the getdents test to test both native getdents and the emulated
2017-12-03 19:33:26 +01:00
Jakob Unterwurzacher e33593d30d syscallcompat: add Fstatat + emulation + test
Fstatat has recently been added to x/sys/unix. Make
it available for use in gocryptfs.
2017-12-03 19:32:59 +01:00
Jakob Unterwurzacher 91e042e2ba syscallcompat: add OpenNofollow helper
OpenNofollow = symlink-race-safe Open

Prepares fixing
2017-12-02 20:35:44 +01:00
Jakob Unterwurzacher 1d28973611 syscallcompat: move test setup into its own file
The infrastructure will also be used by the upcoming
OpenNofollow tests.
2017-12-02 19:57:23 +01:00
Jakob Unterwurzacher 77191c3485 syscallcompat: use Unlinkat and Symlinkat from x/sys/unix
I'm unsure why I did not notice this earlier, but the
syscall wrappers provided by x/sys/unix seem to do just

Drop our own version.
2017-12-02 18:36:18 +01:00
Sebastian Lackner 616a468180 syscallcompat: Improve the Openat and Mknodat syscall emulation
This avoids the conversion to an absolute path.
2017-12-01 09:41:52 +01:00
Sebastian Lackner f30522a0c1 syscallcompat: Fix syscall emulation for absolute paths
For absolute paths, the file descriptor should be ignored. In such a case
there is also no need to hold the lock or change the working directory.
2017-12-01 09:41:52 +01:00
Jakob Unterwurzacher e97c23e083 syscallcompat: check that we get NOFOLLOW wherever possible
...and fix the instances where the AT_SYMLINK_NOFOLLOW /
O_NOFOLLOW / O_EXCL flag was missing.
2017-11-30 19:40:53 +01:00
Jakob Unterwurzacher 22282aefe6 syscallcompat: add tests for emulated syscalls
Also fix the bug in emulateFchmodat that was found by the tests.
2017-11-30 19:10:21 +01:00
Jakob Unterwurzacher bd79a8cd0d syscallcompat: build emulated syscalls under linux, too
This will allow to test them under linux as well.
2017-11-30 17:07:55 +01:00
Sebastian Lackner 614745ee57 fusefrontend: allow_other: close race between mkdir and chown
Fixes the same problem as described in 72b975867a,
except for directories instead of device nodes.
2017-11-29 13:28:04 +01:00
Sebastian Lackner 0162392a28 fusefrontend: Use Fchmodat to implement Chmod 2017-11-29 12:55:41 +01:00
Sebastian Lackner 0f44c617d0 syscallcompat: Introduce unlinkat syscall with flags argument 2017-11-29 12:41:23 +01:00
Sebastian Lackner 5a56810603 fusefrontend: Use the Symlinkat syscall for longname handling 2017-11-28 09:28:06 +01:00
Sebastian Lackner 8c5069c637 syscallcompat: Fix Fchownat syscall wrapper on darwin
* Acquire the lock before reading the current directory
* Fix a file descriptor leak
2017-11-28 09:28:06 +01:00
Jakob Unterwurzacher 72b975867a fusefronted: allow_other: close race between mknod and chown
If the user manages to replace the directory with
a symlink at just the right time, we could be tricked
into chown'ing the wrong file.

This change fixes the race by using fchownat, which
unfortunately is not available on darwin, hence a compat
wrapper is added.

Scenario, as described by @slackner at :

1. Create a forward mount point with `plaintextnames` enabled
2. Mount as root user with `allow_other`
3. For testing purposes create a file `/tmp/file_owned_by_root`
   which is owned by the root user
4. As a regular user run inside of the GoCryptFS mount:

mkdir tempdir
mknod tempdir/file_owned_by_root p &
mv tempdir tempdir2
ln -s /tmp tempdir

When the steps are done fast enough and in the right order
(run in a loop!), the device file will be created in
`tempdir`, but the `lchown` will be executed by following
the symlink. As a result, the ownership of the file located
at `/tmp/file_owned_by_root` will be changed.
2017-11-27 21:04:45 +01:00
Jakob Unterwurzacher 34547a6c39 tests: don't read /proc, the number of entries changes too quickly
This could lead to test failures like this:

  --- FAIL: TestGetdents (0.02s)
  	getdents_test.go:57: len(getdentsEntries)=362, len(readdirEntries)=360
2017-10-22 18:13:08 +02:00
Jakob Unterwurzacher 538cae610c syscallcompat: Getdents: warn once if we get DT_UNKNOWN
...and if Getdents is not available at all.

Due to this warning I now know that SSHFS always returns DT_UNKNOWN:

    gocryptfs[8129]: Getdents: convertDType: received DT_UNKNOWN, falling back to Lstat

This behavoir is confirmed at

    "With sshfs, I finally found that obscure case. The dtype is always set to DT_UNKNOWN [...]"
2017-09-03 15:05:54 +02:00
Jakob Unterwurzacher e50a6a57e5 syscallcompat: implement Getdents()
The Readdir function provided by os is inherently slow because
it calls Lstat on all files.

Getdents gives us all the information we need, but does not have
a proper wrapper in the stdlib.

Implement the "Getdents()" wrapper function that calls
syscall.Getdents() and parses the returned byte blob to a
fuse.DirEntry slice.
2017-08-15 19:03:57 +02:00
Jakob Unterwurzacher a4563e21ec main, syscallcompat: use Dup3 instead of Dup2
Dup2 is not implemented on linux/arm64.

Fixes .

Also adds cross-compilation to CI.
2017-06-18 15:43:22 +02:00
Jakob Unterwurzacher 8bbc1038fe syscallcompat: OSX compat: fix variable warnings
As suggested by
2017-02-16 19:23:17 +01:00
Jakob Unterwurzacher a4956fa6bf A few more lint fixes 2016-10-04 23:30:05 +02:00
Valient Gough b764917cd5 lint fixes 2016-10-04 23:18:33 +02:00
Jakob Unterwurzacher e8a234f658 Add godoc comments to all internal packages 2016-07-06 21:51:25 +02:00
Jakob Unterwurzacher 741bf0726e syscallcompat: OSX: add Mknodat wrapper
Protip: find naked *at syscalls using:

   git grep "syscall." | grep "at(" | grep -v syscallcompat
2016-07-03 20:22:22 +02:00
Jakob Unterwurzacher d8524c7369 syscallcompat: OSX: add Unlinkat wrapper
Also, replace remaining naked syscall.Openat calls.
2016-07-03 20:17:40 +02:00
Jakob Unterwurzacher 79851bf6cc syscallcompat: OSX: add Renamat wrapper
Adds a poor man's renameat implementation for OSX.
2016-07-03 20:05:32 +02:00
Jakob Unterwurzacher 9b725c15cf syscallcompat: OSX: add Fallocate and Openat wrappers
...and convert all calls to syscall.{Fallocate,Openat}
to syscallcompat .

Both syscalls are not available on OSX. We emulate Openat and just
return EOPNOTSUPP for Fallocate.
2016-07-03 19:18:34 +02:00
Jakob Unterwurzacher c9a472c12f syscallcompat: move syscall wrapper to their own package
We will get more of them as OSX also lacks support for openat.
2016-07-03 17:51:40 +02:00