Setting/removing extended attributes on directories was partially fixed with
commit eff35e60b6. However, on most file systems
it is also possible to do these operations without read access (see tests).
Since we cannot open a write-access fd to a directory, we have to use the
/proc/self/fd trick (already used for ListXAttr) for the other operations aswell.
For simplicity, let's separate the Linux and Darwin code again (basically revert
commit f320b76fd1), and always use the
/proc/self/fd trick on Linux. On Darwin we use the best-effort approach with
openBackingFile() as a fallback.
More discussion about the available options is available in
https://github.com/rfjakob/gocryptfs/issues/308.
Give the gocryptfs process one extra millisecond to close
files. Allows us to drop several other sleeps.
UnmountErr now really returns an error when it detects an fd leak
instead of just printing a message.
An Open() a fifo blocks until it is opened for writing.
This meant that xattr operations on FIFOs would block.
Pass O_NONBLOCK to fix that, and add a test.
Retry with length 1000 if length 4000 fails, which
should work on all filesystems.
Failure was:
--- FAIL: TestTooLongSymlink (0.00s)
correctness_test.go:198: symlink xxx[...]xxxx /tmp/xfs.mnt/gocryptfs-test-parent/549823072/365091391/TooLongSymlink: file name too long
https://github.com/rfjakob/gocryptfs/issues/267
Instead of reporting the consequence:
matrix_test.go:906: modeHave 0664 != modeWant 0777
Report it if chmod itself fails, and also report the old file mode:
matrix_test.go:901: chmod 000 -> 777 failed: bad file descriptor
The cipherdir path is used as the fsname, as displayed
in "df -T". Now, having a comma in fsname triggers a sanity check
in go-fuse, aborting the mount with:
/bin/fusermount: mount failed: Invalid argument
fuse.NewServer failed: fusermount exited with code 256
Sanitize fsname by replacing any commas with underscores.
https://github.com/rfjakob/gocryptfs/issues/262
Rename openBackingPath to openBackingDir and use OpenDirNofollow
to be safe against symlink races. Note that openBackingDir is
not used in several important code paths like Create().
But it is used in Unlink, and the performance impact in the RM benchmark
to be acceptable:
Before
$ ./benchmark.bash
Testing gocryptfs at /tmp/benchmark.bash.bYO: gocryptfs v1.6-12-g930c37e-dirty; go-fuse v20170619-49-gb11e293; 2018-09-08 go1.10.3
WRITE: 262144000 bytes (262 MB, 250 MiB) copied, 1.07979 s, 243 MB/s
READ: 262144000 bytes (262 MB, 250 MiB) copied, 0.882413 s, 297 MB/s
UNTAR: 16.703
MD5: 7.606
LS: 1.349
RM: 3.237
After
$ ./benchmark.bash
Testing gocryptfs at /tmp/benchmark.bash.jK3: gocryptfs v1.6-13-g84d6faf-dirty; go-fuse v20170619-49-gb11e293; 2018-09-08 go1.10.3
WRITE: 262144000 bytes (262 MB, 250 MiB) copied, 1.06261 s, 247 MB/s
READ: 262144000 bytes (262 MB, 250 MiB) copied, 0.947228 s, 277 MB/s
UNTAR: 17.197
MD5: 7.540
LS: 1.364
RM: 3.410
If we encounter a 128KB block of zeros, try to skip to the next
data section by calling File.SeekData().
This fixes xfstests generic/285, which creates a 17TB sparse file,
and runs fsck afterwards. Without this optimization, fsck would
take ages.
TrezorPayload stores 32 random bytes used for unlocking
the master key using a Trezor security module. The randomness makes sure
that a unique unlock value is used for each gocryptfs filesystem.
When mounted via /etc/fstab like this,
/a /b fuse.gocryptfs default 0 0
we always get extra options passed. As reported by @mahkoh
at https://github.com/rfjakob/gocryptfs/pull/233 :
mount passes `-o noexec` if `-o user` is set and `-o exec` is not set.
If both `-o user` and `-o exec` are set, it passes `-o exec`.
Make these options work, and in addtion, also make -suid and -rw
work the same way.
Reported-by: @mahkoh
mv is unhappy when we return EPERM when it tries to set
system.posix_acl_access:
mv: preserving permissions for ‘b/x’: Operation not permitted
Now we return EOPNOTSUPP like tmpfs does and mv seems happy.
Values a binary-safe, there is no need to base64-encode them.
Old, base64-encoded values are supported transparently
on reading. Writing xattr values now always writes them binary.
The tests write to the example_filesystems folder, which
1) May leave your source tree in a modified state
2) Triggers test failures when the fsck tests run concurrently,
which happens on Travis CI every now and then.
Fix both problem by copying the example_filesystems folder
to a private location in /tmp.
And fix two in test_helpers.Mount().
Leftover fds can cause an unmount failure like this later:
fusermount: failed to unmount /tmp/gocryptfs-test-parent/873632270/default-plain: Device or resource busy
so try to catch them early.
Most corruption cases except xattr should be covered.
With test filesystem.
The output is still pretty ugly. xattr support will
be added in the next commits.
SwitchTestParentDir changes testParentDir. This is used when you want
to perform tests on a special filesystem. For example, the xattr tests
cannot run on tmpfs and use /var/tmp instead of /tmp.
Extracting the symlink fails with
linux-3.0/arch/microblaze/boot/dts/system.dts: Can't set permissions to 0755
so just exclude it.
The ln error Looks scary but is harmless, so get rid of it.
The symlink is only created to make it more convenient to view the
csv log.
macos does not have lazy unmount, so let's not use it
on linux either.
If the unmount fails, run "lsof" to find the open file.
Also fix the first bug we found this way.
We relied on the finalizer to close a few fds.
For some reason, this did not cause problems on Linux,
but on MacOS, it causes unmount failures:
umount(/private/tmp/gocryptfs-test-parent/194654785/default-plain): Resource busy -- try 'diskutil unmount'
To Go test logic waits for stderr and stdout to close, so
when we share it with a subprocess, it will wait for it to
exit as well.
We don't want the tests to hang when the unmount fails.
Seen on MacOS as reported at
https://github.com/rfjakob/gocryptfs/issues/213
