xfstests generic/083 fills the filesystem almost completely while
running fsstress in parallel. In fsck, these would show up:
readFileID 2580: incomplete file, got 18 instead of 19 bytes
This could happen when writing the file header works, but writing
the actual data fails.
Now we kill the header again by truncating the file to zero.
If the underlying filesystem is full, writing to gocryptfs.diriv may
fail, and later fsck show this:
OpenDir "xyz": could not read gocryptfs.diriv: wanted 16 bytes, got 0
Uncovered by xfstests generic/083.
Also fixes a fd leak in the error path.
* extend the diriv cache to 100 entries
* add special handling for the immutable root diriv
The better cache allows to shed some complexity from the path
encryption logic (parent-of-parent check).
Mitigates https://github.com/rfjakob/gocryptfs/issues/127
Dir is like filepath.Dir but returns "" instead of ".".
This was already implemented in fusefrontend_reverse as saneDir().
We will need it in nametransform for the improved diriv caching.
When a user calls into a deep directory hierarchy, we often
get a sequence like this from the kernel:
LOOKUP a
LOOKUP a/b
LOOKUP a/b/c
LOOKUP a/b/c/d
The diriv cache was not effective for this pattern, because it
was designed for this:
LOOKUP a/a
LOOKUP a/b
LOOKUP a/c
LOOKUP a/d
By also using the cached entry of the grandparent we can avoid lots
of diriv reads.
This benchmark is against a large encrypted directory hosted on NFS:
Before:
$ time ls -R nfs-backed-mount > /dev/null
real 1m35.976s
user 0m0.248s
sys 0m0.281s
After:
$ time ls -R nfs-backed-mount > /dev/null
real 1m3.670s
user 0m0.217s
sys 0m0.403s
As reported at https://github.com/rfjakob/gocryptfs/issues/105 ,
the "ioutil.WriteFile(file, iv, 0400)" call causes "permissions denied"
errors on an NFSv4 setup.
"strace"ing diriv creation and gocryptfs.conf creation shows this:
conf (works on the user's NFSv4 mount):
openat(AT_FDCWD, "/tmp/a/gocryptfs.conf.tmp", O_WRONLY|O_CREAT|O_EXCL|O_CLOEXEC, 0400) = 3
diriv (fails):
openat(AT_FDCWD, "/tmp/a/gocryptfs.diriv", O_WRONLY|O_CREAT|O_TRUNC|O_CLOEXEC, 0400) = 3
This patch creates the diriv file with the same flags that are used for
creating the conf:
openat(AT_FDCWD, "/tmp/a/gocryptfs.diriv", O_WRONLY|O_CREAT|O_EXCL|O_CLOEXEC, 0400) = 3
Closes https://github.com/rfjakob/gocryptfs/issues/105
As ReadDirIV operates on a path anyway, opening the directory
has no clear safety advantage w.r.t. concurrent renames.
If the backing directory is a reverse-mounted gocryptfs filesystem,
each directory open is an OPENDIR, and this causes a full directory
read!
This patch improves the "ls -lR" performance of an
DIR --> gocryptfs-reverse --> gocryptfs
chain by a factor of ~10.
OPENDIR counts for ls -lR:
Before 15570
After 2745
Jakob Unterwurzacher