Commit Graph

32 Commits

Author SHA1 Message Date
Jakob Unterwurzacher
f6dad8d0fa nametransform: simplify WriteDirIV to WriteDirIVAt
Un-spaghettify the function and let the callers open
the directory.
2019-01-03 15:31:13 +01:00
Jakob Unterwurzacher
2b12bba274 fusefronted: make EncryptPath symlink-safe
Finally allows us to delete EncryptPathDirIV.
2019-01-02 21:45:40 +01:00
Jakob Unterwurzacher
21f1f858b9 fusefrontend: make OpenDir() symlink-safe
Interestingly, little or no performance impact:

$ ./benchmark.bash
Testing gocryptfs at /tmp/benchmark.bash.39W: gocryptfs v1.6-42-g30c2349-dirty; go-fuse v20170619-66-g6df8ddc; 2018-11-04 go1.11
Downloading linux-3.0.tar.gz
/tmp/linux-3.0.tar.gz                  100%[=========================================================================>]  92.20M  2.93MB/s    in 31s
2018-11-04 21:44:44 URL:https://cdn.kernel.org/pub/linux/kernel/v3.0/linux-3.0.tar.gz [96675825/96675825] -> "/tmp/linux-3.0.tar.gz" [1]
WRITE: 262144000 bytes (262 MB, 250 MiB) copied, 1.1808 s, 222 MB/s
READ:  262144000 bytes (262 MB, 250 MiB) copied, 0.866438 s, 303 MB/s
UNTAR: 24.745
MD5:   12.050
LS:    3.525
RM:    9.544

Note: kernel has been updated:

$ uname -a
Linux brikett 4.18.16-200.fc28.x86_64 #1 SMP Sat Oct 20 23:53:47 UTC 2018 x86_64 x86_64 x86_64 GNU/Linux
2019-01-01 16:24:25 +01:00
Jakob Unterwurzacher
932efbd459 fusefrontend: make openBackingDir() symlink-safe
openBackingDir() used encryptPath(), which is not symlink-safe
itself. Drop encryptPath() and implement our own directory walk.

Adds three seconds to untar and two seconds to rm:

$ ./benchmark.bash
Testing gocryptfs at /tmp/benchmark.bash.MzG: gocryptfs v1.6-36-g8fb3c2f-dirty; go-fuse v20170619-66-g6df8ddc; 2018-10-14 go1.11
WRITE: 262144000 bytes (262 MB, 250 MiB) copied, 1.25078 s, 210 MB/s
READ:  262144000 bytes (262 MB, 250 MiB) copied, 1.0318 s, 254 MB/s
UNTAR: 20.941
MD5:   11.568
LS:    1.638
RM:    5.337
2019-01-01 16:24:09 +01:00
Jakob Unterwurzacher
545a03da24 nametransform: comments: directly link to ioutil.WriteFile fix
So the reader does not have to read through the whole ticket.
The commit message has a nice summary of the problem.
2019-01-01 16:23:28 +01:00
Jakob Unterwurzacher
c270b21efc fusefrontend: get rid of os.File* wrapping
Directly use int file descriptors for the dirfd
and get rid of one level of indirection.
2018-09-23 12:17:26 +02:00
Jakob Unterwurzacher
c70df522d2 fusefrontend: doWrite: delete file header if first write fails
xfstests generic/083 fills the filesystem almost completely while
running fsstress in parallel. In fsck, these would show up:

  readFileID 2580: incomplete file, got 18 instead of 19 bytes

This could happen when writing the file header works, but writing
the actual data fails.

Now we kill the header again by truncating the file to zero.
2018-07-15 15:12:55 +02:00
Jakob Unterwurzacher
bbf5b72fff WriteDirIV: delete incomplete gocryptfs.diriv file if write fails
If the underlying filesystem is full, writing to gocryptfs.diriv may
fail, and later fsck show this:

	OpenDir "xyz": could not read gocryptfs.diriv: wanted 16 bytes, got 0

Uncovered by xfstests generic/083.

Also fixes a fd leak in the error path.
2018-07-15 12:02:39 +02:00
Jakob Unterwurzacher
b6c8960b01 fsck: clean up log output
Make sure we get only 1 warning output per
problem.

Also, add new corruption types to broken_fs_v1.4.
2018-04-02 18:32:30 +02:00
Jakob Unterwurzacher
9f8d0d8e57 gccgo: replace syscall.NAME_MAX with unix.NAME_MAX
For some reason the syscall.NAME_MAX constant does not exist
on gccgo, and it does not hurt us to use unix.NAME_MAX instead.

https://github.com/rfjakob/gocryptfs/issues/201
2018-02-01 23:50:11 +01:00
Sebastian Lackner
9bcde0c09e fusefrontend: Improve documentation of mkdirWithIv and WriteDirIV
As requested in https://github.com/rfjakob/gocryptfs/pull/179
2017-12-01 09:41:52 +01:00
Jakob Unterwurzacher
e97c23e083 syscallcompat: check that we get NOFOLLOW wherever possible
...and fix the instances where the AT_SYMLINK_NOFOLLOW /
O_NOFOLLOW / O_EXCL flag was missing.
2017-11-30 19:40:53 +01:00
Sebastian Lackner
614745ee57 fusefrontend: allow_other: close race between mkdir and chown
Fixes the same problem as described in 72b975867a,
except for directories instead of device nodes.
2017-11-29 13:28:04 +01:00
Jakob Unterwurzacher
e80b5f2049 nametransform: extend diriv cache to 100 entries
* extend the diriv cache to 100 entries
* add special handling for the immutable root diriv

The better cache allows to shed some complexity from the path
encryption logic (parent-of-parent check).

Mitigates https://github.com/rfjakob/gocryptfs/issues/127
2017-08-09 22:00:53 +02:00
Jakob Unterwurzacher
75ec94a87a nametransform: add Dir() function
Dir is like filepath.Dir but returns "" instead of ".".
This was already implemented in fusefrontend_reverse as saneDir().

We will need it in nametransform for the improved diriv caching.
2017-08-06 23:14:39 +02:00
Jakob Unterwurzacher
5190cc09bb nametransform: move diriv cache into it's own package
Needs some space to grow.

renamed:    internal/nametransform/diriv_cache.go -> internal/nametransform/dirivcache/dirivcache.go
2017-08-06 21:59:15 +02:00
Jakob Unterwurzacher
32611ff97a nametransform: deduplicate code to encryptAndHashName()
This operation has been done three time by identical
sections of code. Create a function for it.
2017-08-06 21:23:42 +02:00
Jakob Unterwurzacher
9a3f9350fe nametransform: reject all-zero dirIV
This should never happen in normal operation and is a sign of
data corruption. Catch it early.
2017-05-25 14:21:55 +02:00
Jakob Unterwurzacher
245b84c887 nametransform: diriv cache: fall back to the grandparent
When a user calls into a deep directory hierarchy, we often
get a sequence like this from the kernel:

LOOKUP a
LOOKUP a/b
LOOKUP a/b/c
LOOKUP a/b/c/d

The diriv cache was not effective for this pattern, because it
was designed for this:

LOOKUP a/a
LOOKUP a/b
LOOKUP a/c
LOOKUP a/d

By also using the cached entry of the grandparent we can avoid lots
of diriv reads.

This benchmark is against a large encrypted directory hosted on NFS:

Before:

  $ time ls -R nfs-backed-mount > /dev/null
  real	1m35.976s
  user	0m0.248s
  sys	0m0.281s

After:

  $ time ls -R nfs-backed-mount > /dev/null
  real	1m3.670s
  user	0m0.217s
  sys 	0m0.403s
2017-05-22 22:36:54 +02:00
Jakob Unterwurzacher
7d38f80a78 nametransform: WriteDirIV: replace ioutil.WriteFile
As reported at https://github.com/rfjakob/gocryptfs/issues/105 ,
the "ioutil.WriteFile(file, iv, 0400)" call causes "permissions denied"
errors on an NFSv4 setup.

"strace"ing diriv creation and gocryptfs.conf creation shows this:

conf (works on the user's NFSv4 mount):
openat(AT_FDCWD, "/tmp/a/gocryptfs.conf.tmp", O_WRONLY|O_CREAT|O_EXCL|O_CLOEXEC, 0400) = 3

diriv (fails):
openat(AT_FDCWD, "/tmp/a/gocryptfs.diriv", O_WRONLY|O_CREAT|O_TRUNC|O_CLOEXEC, 0400) = 3

This patch creates the diriv file with the same flags that are used for
creating the conf:
openat(AT_FDCWD, "/tmp/a/gocryptfs.diriv", O_WRONLY|O_CREAT|O_EXCL|O_CLOEXEC, 0400) = 3

Closes https://github.com/rfjakob/gocryptfs/issues/105
2017-04-29 14:15:13 +02:00
Jakob Unterwurzacher
5b54577d2e nametransform: fix Raw64 not affecting longnames
HashLongName() incorrectly hardcoded the call to base64.URLEncoding.
2017-03-05 22:25:41 +01:00
Jakob Unterwurzacher
df28fc5a11 nametransform: get rid of leading "./"
Paths in the root directory were encrypted to this:

    foobar -> ./N9vPc0gXUY4PDSt0-muYXQ==
2016-11-09 23:41:47 +01:00
Jakob Unterwurzacher
df1e3a10c4 nametransform: nicer error message on empty gocryptfs.diriv
Old:

	Nov 06 13:34:38 brikett gocryptfs[16228]: ReadDirIVAt: Read failed: EOF
	Nov 06 13:34:38 brikett gocryptfs[16228]: go-fuse: can't convert error type: EOF

New:

	Nov 06 14:08:43 brikett gocryptfs[17361]: ReadDirIVAt: wanted 16 bytes, got 0. Returning EINVAL.
2016-11-06 14:09:34 +01:00
Jakob Unterwurzacher
a9c7565b80 fusefrontend: refactor dirIVCache
Simplify the code a bit.
2016-11-01 10:34:41 +01:00
Jakob Unterwurzacher
aeda9721d0 Fix misspellings
Close https://github.com/rfjakob/gocryptfs/issues/54
2016-10-24 19:18:13 +02:00
Jakob Unterwurzacher
53257f4ee5 nametransform: better error code on invalid diriv length
go-fuse translates errors unknown to it into "function not
implemented", which is wrong in this case.
2016-10-07 22:40:30 +02:00
Valient Gough
b764917cd5 lint fixes 2016-10-04 23:18:33 +02:00
Jakob Unterwurzacher
c7b3150afc nametransform: delete unused function DecryptPathDirIV 2016-09-25 18:56:23 +02:00
Jakob Unterwurzacher
b34a665da4 nametransform: implement ReadDirIV without Openat
As ReadDirIV operates on a path anyway, opening the directory
has no clear safety advantage w.r.t. concurrent renames.

If the backing directory is a reverse-mounted gocryptfs filesystem,
each directory open is an OPENDIR, and this causes a full directory
read!

This patch improves the "ls -lR" performance of an

  DIR --> gocryptfs-reverse --> gocryptfs

chain by a factor of ~10.

OPENDIR counts for ls -lR:
  Before 15570
  After   2745
2016-09-25 16:43:17 +02:00
Jakob Unterwurzacher
ac1221395e nametransform: export DirIVLen constant
Will be needed by reverse mode.
2016-09-25 16:43:17 +02:00
Jakob Unterwurzacher
d8524c7369 syscallcompat: OSX: add Unlinkat wrapper
Also, replace remaining naked syscall.Openat calls.
2016-07-03 20:17:40 +02:00
Jakob Unterwurzacher
7e92ebe16a Rename nametransform, contentenc source files
Let's have shorter names, and merge *_api.go into the "main"
file.

No code changes.
2016-06-16 19:02:47 +02:00