Commit Graph

45 Commits

Author SHA1 Message Date
Jakob Unterwurzacher
9a3f9350fe nametransform: reject all-zero dirIV
This should never happen in normal operation and is a sign of
data corruption. Catch it early.
2017-05-25 14:21:55 +02:00
Jakob Unterwurzacher
e827763f2e nametransform: harden name decryption against invalid input
This fixes a few issues I have found reviewing the code:

1) Limit the amount of data ReadLongName() will read. Previously,
you could send gocryptfs into out-of-memory by symlinking
gocryptfs.diriv to /dev/zero.

2) Handle the empty input case in unPad16() by returning an
error. Previously, it would panic with an out-of-bounds array
read. It is unclear to me if this could actually be triggered.

3) Reject empty names after base64-decoding in DecryptName().
An empty name crashes emeCipher.Decrypt().
It is unclear to me if B64.DecodeString() can actually return
a non-error empty result, but let's guard against it anyway.
2017-05-23 21:26:38 +02:00
Jakob Unterwurzacher
245b84c887 nametransform: diriv cache: fall back to the grandparent
When a user calls into a deep directory hierarchy, we often
get a sequence like this from the kernel:

LOOKUP a
LOOKUP a/b
LOOKUP a/b/c
LOOKUP a/b/c/d

The diriv cache was not effective for this pattern, because it
was designed for this:

LOOKUP a/a
LOOKUP a/b
LOOKUP a/c
LOOKUP a/d

By also using the cached entry of the grandparent we can avoid lots
of diriv reads.

This benchmark is against a large encrypted directory hosted on NFS:

Before:

  $ time ls -R nfs-backed-mount > /dev/null
  real	1m35.976s
  user	0m0.248s
  sys	0m0.281s

After:

  $ time ls -R nfs-backed-mount > /dev/null
  real	1m3.670s
  user	0m0.217s
  sys 	0m0.403s
2017-05-22 22:36:54 +02:00
Jakob Unterwurzacher
26881538e1 nametranform, fusefrontend: better errors on invalid names
nametransform.DecryptName() now always returns syscall.EBADMSG if
the name was invalid.

fusefrontend.OpenDir error messages have been normalized.
2017-05-07 20:58:27 +02:00
Jakob Unterwurzacher
7d38f80a78 nametransform: WriteDirIV: replace ioutil.WriteFile
As reported at https://github.com/rfjakob/gocryptfs/issues/105 ,
the "ioutil.WriteFile(file, iv, 0400)" call causes "permissions denied"
errors on an NFSv4 setup.

"strace"ing diriv creation and gocryptfs.conf creation shows this:

conf (works on the user's NFSv4 mount):
openat(AT_FDCWD, "/tmp/a/gocryptfs.conf.tmp", O_WRONLY|O_CREAT|O_EXCL|O_CLOEXEC, 0400) = 3

diriv (fails):
openat(AT_FDCWD, "/tmp/a/gocryptfs.diriv", O_WRONLY|O_CREAT|O_TRUNC|O_CLOEXEC, 0400) = 3

This patch creates the diriv file with the same flags that are used for
creating the conf:
openat(AT_FDCWD, "/tmp/a/gocryptfs.diriv", O_WRONLY|O_CREAT|O_EXCL|O_CLOEXEC, 0400) = 3

Closes https://github.com/rfjakob/gocryptfs/issues/105
2017-04-29 14:15:13 +02:00
Jakob Unterwurzacher
445b5019e3 nametransform: fix Raw64 not affecting symlink targets
The symlink functions incorrectly hardcoded the padded
base64 variant.
2017-03-05 22:59:25 +01:00
Jakob Unterwurzacher
5b54577d2e nametransform: fix Raw64 not affecting longnames
HashLongName() incorrectly hardcoded the call to base64.URLEncoding.
2017-03-05 22:25:41 +01:00
Jakob Unterwurzacher
966308eeb7 Drop Go 1.4 compatability code everywhere
Yields a nice reduction in code size.
2017-03-05 17:44:14 +01:00
Jakob Unterwurzacher
e032539e2c cryptocore: use eme v1.1 interface
Version 1.1 of the EME package (github.com/rfjakob/eme) added
a more convenient interface. Use it.

Note that you have to upgrade your EME package (go get -u)!
2017-03-05 13:58:24 +01:00
Jakob Unterwurzacher
c9f4400e6d Replace all calls to naked panic() with log.Panic()
We want all panics to show up in the syslog.
2016-12-10 11:54:36 +01:00
Jakob Unterwurzacher
e3c5e3f1c8 fusefronted: preserve owner for device nodes and sockets
https://github.com/rfjakob/gocryptfs/issues/64
2016-11-28 23:09:47 +01:00
Jakob Unterwurzacher
e7f57695a6 Fix golint warnings
$ golint ./... | grep -v underscore | grep -v ALL_CAPS
internal/fusefrontend_reverse/rfs.go:52:36: exported func NewFS returns unexported type *fusefrontend_reverse.reverseFS, which can be annoying to use
internal/nametransform/raw64_go1.5.go:10:2: exported const HaveRaw64 should have comment (or a comment on this block) or be unexported
2016-11-10 00:38:01 +01:00
Jakob Unterwurzacher
df28fc5a11 nametransform: get rid of leading "./"
Paths in the root directory were encrypted to this:

    foobar -> ./N9vPc0gXUY4PDSt0-muYXQ==
2016-11-09 23:41:47 +01:00
Jakob Unterwurzacher
df1e3a10c4 nametransform: nicer error message on empty gocryptfs.diriv
Old:

	Nov 06 13:34:38 brikett gocryptfs[16228]: ReadDirIVAt: Read failed: EOF
	Nov 06 13:34:38 brikett gocryptfs[16228]: go-fuse: can't convert error type: EOF

New:

	Nov 06 14:08:43 brikett gocryptfs[17361]: ReadDirIVAt: wanted 16 bytes, got 0. Returning EINVAL.
2016-11-06 14:09:34 +01:00
Jakob Unterwurzacher
d15122d3d6 Add Go 1.4 compatibility layer for raw64
Using raw64 will not work, but at least it will compile.
2016-11-01 19:25:59 +01:00
Jakob Unterwurzacher
2b991c9743 Add support for unpadded base64 filenames, "-raw64"
Through base64.RawURLEncoding.

New command-line parameter "-raw64".
2016-11-01 18:43:22 +01:00
Jakob Unterwurzacher
944eaf2fb5 fusefronted: expire dir IV cache after one second
The Back In Time backup tool (https://github.com/bit-team/backintime)
wants to write directly into the ciphertext dir.
This may cause the cached directory IV to become out-of-date.
Having an expiry time limits the inconstency to one second, like
attr_timeout does for the kernel getattr cache.
2016-11-01 10:57:39 +01:00
Jakob Unterwurzacher
a9c7565b80 fusefrontend: refactor dirIVCache
Simplify the code a bit.
2016-11-01 10:34:41 +01:00
Jakob Unterwurzacher
aeda9721d0 Fix misspellings
Close https://github.com/rfjakob/gocryptfs/issues/54
2016-10-24 19:18:13 +02:00
Jakob Unterwurzacher
53257f4ee5 nametransform: better error code on invalid diriv length
go-fuse translates errors unknown to it into "function not
implemented", which is wrong in this case.
2016-10-07 22:40:30 +02:00
Valient Gough
b764917cd5 lint fixes 2016-10-04 23:18:33 +02:00
Jakob Unterwurzacher
28838a489d nametransform: downgrade invalid length warning
This happens all the time in reverse mode when somebody stats
an encrypted symlink target.
2016-09-25 19:50:16 +02:00
Jakob Unterwurzacher
c7b3150afc nametransform: delete unused function DecryptPathDirIV 2016-09-25 18:56:23 +02:00
Jakob Unterwurzacher
b883dd10a6 reverse: add symlink encryption and Readlink support 2016-09-25 18:01:24 +02:00
Jakob Unterwurzacher
a6a7b424f8 reverse: resolve long names in Open and GetAttr
The last patch added functionality for generating gocryptfs.longname.*
files, this patch adds support for mapping them back to the full
filenames.

Note that resolving a long name needs a full readdir. A cache
will be implemented later on to improve performance.
2016-09-25 16:43:17 +02:00
Jakob Unterwurzacher
b34a665da4 nametransform: implement ReadDirIV without Openat
As ReadDirIV operates on a path anyway, opening the directory
has no clear safety advantage w.r.t. concurrent renames.

If the backing directory is a reverse-mounted gocryptfs filesystem,
each directory open is an OPENDIR, and this causes a full directory
read!

This patch improves the "ls -lR" performance of an

  DIR --> gocryptfs-reverse --> gocryptfs

chain by a factor of ~10.

OPENDIR counts for ls -lR:
  Before 15570
  After   2745
2016-09-25 16:43:17 +02:00
Jakob Unterwurzacher
01c18dbcab nametransform: return EINVAL on invalid padding
With the generic fmt.Errorf we trigger a warning from go-fuse:

  2016/09/21 21:42:31 can't convert error type: Invalid padding
2016-09-25 16:43:17 +02:00
Jakob Unterwurzacher
ac1221395e nametransform: export DirIVLen constant
Will be needed by reverse mode.
2016-09-25 16:43:17 +02:00
Jakob Unterwurzacher
e8a234f658 Add godoc comments to all internal packages 2016-07-06 21:51:25 +02:00
Jakob Unterwurzacher
d8524c7369 syscallcompat: OSX: add Unlinkat wrapper
Also, replace remaining naked syscall.Openat calls.
2016-07-03 20:17:40 +02:00
Jakob Unterwurzacher
e574a6cc1f nametransform: hide detailed padding error behind the debug flag
unPad16 returns detailed errors including the position of the
incorrect bytes. Kill a possible padding oracle by lumping
everything into a generic error.

The detailed error is only logged if debug is active.
2016-07-03 15:35:58 +02:00
Jakob Unterwurzacher
3d59a72ba9 Drop deprecated "-emenames" option
The EMENames feature flag is already mandatory, dropping the command
line option is the final step.
2016-06-23 21:56:50 +02:00
Jakob Unterwurzacher
e970b1fdb5 nametransform: drop unused noiv functions
As DirIV is now mandatory there is no user for the noiv functions.
2016-06-23 21:39:04 +02:00
Jakob Unterwurzacher
7e92ebe16a Rename nametransform, contentenc source files
Let's have shorter names, and merge *_api.go into the "main"
file.

No code changes.
2016-06-16 19:02:47 +02:00
Jakob Unterwurzacher
6c3f97399a Rename internal "toggledlog" package to "tlog"
tlog is used heavily everywhere and deserves a shorter name.

Renamed using sed magic, without any manual rework:

   find * -type f -exec sed -i 's/toggledlog/tlog/g' {} +
2016-06-15 23:30:44 +02:00
Jakob Unterwurzacher
80b027f830 nametransform, main: better error messages 2016-06-01 20:07:43 +02:00
Jakob Unterwurzacher
db72fcea41 longnames: fix fsstress failure, use dirfd
Using dirfd-relative operations allows safe lockless handling
of the ".name" files.
2016-04-10 21:31:08 +02:00
Jakob Unterwurzacher
63d3e51734 longnames: use symbolic constants instead of naked ints 2016-04-10 12:36:43 +02:00
Jakob Unterwurzacher
37a9b4c3ee Enable openssl in tests to support old Go versions
Go 1.4 and older do not support 128-bit IVs which caused
the tests to panic.
2016-03-03 00:59:58 +01:00
Jakob Unterwurzacher
653d4a619c longnames part II: Rename, Unlink, Rmdir, Mknod, Mkdir + tests 2016-02-07 14:02:09 +01:00
Jakob Unterwurzacher
e111e20649 longnames part I: Create and OpenDir work with long filenames > 176 bytes
Todo: Rename, Unlink, Rmdir, Mknod, Mkdir
2016-02-06 22:54:14 +01:00
Jakob Unterwurzacher
6b27b6a1e7 Move dirIVCache to its own file 2016-02-06 20:27:28 +01:00
Jakob Unterwurzacher
c74772bc8d Run go fmt 2016-02-06 20:23:36 +01:00
Jakob Unterwurzacher
b0ee5258b1 Fix tests - were broken by the refactoring 2016-02-06 20:22:45 +01:00
Jakob Unterwurzacher
2b8cbd9441 Major refactoring: Split up "cryptfs" into several internal packages
"git status" for reference:

deleted:    cryptfs/cryptfs.go
deleted:    cryptfs/names_core.go
modified:   integration_tests/cli_test.go
modified:   integration_tests/helpers.go
renamed:    cryptfs/config_file.go -> internal/configfile/config_file.go
renamed:    cryptfs/config_test.go -> internal/configfile/config_test.go
renamed:    cryptfs/config_test/.gitignore -> internal/configfile/config_test/.gitignore
renamed:    cryptfs/config_test/PlaintextNames.conf -> internal/configfile/config_test/PlaintextNames.conf
renamed:    cryptfs/config_test/StrangeFeature.conf -> internal/configfile/config_test/StrangeFeature.conf
renamed:    cryptfs/config_test/v1.conf -> internal/configfile/config_test/v1.conf
renamed:    cryptfs/config_test/v2.conf -> internal/configfile/config_test/v2.conf
renamed:    cryptfs/kdf.go -> internal/configfile/kdf.go
renamed:    cryptfs/kdf_test.go -> internal/configfile/kdf_test.go
renamed:    cryptfs/cryptfs_content.go -> internal/contentenc/content.go
new file:   internal/contentenc/content_api.go
renamed:    cryptfs/content_test.go -> internal/contentenc/content_test.go
renamed:    cryptfs/file_header.go -> internal/contentenc/file_header.go
renamed:    cryptfs/intrablock.go -> internal/contentenc/intrablock.go
renamed:    cryptfs/address_translation.go -> internal/contentenc/offsets.go
new file:   internal/cryptocore/crypto_api.go
renamed:    cryptfs/gcm_go1.4.go -> internal/cryptocore/gcm_go1.4.go
renamed:    cryptfs/gcm_go1.5.go -> internal/cryptocore/gcm_go1.5.go
renamed:    cryptfs/nonce.go -> internal/cryptocore/nonce.go
renamed:    cryptfs/openssl_aead.go -> internal/cryptocore/openssl_aead.go
renamed:    cryptfs/openssl_benchmark.bash -> internal/cryptocore/openssl_benchmark.bash
renamed:    cryptfs/openssl_test.go -> internal/cryptocore/openssl_test.go
new file:   internal/nametransform/name_api.go
new file:   internal/nametransform/names_core.go
renamed:    cryptfs/names_diriv.go -> internal/nametransform/names_diriv.go
renamed:    cryptfs/names_noiv.go -> internal/nametransform/names_noiv.go
renamed:    cryptfs/names_test.go -> internal/nametransform/names_test.go
new file:   internal/nametransform/pad16.go
renamed:    cryptfs/log.go -> internal/toggledlog/log.go
renamed:    cryptfs/log_go1.4.go -> internal/toggledlog/log_go1.4.go
renamed:    cryptfs/log_go1.5.go -> internal/toggledlog/log_go1.5.go
modified:   main.go
modified:   masterkey.go
modified:   pathfs_frontend/file.go
modified:   pathfs_frontend/file_holes.go
modified:   pathfs_frontend/fs.go
modified:   pathfs_frontend/fs_dir.go
modified:   pathfs_frontend/names.go
modified:   test.bash
2016-02-06 19:22:35 +01:00