Commit Graph

1038 Commits

Author SHA1 Message Date
Jakob Unterwurzacher
a24faa3ba5 fusefrontend: write: consolidate and move encryption to contentenc
Collect all the plaintext and pass everything to contentenc in
one call.

This will allow easier parallization of the encryption.

https://github.com/rfjakob/gocryptfs/issues/116
2017-06-01 22:19:27 +02:00
Jakob Unterwurzacher
53b7c17261 Don't cap GOMAXPROCS at 4.
Before Go 1.5, GOMAXPROCS defaulted to 1, hence it made
sense to unconditionally increase it to 4.

But since Go 1.5, GOMAXPROCS defaults to the number of cores,
so don't keep it from increasing above 4.

Also, update the performance numbers.
2017-06-01 20:55:13 +02:00
Jakob Unterwurzacher
f44902aaae Fix two comments
One out-of-date and the other with a typo.
2017-06-01 18:53:57 +02:00
Jakob Unterwurzacher
f91ce0b004 main: increase max write size to maximum
Previously, it was at the go-fuse default of 64KiB. Getting
bigger writes should increase throughput somewhat.

Testing on tmpfs shows an improvement from 112MiB/s to 120MiB/s.
2017-06-01 18:52:02 +02:00
Charles Duffy
cf1ded5236 Implement force_owner option to display ownership as a specific user. 2017-06-01 00:26:17 +02:00
Jakob Unterwurzacher
fc2a5f5ab0 pathiv: fix test failure on Go 1.6
Travis failed on Go 1.6.3 with this error:

	internal/pathiv/pathiv_test.go:20: no args in Error call

This change should solve the problem and provides a better error
message on (real) test failure.
2017-05-31 08:21:36 +02:00
Jakob Unterwurzacher
1e598e96fc main: add "-info" option
Pretty-prints the config while stripping out sensitive
(and uninteresting) data

https://github.com/rfjakob/gocryptfs/issues/111
2017-05-30 19:01:32 +02:00
Jakob Unterwurzacher
ca5f469ea3 README: update changelog with short help text 2017-05-30 18:01:03 +02:00
Jakob Unterwurzacher
df2f4b1c40 main: add short help text
We have accumulated so many options over time that they
no longer fit on the screen.

Display only a useful subset of options to the user unless
they pass "-hh".
2017-05-30 17:59:13 +02:00
Jakob Unterwurzacher
9a217ce786 pathiv: move block IV algorithm into this package
This was implemented in fusefrontend_reverse, but we need it
in fusefrontend as well. Move the algorithm into pathiv.BlockIV().
2017-05-30 17:04:46 +02:00
Jakob Unterwurzacher
e43eb36da3 tests: add v1.3-reverse example filesystem
We check the md5 sum of the encrypted version of a file to make sure we don't
accidentially change the ciphertext generation.
2017-05-30 17:04:46 +02:00
Jakob Unterwurzacher
d202a456f5 pathiv: move derivedIVContainer into the package
...under the new name "FileIVs".

This will also be used by forward mode.
2017-05-30 17:04:46 +02:00
Jakob Unterwurzacher
857507e8b1 fusefrontend_reverse: move pathiv to its own package
We will also need it in forward mode.
2017-05-30 17:04:46 +02:00
Jakob Unterwurzacher
4d2cc551cf package.bash: RHEL: strip brackets
We now convert "CentOS release 5.11 (Final)"
to             "CentOS_release_5.11_Final"

https://github.com/rfjakob/gocryptfs/issues/113
2017-05-29 08:42:01 +02:00
Jakob Unterwurzacher
d59e7da6a6 gocryptfs-xray: dumpmasterkey: disable "Reading password from stdin"
...and also exit with the proper exit code when we get an error.
2017-05-28 17:43:09 +02:00
Jakob Unterwurzacher
7f5ae5f843 gocryptfs-xray: add function to dump the master key
Fixes https://github.com/rfjakob/gocryptfs/issues/83
2017-05-28 17:11:50 +02:00
Jakob Unterwurzacher
c36a55e985 package.bash: make it work somewhat on RHEL
RHEL and CentOS do not have /etc/os-release yet. Read from
/etc/redhat-release instead.

Fixes https://github.com/rfjakob/gocryptfs/issues/113
2017-05-27 15:22:27 +02:00
Jakob Unterwurzacher
d6ef283c3f cryptocore: improve comments and add tests for hkdfDerive
These should make it easier to re-implement the key derivation
that was enabled with the "HKDF" feature flag.
2017-05-27 14:41:20 +02:00
Jakob Unterwurzacher
ce4aaf16d8 README: update changelog 2017-05-25 21:36:49 +02:00
Jakob Unterwurzacher
9ecf2d1a3f fusefrontend_reverse: store derived values for hard-linked files
With hard links, the path to a file is not unique. This means
that the ciphertext data depends on the path that is used to access
the files.

Fix that by storing the derived values when we encounter a hard-linked
file. This means that the first path wins.
2017-05-25 21:33:16 +02:00
Jakob Unterwurzacher
bfc8d47747 doc: add performance numbers for 1.3-27 2017-05-25 21:30:58 +02:00
Jakob Unterwurzacher
9a3f9350fe nametransform: reject all-zero dirIV
This should never happen in normal operation and is a sign of
data corruption. Catch it early.
2017-05-25 14:21:55 +02:00
Jakob Unterwurzacher
2ce269ec63 contenenc: reject all-zero file ID
This should never happen in normal operation and is a sign of
data corruption. Catch it early.
2017-05-25 14:20:27 +02:00
Jakob Unterwurzacher
c0e411f81d contentenc: better error reporting in ParseHeader
Log the message ourselves and return EINVAL.

Before:

	gocryptfs[26962]: go-fuse: can't convert error type: ParseHeader: invalid version: got 0, want 2

After:

	gocryptfs[617]: ParseHeader: invalid version: want 2, got 0. Returning EINVAL.
2017-05-25 14:18:44 +02:00
Jakob Unterwurzacher
70c16fde4c main: replace paniclog with logger(1)
Instead of redirecting stdout and stderr to /tmp/gocryptfs_paniclog,
where it is hard to find, redirect them to a newly spawned logger(1)
instance that forwards the messages to syslog.

See https://github.com/rfjakob/gocryptfs/issues/109 for an example
where the paniclog was lost due to a reboot.

Also, instead of closing stdin, redirect it to /dev/null, like most
daemons seem to do.
2017-05-24 19:57:45 +02:00
Jakob Unterwurzacher
e827763f2e nametransform: harden name decryption against invalid input
This fixes a few issues I have found reviewing the code:

1) Limit the amount of data ReadLongName() will read. Previously,
you could send gocryptfs into out-of-memory by symlinking
gocryptfs.diriv to /dev/zero.

2) Handle the empty input case in unPad16() by returning an
error. Previously, it would panic with an out-of-bounds array
read. It is unclear to me if this could actually be triggered.

3) Reject empty names after base64-decoding in DecryptName().
An empty name crashes emeCipher.Decrypt().
It is unclear to me if B64.DecodeString() can actually return
a non-error empty result, but let's guard against it anyway.
2017-05-23 21:26:38 +02:00
Jakob Unterwurzacher
508fd9e1d6 main: downgrade panic log create failure from fatal error to warning
Exiting with a fatal error just pushes users to use "-nosyslog",
which is even worse than not having a paniclog.
2017-05-23 18:01:21 +02:00
Jakob Unterwurzacher
245b84c887 nametransform: diriv cache: fall back to the grandparent
When a user calls into a deep directory hierarchy, we often
get a sequence like this from the kernel:

LOOKUP a
LOOKUP a/b
LOOKUP a/b/c
LOOKUP a/b/c/d

The diriv cache was not effective for this pattern, because it
was designed for this:

LOOKUP a/a
LOOKUP a/b
LOOKUP a/c
LOOKUP a/d

By also using the cached entry of the grandparent we can avoid lots
of diriv reads.

This benchmark is against a large encrypted directory hosted on NFS:

Before:

  $ time ls -R nfs-backed-mount > /dev/null
  real	1m35.976s
  user	0m0.248s
  sys	0m0.281s

After:

  $ time ls -R nfs-backed-mount > /dev/null
  real	1m3.670s
  user	0m0.217s
  sys 	0m0.403s
2017-05-22 22:36:54 +02:00
Jakob Unterwurzacher
5672e41f5a README: v1.3.1 will be called v1.4 2017-05-22 21:42:47 +02:00
Jakob Unterwurzacher
f5d6645a67 README: start changelog for v1.3.1 2017-05-15 00:11:33 +02:00
Jakob Unterwurzacher
86c06fc172 doc: CLI_ABI.md: fix exitcodes.go link
Needs a leading "..".
2017-05-14 14:35:46 +02:00
Jakob Unterwurzacher
22bac16201 doc: update CLI_ABI.md with new exit codes
All exit codes that are likely to occour are listed.
2017-05-14 14:34:50 +02:00
Jakob Unterwurzacher
c44389d942 exitcodes: specific codes for failure to read or write gocryptfs.conf
New codes:
* OpenConf = 23
* WriteConf = 24
2017-05-14 14:30:50 +02:00
Jakob Unterwurzacher
2aea2d3d62 exitcodes: add code 22 for "password is empty"
Empty passwords are not allowed. Let's give the error
it's own exit code.
2017-05-14 14:02:08 +02:00
Jakob Unterwurzacher
8aabc54276 exitcodes: get rid of generic "Mount" exit code
Instead, create three new specific exit codes:
* FuseNewServer = 19
* CtlSock = 20
* PanicLogCreate = 21
2017-05-14 13:51:26 +02:00
Jakob Unterwurzacher
18f354d84b main: password change: exit with code 12 on wrong password
We used to return code 8, now we return code 12 as documented in
the man page.

Also adds a test.
2017-05-14 13:14:00 +02:00
Jakob Unterwurzacher
d5adde1eeb exitcodes: pull all exit code definitions into the package
This commit defines all exit codes in one place in the exitcodes
package.

Also, it adds a test to verify the exit code on incorrect
password, which is what SiriKali cares about the most.

Fixes https://github.com/rfjakob/gocryptfs/issues/77 .
2017-05-07 22:16:22 +02:00
Jakob Unterwurzacher
ad7942f434 fusefrontend: implement path decryption via ctlsock
Closes https://github.com/rfjakob/gocryptfs/issues/84 .
2017-05-07 21:01:39 +02:00
Jakob Unterwurzacher
26881538e1 nametranform, fusefrontend: better errors on invalid names
nametransform.DecryptName() now always returns syscall.EBADMSG if
the name was invalid.

fusefrontend.OpenDir error messages have been normalized.
2017-05-07 20:58:27 +02:00
Jakob Unterwurzacher
68387b470c Fix typos found by Misspell
Misspell Finds commonly misspelled English words
gocryptfs/internal/configfile/scrypt.go
Line 41: warning: "paramter" is a misspelling of "parameter" (misspell)
gocryptfs/internal/ctlsock/ctlsock_serve.go
Line 1: warning: "implementes" is a misspelling of "implements" (misspell)
gocryptfs/tests/test_helpers/helpers.go
Line 27: warning: "compatability" is a misspelling of "compatibility" (misspell)
2017-05-07 12:22:15 +02:00
Jakob Unterwurzacher
cd10d0a6e4 main: more specific default values for GitVersion etc
Now looks like this:

  $ ./gocryptfs -version
  gocryptfs [GitVersion not set - please compile using ./build.bash]; go-fuse [GitVersionFuse not set - please compile using ./build.bash]; 0000-00-00 go1.8

Hopefully easier to grep for.
2017-05-06 14:29:34 +02:00
Jakob Unterwurzacher
a48893b653 main: add "-race" to the version string
...if we were compiled with "-race".
2017-05-06 14:26:34 +02:00
Jakob Unterwurzacher
c52e1abc58 fusefrontend: log "too many open files" errors
This usually indicates that the open file limit for gocryptfs is
too low. We should report this to the user.
2017-05-03 23:46:52 +02:00
Jakob Unterwurzacher
b32fc212af main: RLIMIT_NOFILE: cap to 4096
We don't want to set the limit to unlimited if "Max" is unlimited.
Hard-cap to 4096.
2017-05-03 08:59:21 +02:00
Jakob Unterwurzacher
2d43288bc8 main: increase open file limit to 4096
Linux by default has a soft limit of 1024 and a hard limit of
4096 on open files. We can increase it so 4096 without root
permissions.

This should help reduce the risk of gocryptfs running out of
file descriptors, as reported at
https://github.com/rfjakob/gocryptfs/issues/82 .
2017-05-02 23:15:18 +02:00
Jakob Unterwurzacher
daada9d7c6 tests: support encfs in the fsstress script
May be helpful for https://github.com/vgough/encfs/issues/214 .
2017-05-01 23:50:49 +02:00
Jakob Unterwurzacher
fb3cc6ea40 openfiletable: rename WriteLock to ContentLock
...and IDLock to HeaderLock. This matches what the locks actually
protect.
2017-05-01 21:57:18 +02:00
Jakob Unterwurzacher
f322ee87e3 fusefrontend: rely on nodefs.defaultFile for no-op functions
Now that we embed nodefs.NewDefaultFile(), we can drop our own
no-ops.
2017-05-01 19:12:37 +02:00
Jakob Unterwurzacher
1a89919d80 contentenc: downgrade "interrupted write?" warning to debug
This can happen during normal operation, and is harmless since

14038a1644
"fusefrontend: readFileID: reject files that consist only of a header"

causes dormant header-only files to be rewritten on the next write.
2017-05-01 18:44:18 +02:00
Jakob Unterwurzacher
9768376bd1 tests: add TestXfs124
This test reproduces the problem xfstests generic/124 uncovered.
The warning itself is harmless, but we should either (1) add locking
so that this cannot happen anymore or (2) drop the warning.

Currently fails:

$ go test -v
=== RUN   Test1980Tar
--- PASS: Test1980Tar (0.00s)
=== RUN   TestCtlSock
--- PASS: TestCtlSock (0.10s)
=== RUN   TestOpenTruncateRead
--- PASS: TestOpenTruncateRead (0.00s)
=== RUN   TestWORead
--- PASS: TestWORead (0.00s)
=== RUN   TestXfs124
cipherSize 18 == header size: interrupted write?
-wpanic turns this warning into a panic: cipherSize 18 == header size: interrupted write?
2017-05-01 18:44:03 +02:00