Commit Graph

1200 Commits

Author SHA1 Message Date
Sebastian Lackner c547673529 nametransform: Return error if decrypted name is '.' or '..' 2017-11-22 23:42:08 +01:00
Sebastian Lackner f3c777d5ea main: Add '-devrandom' commandline option
Allows to use /dev/random for generating the master key instead of the
default Go implementation. When the kernel random generator has been
properly initialized both are considered equally secure, however:

* Versions of Go prior to 1.9 just fall back to /dev/urandom if the
  getrandom() syscall would be blocking (Go Bug #19274)

* Kernel versions prior to 3.17 do not support getrandom(), and there
  is no check if the random generator has been properly initialized
  before reading from /dev/urandom

This is especially useful for embedded hardware with low-entroy. Please
note that generation of the master key might block indefinitely if the
kernel cannot harvest enough entropy.
2017-11-21 23:37:06 +01:00
Jakob Unterwurzacher 1b0426bcb2 main: print clear error message if CIPHERDIR is missing
Getting just the help text in response to

	gocryptfs -info -config external.config

is confusing: https://github.com/rfjakob/gocryptfs/issues/157
2017-11-15 20:30:21 +01:00
Jakob Unterwurzacher e36a0ebf18 main: add "-sharedstorage" flag
At the moment, it does two things:

1. Disable stat() caching so changes to the backing storage show up
   immediately.
2. Disable hard link tracking, as the inode numbers on the backing
   storage are not stable when files are deleted and re-created behind
   our back. This would otherwise produce strange "file does not exist"
   and other errors.

Mitigates https://github.com/rfjakob/gocryptfs/issues/156
2017-11-12 20:06:13 +01:00
Jakob Unterwurzacher 9ab6cdb9b9 test.bash: don't run "go tool vet" if vendor dir exists
...this fails in a thousand ways:

[...]
vendor/golang.org/x/crypto/sha3/keccakf_amd64.s:324: [amd64] keccakF1600: unknown variable state; offset 0 is a+0(FP)
vendor/golang.org/x/crypto/ssh/certs.go:172: declaration of "err" shadows declaration at vendor/golang.org/x/crypto/ssh/certs.go:166
vendor/golang.org/x/crypto/ssh/certs.go:187: declaration of "rest" shadows declaration at vendor/golang.org/x/crypto/ssh/certs.go:161
vendor/golang.org/x/crypto/ssh/certs.go:187: declaration of "ok" shadows declaration at vendor/golang.org/x/crypto/ssh/certs.go:161
vendor/golang.org/x/crypto/ssh/client_auth.go:226: declaration of "err" shadows declaration at vendor/golang.org/x/crypto/ssh/client_auth.go:193
vendor/golang.org/x/crypto/ssh/client_auth.go:394: declaration of "err" shadows declaration at vendor/golang.org/x/crypto/ssh/client_auth.go:380
vendor/golang.org/x/crypto/ssh/client_auth.go:405: declaration of "err" shadows declaration at vendor/golang.org/x/crypto/ssh/client_auth.go:380
vendor/golang.org/x/crypto/ssh/handshake.go:566: declaration of "err" shadows declaration at vendor/golang.org/x/crypto/ssh/handshake.go:547
vendor/golang.org/x/crypto/ssh/handshake.go:592: declaration of "err" shadows declaration at vendor/golang.org/x/crypto/ssh/handshake.go:547
vendor/golang.org/x/crypto/ssh/handshake.go:630: declaration of "err" shadows declaration at vendor/golang.org/x/crypto/ssh/handshake.go:620
[...]
2017-11-12 13:05:27 +01:00
Jakob Unterwurzacher 843138168f package-source.bash: replace plus sign in file name
Plus signs are apparently not supported on github,
and replaced by a dot.
2017-11-01 20:05:15 +01:00
Jakob Unterwurzacher 5428567fa0 package-source.bash: create VERSION file
...and delete if after packaging is done.
2017-11-01 19:41:47 +01:00
Jakob Unterwurzacher 6c2c2b9a8b README: Update changelog for v1.4.2 2017-11-01 19:29:11 +01:00
Jakob Unterwurzacher 2aaf9c6387 dep: update dependencies 2017-11-01 19:11:26 +01:00
Jakob Unterwurzacher 4c11e8a4e5 travis: update Go versions
...to latest point releases acc. to https://golang.org/dl/
2017-11-01 19:02:34 +01:00
Jakob Unterwurzacher 39839ade70 Add source packaging script 2017-11-01 18:58:10 +01:00
Jakob Unterwurzacher 9a3791fbc1 build.bash: support VERSION file and vendored go-fuse
Prepares for the release of all-in-one source tarballs
that include all non-stdlib dependencies.
2017-11-01 16:09:47 +01:00
Jakob Unterwurzacher a1a98abfbb main: disallow recursively encrypting ourselves
From https://github.com/rfjakob/gocryptfs/issues/150:

  mkdir a
  mkdir a/b
  gocryptsfs -init -reverse a/
  gocryptfs -reverse a/ a/b

  Now directory a/b/ contains encrypted view of 'a' but it
  is possible to descend into encrypted version of b (e.g.
  a/b/43873uhj538765387/) which contains double encrypted
  'a' and so on.

Reported-by: https://github.com/tigmac
2017-10-31 19:48:01 +01:00
Jakob Unterwurzacher 34547a6c39 tests: don't read /proc, the number of entries changes too quickly
This could lead to test failures like this:

  --- FAIL: TestGetdents (0.02s)
  	getdents_test.go:57: len(getdentsEntries)=362, len(readdirEntries)=360
  FAIL
2017-10-22 18:13:08 +02:00
Jakob Unterwurzacher e9f6c7ad67 Revert "test.bash: use "go vet" instead of "go tool vet""
"go vet" on Go 1.8 and older does not support flags:

$ go version
go version go1.8.3 linux/amd64

$ ./test.bash
gocryptfs v1.4.1-27-g8c1b363 without_openssl; go-fuse v20170619-21-gcf21bc2; 2017-10-22 go1.8.3
gocryptfs v1.4.1-27-g8c1b363; go-fuse v20170619-21-gcf21bc2; 2017-10-22 go1.8.3
flag provided but not defined: -all
usage: vet [-n] [-x] [build flags] [packages]
Vet runs the Go vet command on the packages named by the import paths.
For more about vet, see 'go doc cmd/vet'.
For more about specifying packages, see 'go help packages'.
To run the vet tool with specific options, run 'go tool vet'.
The -n flag prints commands that would be executed.
The -x flag prints commands as they are executed.
For more about build flags, see 'go help build'.
See also: go fmt, go fix.

This reverts commit a1170be979.
2017-10-22 15:00:19 +02:00
Jakob Unterwurzacher 8c1b363f74 reverse mode: disable ClientInodes (hard link tracking)
Disable hard link tracking to avoid strange breakage on duplicate
inode numbers ( https://github.com/rfjakob/gocryptfs/issues/149 ).

Reverse mode is read-only, so we don't need a working link().
2017-10-22 14:43:24 +02:00
Jakob Unterwurzacher a1170be979 test.bash: use "go vet" instead of "go tool vet"
"go vet" automatically skips the vendor directory.
"go tool vet" does not, and it will complain about a lot
of things in there.
2017-10-22 14:37:44 +02:00
Jakob Unterwurzacher 4954c87979 Always set "max_read" kernel option
We use fixed-size byte slice pools (sync.Pool) and cannot
handle larger requests. So ask the kernel to not send
bigger ones.

Fixes https://github.com/rfjakob/gocryptfs/issues/145
2017-10-21 18:06:55 +02:00
Jakob Unterwurzacher 268e0484e2 Revert most of "fusefrontend: clamp oversized reads"
We cannot return less data than requested to the kernel!

From https://libfuse.github.io/doxygen/structfuse__operations.html:

  Read should return exactly the number of bytes
  requested except on EOF or error, otherwise the
  rest of the data will be substituted with
  zeroes.

Reverts commit 3009ec9852 minus
the formatting improvements we want to keep.

Fixes https://github.com/rfjakob/gocryptfs/issues/147
Reopens https://github.com/rfjakob/gocryptfs/issues/145
2017-10-21 17:43:21 +02:00
Jakob Unterwurzacher b3c20e512f MANPAGE: explain that you may have to pass -aessiv with -masterkey
...if the filesystem was created with that option (or reverse
mode).

Mitigates https://github.com/rfjakob/gocryptfs/issues/148
2017-10-19 22:04:46 +02:00
Jakob Unterwurzacher 29445c976d contentenc: reserve one additional block in CReqPool
...to account for unaligned reads.

I have not seen this happen in the wild because the kernel
always seems to issue 4k-aligned requests. But the cost
of the additional block in the pool is low and prevents
a buffer overrun panic when an unaligned read does happen.
2017-10-19 09:23:10 +02:00
Jakob Unterwurzacher 2783eadc8f main: call logger with full path
If $PATH contains the mountpoint, searching through it
will lock us up. Use an absolute path to avoid looking
at $PATH.

Fixes https://github.com/rfjakob/gocryptfs/issues/146
2017-10-18 20:25:34 +02:00
Jakob Unterwurzacher 3009ec9852 fusefrontend: clamp oversized reads
Our byte cache pools are sized acc. to MAX_KERNEL_WRITE, but the
running kernel may have a higher limit set. Clamp to what we can
handle.

Fixes a panic on a Synology NAS reported at
https://github.com/rfjakob/gocryptfs/issues/145
2017-10-17 21:48:29 +02:00
Jakob Unterwurzacher 64e5906ffa fusefrontend_reverse: workaround ext4 test failure
The extended TestLongnameStat() exposes a pathological case
when run on ext4, as ext4 reuses inode numbers immediately.

This change modifies the test to not delete the files immediately,
so the inode numbers cannot be reused immediately.

Fix for the underlying issue is a TODO.
2017-10-03 21:15:17 +02:00
Jakob Unterwurzacher 4da245c69d fusefrontend_reverse: fix 176-byte names
A file with a name of exactly 176 bytes length caused this error:

  ls: cannot access ./tmp/dsg/sXSGJLTuZuW1FarwIkJs0w/b6mGjdxIRpaeanTo0rbh0A/QjMRrQZC_4WLhmHI1UOBcA/gocryptfs.longname.QV-UipdDXeUVdl05WruoEzBNPrQCfpu6OzJL0_QnDKY: No such file or directory
  ls: cannot access ./tmp/dsg/sXSGJLTuZuW1FarwIkJs0w/b6mGjdxIRpaeanTo0rbh0A/QjMRrQZC_4WLhmHI1UOBcA/gocryptfs.longname.QV-UipdDXeUVdl05WruoEzBNPrQCfpu6OzJL0_QnDKY.name: No such file or directory
  -????????? ? ?     ?             ?            ? gocryptfs.longname.QV-UipdDXeUVdl05WruoEzBNPrQCfpu6OzJL0_QnDKY
  -????????? ? ?     ?             ?            ? gocryptfs.longname.QV-UipdDXeUVdl05WruoEzBNPrQCfpu6OzJL0_QnDKY.name

Root cause was a wrong shortNameMax constant that failed to
account for the obligatory padding byte.

Fix the constant and also expand the TestLongnameStat test case
to test ALL file name lengths from 1-255 bytes.

Fixes https://github.com/rfjakob/gocryptfs/issues/143 .
2017-10-01 13:50:25 +02:00
Jakob Unterwurzacher 0072a96f20 siv_aead: fix trivial typo in comment 2017-09-17 11:42:46 +02:00
Jakob Unterwurzacher 885fdcabda contentenc: deduplicate AD packing into new concatAD() func
The encrypt and decrypt path both had a copy that were equivalent
but ordered differently, which was confusing.

Consolidate it in a new dedicated function.
2017-09-17 11:21:48 +02:00
Jakob Unterwurzacher 4bd2c6736a contentenc: DecryptBlocks: give block number counter a clearer name
Using firstBlockNo as the counter is confusing, create a
copy named "blockNo" and use that.
2017-09-17 10:59:04 +02:00
Jakob Unterwurzacher f0e29d9b90 performance.txt: specify READ tests 2017-09-10 18:42:06 +02:00
Jakob Unterwurzacher 830cbb7218 build.bash: make reproduceable builds easier
* Reduce the build time precision from seconds to days
* Allow to specify an arbitrary build date through an
  env variable
2017-09-06 21:41:22 +02:00
Jakob Unterwurzacher 512be8f081 Add "dep" files: Gopkg.toml and Gopkg.lock
Allows users to get a reproduceable build. Still needs to
be integrated into build.bash.

Suggested at https://github.com/rfjakob/gocryptfs/issues/142
2017-09-06 20:52:30 +02:00
Jakob Unterwurzacher 604b0779d4 macos: automatically remove .DS_Store on Rmdir
MacOS sprinkles .DS_Store files everywhere. This is hard to avoid for
users, so handle it transparently in Rmdir().

Mitigates https://github.com/rfjakob/gocryptfs/issues/140
2017-09-05 22:47:15 +02:00
Jakob Unterwurzacher 6f3b65d924 fusefrontend: reorder logic in Rmdir to get rid of one indentation level
Handle the errors first so that the normal code path is not indented.

This should not cause any behavoir changes.
2017-09-05 22:10:08 +02:00
Jakob Unterwurzacher 3a5a783b54 macos: don't throw IO errors because of .DS_Store files
MacOS creates lots of these files, and if the directory is otherwise
empty, we would throw an IO error to the unsuspecting user.

With this patch, we log a warning, but otherwise pretend we did not
see it.

Mitigates https://github.com/rfjakob/gocryptfs/issues/140
2017-09-05 21:47:05 +02:00
Jakob Unterwurzacher 538cae610c syscallcompat: Getdents: warn once if we get DT_UNKNOWN
...and if Getdents is not available at all.

Due to this warning I now know that SSHFS always returns DT_UNKNOWN:

    gocryptfs[8129]: Getdents: convertDType: received DT_UNKNOWN, falling back to Lstat

This behavoir is confirmed at http://ahefner.livejournal.com/16875.html:

    "With sshfs, I finally found that obscure case. The dtype is always set to DT_UNKNOWN [...]"
2017-09-03 15:05:54 +02:00
Jakob Unterwurzacher a710451d92 performance.txt: update for v1.4.1 latest commits, and kernel update
$ uname -a
Linux brikett 4.12.5-300.fc26.x86_64 #1 SMP Mon Aug 7 15:27:25 UTC 2017 x86_64 x86_64 x86_64 GNU/Linux
2017-09-03 14:25:01 +02:00
Jakob Unterwurzacher 276567eb13 fusefrontend: use DirIVCache in OpenDir()
Previously, OpenDir() did not use the cache at all, missing
an opportunity to speed up repeated directory reads.
2017-09-03 13:59:53 +02:00
Jakob Unterwurzacher 7da0e97c8b dirivcache: add better function comments + a sanity check on Store()
The comments were unclear on whether relative or absolute paths
have to be passed.
2017-09-03 13:53:50 +02:00
Jakob Unterwurzacher 8f92dd15e1 profiling: add write-trace.bash
Save an execution trace of writing 100MB of data
to a new gocryptfs mount on /tmp
2017-09-03 13:50:53 +02:00
Jakob Unterwurzacher 94496ba840 travis ci: also test Go 1.9 2017-09-02 20:38:52 +02:00
Jakob Unterwurzacher ed046aa359 Fix misspellings reported by goreportcard.com
https://goreportcard.com/report/github.com/rfjakob/gocryptfs#misspell
2017-08-21 21:06:05 +02:00
Jakob Unterwurzacher 26a6e61a46 MANPAGE: update exit codes
The exit codes have been documented in CLI_ABI.md for a while,
but they should also be listed in the man page.

Also fix the rendering of "[-o COMMA-SEPARATED-OPTIONS]", where
the square brackets where interpreted as something. Escape all
square brackets to be safe.
2017-08-21 20:53:25 +02:00
Jakob Unterwurzacher a605e8fed0 README: update changelog for v1.4.1 release 2017-08-21 20:39:25 +02:00
Jakob Unterwurzacher 07f57314af package[-static].bash: stop leaking the local user id in the tarball
The local user id of the packager is not interesting for users who
download the tarball.

Also it will cause the gocryptfs binary to have an unintended owner
when the tarball is extraced as root.

Fix the issue by using "tar --owner=root --group=root" which
overwrites user and group id with zero.
2017-08-21 14:10:05 +02:00
Jakob Unterwurzacher 312ea32bb7 cryptocore: add urandom + randprefetch benchmarks
The benchmark that supported the decision for 512-byte
prefetching previously lived outside the repo.

Let's add it where it belongs so it cannot get lost.
2017-08-16 18:33:00 +02:00
Jakob Unterwurzacher 838bf883df Update performance.txt for Linux kernel upgrade
For some reason, writing became a lot faster in Linux 4.11
(scheduler improvements?).
2017-08-15 19:13:00 +02:00
Jakob Unterwurzacher 069647842c Update performance.txt for to Getdents change 2017-08-15 19:07:08 +02:00
Jakob Unterwurzacher d5671b785a docs: label "ENV CHANGE" column 2017-08-15 19:04:02 +02:00
Jakob Unterwurzacher 989b880989 fusefrontend: use Getdents if available
Getdents avoids calling Lstat on each file.
2017-08-15 19:04:02 +02:00
Jakob Unterwurzacher e50a6a57e5 syscallcompat: implement Getdents()
The Readdir function provided by os is inherently slow because
it calls Lstat on all files.

Getdents gives us all the information we need, but does not have
a proper wrapper in the stdlib.

Implement the "Getdents()" wrapper function that calls
syscall.Getdents() and parses the returned byte blob to a
fuse.DirEntry slice.
2017-08-15 19:03:57 +02:00