...against concurrent closes.
The testcase
(set -e; while true; do truncate -s $RANDOM b; done) &
(set -e; while true; do truncate -s $RANDOM b; done) &
uncovered lots of unnecessary RMW failures that were the result
of concurrent closes.
With this patch, the only remaining error is "Truncate on forgotten file"
that is probably caused by a problem in the go-fuse lib
( https://github.com/hanwen/go-fuse/issues/95 )
Instead of using SetOutput(ioutil.Discard), which means
that Printf is still called for every debug message,
use a simple and fast boolean check.
Streaming write performance improves from 86 to 93 MB/s.
This makes sure writing to a file fails early if the underlying
filesystem does not support fallocate. It also prevents partial header
write due to ENOSPC.
Creating the config file can fail easily, for example if the
password is not entered the same twice. This would leave an
orphaned gocryptfs.diriv behind.
Another 3x performance boost for applications that walk the
directory tree.
Excerpt from performance.txt:
VERSION UNTAR LS RM
v0.4 48 1.5 5
v0.5-rc1 56 7 19
v0.5-rc1-1 54 4.1 9
v0.5-rc1-2 45 1.7 3.4 <---- THIS VERSION
Formerly, we called decryptPath for every name.
That resulted in a directory walk that reads in all diriv files
on the way.
Massive improvement for RM and LS (check performance.txt for details)
VERSION UNTAR RM LS
v0.4 48 5 1.5
v0.5-rc1 56 19 7
v0.5-rc1-1 54 9 4.1 <---- THIS VERSION
Use that option to speed up the automated tests by 7 seconds.
Before:
ok github.com/rfjakob/gocryptfs/integration_tests 26.667s
After:
ok github.com/rfjakob/gocryptfs/integration_tests 19.534s
A file never gets a cipherSize <= HEADER_LEN in normal operation.
However, this can happen if header write it interrupted or the
underlying filesystem does not support fallocate.
Noticed while trying to store a CIPHERDIR in another gocryptfs mount
(gocryptfs does not support fallocate)
// List of feature flags this filesystem has enabled.
// If gocryptfs encounters a feature flag it does not support, it will refuse
// mounting. This mechanism is analogous to the ext4 feature flags that are
// stored in the superblock.
FeatureFlags []string
Move all the intelligence into the new file address_translation.go.
That the calculations were spread out too much became apparent when adding
the file header. This should make the code much easier to modify in the
future.
Format: [ "Version" uint16 big endian ] [ "Id" 16 random bytes ]
Quoting SECURITY.md:
* Every file has a header that contains a 16-byte random *file id*
* Each block uses the file id and its block number as GCM *authentication data*
* This means the position of the blocks is protected as well. The blocks
can not be reordered or copied between different files without
causing an decryption error.
Jakob Unterwurzacher