29445c976d
contentenc: reserve one additional block in CReqPool
...
...to account for unaligned reads.
I have not seen this happen in the wild because the kernel
always seems to issue 4k-aligned requests. But the cost
of the additional block in the pool is low and prevents
a buffer overrun panic when an unaligned read does happen.
2017-10-19 09:23:10 +02:00
3009ec9852
fusefrontend: clamp oversized reads
...
Our byte cache pools are sized acc. to MAX_KERNEL_WRITE, but the
running kernel may have a higher limit set. Clamp to what we can
handle.
Fixes a panic on a Synology NAS reported at
https://github.com/rfjakob/gocryptfs/issues/145
2017-10-17 21:48:29 +02:00
885fdcabda
contentenc: deduplicate AD packing into new concatAD() func
...
The encrypt and decrypt path both had a copy that were equivalent
but ordered differently, which was confusing.
Consolidate it in a new dedicated function.
2017-09-17 11:21:48 +02:00
4bd2c6736a
contentenc: DecryptBlocks: give block number counter a clearer name
...
Using firstBlockNo as the counter is confusing, create a
copy named "blockNo" and use that.
2017-09-17 10:59:04 +02:00
ed046aa359
Fix misspellings reported by goreportcard.com
...
https://goreportcard.com/report/github.com/rfjakob/gocryptfs#misspell
2017-08-21 21:06:05 +02:00
ccf1a84e41
macos: make testing without openssl work properly
...
On MacOS, building and testing without openssl is much easier.
The tests should skip tests that fail because of missing openssl
instead of aborting.
Fixes https://github.com/rfjakob/gocryptfs/issues/123
2017-07-14 23:22:15 +02:00
b6bda01c33
contentenc: MergeBlocks: short-circuit the trivial case
...
Saves 3% for the tar extract benchmark because we skip the allocation.
2017-07-02 16:23:24 +02:00
12c0101a23
contentenc: add PReqPool and use it in DecryptBlocks
...
This gets us a massive speed boost in streaming reads.
2017-06-30 23:30:57 +02:00
80676c685f
contentenc: add safer "bPool" pool variant; add pBlockPool
...
bPool verifies the lengths of slices going in and out.
Also, add a plaintext block pool - pBlockPool - and use
it for decryption.
2017-06-29 23:44:32 +02:00
3c6fe98eb1
contentenc: use sync.Pool memory pools for encryption
...
We use two levels of buffers:
1) 4kiB+overhead for each ciphertext block
2) 128kiB+overhead for each FUSE write (32 ciphertext blocks)
This commit adds a sync.Pool for both levels.
The memory-efficiency for small writes could be improved,
as we now always use a 128kiB buffer.
2017-06-20 21:22:00 +02:00
e52594dae6
contentenc: parallelize encryption for 128kiB writes
...
128kiB = 32 x 4kiB pages is the maximum we get from the kernel. Splitting
up smaller writes is probably not worth it.
Parallelism is limited to two for now.
2017-06-11 21:56:16 +02:00
294628b384
contentenc: move EncryptBlocks() loop into its own functions
...
This allows easy parallelization in the future.
2017-06-07 22:09:15 +02:00
a24faa3ba5
fusefrontend: write: consolidate and move encryption to contentenc
...
Collect all the plaintext and pass everything to contentenc in
one call.
This will allow easier parallization of the encryption.
https://github.com/rfjakob/gocryptfs/issues/116
2017-06-01 22:19:27 +02:00
f44902aaae
Fix two comments
...
One out-of-date and the other with a typo.
2017-06-01 18:53:57 +02:00
2ce269ec63
contenenc: reject all-zero file ID
...
This should never happen in normal operation and is a sign of
data corruption. Catch it early.
2017-05-25 14:20:27 +02:00
c0e411f81d
contentenc: better error reporting in ParseHeader
...
Log the message ourselves and return EINVAL.
Before:
gocryptfs[26962]: go-fuse: can't convert error type: ParseHeader: invalid version: got 0, want 2
After:
gocryptfs[617]: ParseHeader: invalid version: want 2, got 0. Returning EINVAL.
2017-05-25 14:18:44 +02:00
1a89919d80
contentenc: downgrade "interrupted write?" warning to debug
...
This can happen during normal operation, and is harmless since
14038a1644
2017-05-01 18:44:18 +02:00
3409ade272
forcedecode: tighten checks
...
...and fix a few golint issues and print a scary warning message on mount.
Also, force the fs to ro,noexec.
2017-04-24 00:25:02 +02:00
f1945c4daa
Add -forcedecode
...
Force decode of encrypted files even if the integrity check fails, instead of
failing with an IO error. Warning messages are still printed to syslog if corrupted
files are encountered.
It can be useful to recover files from disks with bad sectors or other corrupted
media.
Closes https://github.com/rfjakob/gocryptfs/pull/102 .
2017-04-23 23:11:56 +02:00
2f953fdb95
contentenc: catch integer underflow in file size calculation
...
If you truncate a ciphertext file to 19 bytes, you could get the
impression that the plaintext is 18446744073709551585 bytes long,
as reported by "ls -l".
Fix it by clamping the value to zero.
2017-03-07 20:56:50 +01:00
d0bc7970f7
full stack: implement HKDF support
...
...but keep it disabled by default for new filesystems.
We are still missing an example filesystem and CLI arguments
to explicitely enable and disable it.
2017-03-05 21:59:55 +01:00
b732881518
configfile: switch to 128-bit IVs for master key encryption
...
There is no security reason for doing this, but it will allow
to consolidate the code once we drop compatibility with gocryptfs v1.2
(and earlier) filesystems.
2017-03-05 18:03:03 +01:00
c9f4400e6d
Replace all calls to naked panic() with log.Panic()
...
We want all panics to show up in the syslog.
2016-12-10 11:54:36 +01:00
012152f3d1
fusefrontend: I/O error instead of panic on all-zero nonce
...
Running xfstests generic/075 on tmpfs often triggered a panic
for what seems to be a tmpfs bug.
Quoting from the email to lkml,
http://www.spinics.net/lists/kernel/msg2370127.html :
tmpfs seems to be incorrectly returning 0-bytes when reading from
a file that is concurrently being truncated.
2016-10-28 21:18:36 +02:00
b764917cd5
lint fixes
2016-10-04 23:18:33 +02:00
a2510efe12
reverse: use per-purpose nonce generation
...
Also pull all the deterministic nonce code into fusefrontend_reverse
to greatly simplify the normal code path.
2016-09-29 21:56:49 +02:00
e9bb8b800c
reverse: switch from GCM-SIV to AES-SIV
...
GCM-SIV is not yet finalized, and the reference implemenation is
painfully slow at about 2 MB/s. Switch to AES-SIV.
2016-09-26 23:25:13 +02:00
abd61d968d
contentenc: rename constant "IVBitLen" to "DefaultIVBits" and clarify comment
...
128-bit IVs are NOT used everywhere.
2016-09-25 18:40:29 +02:00
12808138ef
contentenc: add "ExternalNonce" mode
...
This will be used for strong symlink encryption in reverse mode.
2016-09-25 17:44:19 +02:00
7bbf6ad6ea
reverse: derive file ID and block IVs from file paths
2016-09-25 16:43:17 +02:00
5f726aaa9d
contentenc: add GCM-SIV support
...
Also add ReverseDummyNonce nonce generation.
2016-09-25 16:43:17 +02:00
7f87ed78f2
cryptocore: add support for GCM-SIV
2016-09-25 16:43:17 +02:00
e42ab3908d
reverse: print value in file offset panic
2016-09-25 16:43:17 +02:00
5931eea387
contentenc: add helpers for reverse mode
...
Add the reverse variant of DecryptBlocks etc:
* EncryptBlocks
* JointPlaintextRange
* ExplodeCipherRange
2016-09-25 16:43:17 +02:00
1d62086742
conentenc: handle zero-sized files in PlainSizeToCipherSize
...
Previously caused an integer underflow.
2016-09-25 16:43:17 +02:00
e8a234f658
Add godoc comments to all internal packages
2016-07-06 21:51:25 +02:00
04ad063515
fusefronted: move Truncate() and Allocate() to their own file
...
These are large complicated implementations that will share some
code.
2016-07-02 15:35:06 +02:00
7b22b426b9
contentenc: rename PlaintextRange and CiphertextRange
...
The name could be misunderstood and actually caused a bug:
doWrite used to always preallocate 4128 instead of the actual
data length.
2016-07-02 00:12:36 +02:00
f2b4d57068
fusefrontend: coalesce grows in Truncate()
...
We were growing the file block-by-block which was pretty
inefficient. We now coalesce all the grows into a single
Ftruncate. Also simplifies the code!
Simplistic benchmark: Before:
$ time truncate -s 1000M foo
real 0m0.568s
After:
$ time truncate -s 1000M foo
real 0m0.205s
2016-07-01 23:32:27 +02:00
7e92ebe16a
Rename nametransform, contentenc source files
...
Let's have shorter names, and merge *_api.go into the "main"
file.
No code changes.
2016-06-16 19:02:47 +02:00
6c3f97399a
Rename internal "toggledlog" package to "tlog"
...
tlog is used heavily everywhere and deserves a shorter name.
Renamed using sed magic, without any manual rework:
find * -type f -exec sed -i 's/toggledlog/tlog/g' {} +
2016-06-15 23:30:44 +02:00
37a9b4c3ee
Enable openssl in tests to support old Go versions
...
Go 1.4 and older do not support 128-bit IVs which caused
the tests to panic.
2016-03-03 00:59:58 +01:00
653d4a619c
longnames part II: Rename, Unlink, Rmdir, Mknod, Mkdir + tests
2016-02-07 14:02:09 +01:00
c74772bc8d
Run go fmt
2016-02-06 20:23:36 +01:00
b0ee5258b1
Fix tests - were broken by the refactoring
2016-02-06 20:22:45 +01:00
2b8cbd9441
Major refactoring: Split up "cryptfs" into several internal packages
...
"git status" for reference:
deleted: cryptfs/cryptfs.go
deleted: cryptfs/names_core.go
modified: integration_tests/cli_test.go
modified: integration_tests/helpers.go
renamed: cryptfs/config_file.go -> internal/configfile/config_file.go
renamed: cryptfs/config_test.go -> internal/configfile/config_test.go
renamed: cryptfs/config_test/.gitignore -> internal/configfile/config_test/.gitignore
renamed: cryptfs/config_test/PlaintextNames.conf -> internal/configfile/config_test/PlaintextNames.conf
renamed: cryptfs/config_test/StrangeFeature.conf -> internal/configfile/config_test/StrangeFeature.conf
renamed: cryptfs/config_test/v1.conf -> internal/configfile/config_test/v1.conf
renamed: cryptfs/config_test/v2.conf -> internal/configfile/config_test/v2.conf
renamed: cryptfs/kdf.go -> internal/configfile/kdf.go
renamed: cryptfs/kdf_test.go -> internal/configfile/kdf_test.go
renamed: cryptfs/cryptfs_content.go -> internal/contentenc/content.go
new file: internal/contentenc/content_api.go
renamed: cryptfs/content_test.go -> internal/contentenc/content_test.go
renamed: cryptfs/file_header.go -> internal/contentenc/file_header.go
renamed: cryptfs/intrablock.go -> internal/contentenc/intrablock.go
renamed: cryptfs/address_translation.go -> internal/contentenc/offsets.go
new file: internal/cryptocore/crypto_api.go
renamed: cryptfs/gcm_go1.4.go -> internal/cryptocore/gcm_go1.4.go
renamed: cryptfs/gcm_go1.5.go -> internal/cryptocore/gcm_go1.5.go
renamed: cryptfs/nonce.go -> internal/cryptocore/nonce.go
renamed: cryptfs/openssl_aead.go -> internal/cryptocore/openssl_aead.go
renamed: cryptfs/openssl_benchmark.bash -> internal/cryptocore/openssl_benchmark.bash
renamed: cryptfs/openssl_test.go -> internal/cryptocore/openssl_test.go
new file: internal/nametransform/name_api.go
new file: internal/nametransform/names_core.go
renamed: cryptfs/names_diriv.go -> internal/nametransform/names_diriv.go
renamed: cryptfs/names_noiv.go -> internal/nametransform/names_noiv.go
renamed: cryptfs/names_test.go -> internal/nametransform/names_test.go
new file: internal/nametransform/pad16.go
renamed: cryptfs/log.go -> internal/toggledlog/log.go
renamed: cryptfs/log_go1.4.go -> internal/toggledlog/log_go1.4.go
renamed: cryptfs/log_go1.5.go -> internal/toggledlog/log_go1.5.go
modified: main.go
modified: masterkey.go
modified: pathfs_frontend/file.go
modified: pathfs_frontend/file_holes.go
modified: pathfs_frontend/fs.go
modified: pathfs_frontend/fs_dir.go
modified: pathfs_frontend/names.go
modified: test.bash
2016-02-06 19:22:35 +01:00
Jakob Unterwurzacher