Jakob Unterwurzacher
ed046aa359
Fix misspellings reported by goreportcard.com
...
https://goreportcard.com/report/github.com/rfjakob/gocryptfs#misspell
2017-08-21 21:06:05 +02:00
Jakob Unterwurzacher
ccf1a84e41
macos: make testing without openssl work properly
...
On MacOS, building and testing without openssl is much easier.
The tests should skip tests that fail because of missing openssl
instead of aborting.
Fixes https://github.com/rfjakob/gocryptfs/issues/123
2017-07-14 23:22:15 +02:00
Jakob Unterwurzacher
b6bda01c33
contentenc: MergeBlocks: short-circuit the trivial case
...
Saves 3% for the tar extract benchmark because we skip the allocation.
2017-07-02 16:23:24 +02:00
Jakob Unterwurzacher
12c0101a23
contentenc: add PReqPool and use it in DecryptBlocks
...
This gets us a massive speed boost in streaming reads.
2017-06-30 23:30:57 +02:00
Jakob Unterwurzacher
80676c685f
contentenc: add safer "bPool" pool variant; add pBlockPool
...
bPool verifies the lengths of slices going in and out.
Also, add a plaintext block pool - pBlockPool - and use
it for decryption.
2017-06-29 23:44:32 +02:00
Jakob Unterwurzacher
3c6fe98eb1
contentenc: use sync.Pool memory pools for encryption
...
We use two levels of buffers:
1) 4kiB+overhead for each ciphertext block
2) 128kiB+overhead for each FUSE write (32 ciphertext blocks)
This commit adds a sync.Pool for both levels.
The memory-efficiency for small writes could be improved,
as we now always use a 128kiB buffer.
2017-06-20 21:22:00 +02:00
Jakob Unterwurzacher
e52594dae6
contentenc: parallelize encryption for 128kiB writes
...
128kiB = 32 x 4kiB pages is the maximum we get from the kernel. Splitting
up smaller writes is probably not worth it.
Parallelism is limited to two for now.
2017-06-11 21:56:16 +02:00
Jakob Unterwurzacher
294628b384
contentenc: move EncryptBlocks() loop into its own functions
...
This allows easy parallelization in the future.
2017-06-07 22:09:15 +02:00
Jakob Unterwurzacher
a24faa3ba5
fusefrontend: write: consolidate and move encryption to contentenc
...
Collect all the plaintext and pass everything to contentenc in
one call.
This will allow easier parallization of the encryption.
https://github.com/rfjakob/gocryptfs/issues/116
2017-06-01 22:19:27 +02:00
Jakob Unterwurzacher
f44902aaae
Fix two comments
...
One out-of-date and the other with a typo.
2017-06-01 18:53:57 +02:00
Jakob Unterwurzacher
2ce269ec63
contenenc: reject all-zero file ID
...
This should never happen in normal operation and is a sign of
data corruption. Catch it early.
2017-05-25 14:20:27 +02:00
Jakob Unterwurzacher
c0e411f81d
contentenc: better error reporting in ParseHeader
...
Log the message ourselves and return EINVAL.
Before:
gocryptfs[26962]: go-fuse: can't convert error type: ParseHeader: invalid version: got 0, want 2
After:
gocryptfs[617]: ParseHeader: invalid version: want 2, got 0. Returning EINVAL.
2017-05-25 14:18:44 +02:00
Jakob Unterwurzacher
1a89919d80
contentenc: downgrade "interrupted write?" warning to debug
...
This can happen during normal operation, and is harmless since
14038a1644
"fusefrontend: readFileID: reject files that consist only of a header"
causes dormant header-only files to be rewritten on the next write.
2017-05-01 18:44:18 +02:00
Jakob Unterwurzacher
3409ade272
forcedecode: tighten checks
...
...and fix a few golint issues and print a scary warning message on mount.
Also, force the fs to ro,noexec.
2017-04-24 00:25:02 +02:00
danim7
f1945c4daa
Add -forcedecode
...
Force decode of encrypted files even if the integrity check fails, instead of
failing with an IO error. Warning messages are still printed to syslog if corrupted
files are encountered.
It can be useful to recover files from disks with bad sectors or other corrupted
media.
Closes https://github.com/rfjakob/gocryptfs/pull/102 .
2017-04-23 23:11:56 +02:00
Jakob Unterwurzacher
2f953fdb95
contentenc: catch integer underflow in file size calculation
...
If you truncate a ciphertext file to 19 bytes, you could get the
impression that the plaintext is 18446744073709551585 bytes long,
as reported by "ls -l".
Fix it by clamping the value to zero.
2017-03-07 20:56:50 +01:00
Jakob Unterwurzacher
d0bc7970f7
full stack: implement HKDF support
...
...but keep it disabled by default for new filesystems.
We are still missing an example filesystem and CLI arguments
to explicitely enable and disable it.
2017-03-05 21:59:55 +01:00
Jakob Unterwurzacher
b732881518
configfile: switch to 128-bit IVs for master key encryption
...
There is no security reason for doing this, but it will allow
to consolidate the code once we drop compatibility with gocryptfs v1.2
(and earlier) filesystems.
2017-03-05 18:03:03 +01:00
Jakob Unterwurzacher
c9f4400e6d
Replace all calls to naked panic() with log.Panic()
...
We want all panics to show up in the syslog.
2016-12-10 11:54:36 +01:00
Jakob Unterwurzacher
012152f3d1
fusefrontend: I/O error instead of panic on all-zero nonce
...
Running xfstests generic/075 on tmpfs often triggered a panic
for what seems to be a tmpfs bug.
Quoting from the email to lkml,
http://www.spinics.net/lists/kernel/msg2370127.html :
tmpfs seems to be incorrectly returning 0-bytes when reading from
a file that is concurrently being truncated.
2016-10-28 21:18:36 +02:00
Valient Gough
b764917cd5
lint fixes
2016-10-04 23:18:33 +02:00
Jakob Unterwurzacher
a2510efe12
reverse: use per-purpose nonce generation
...
Also pull all the deterministic nonce code into fusefrontend_reverse
to greatly simplify the normal code path.
2016-09-29 21:56:49 +02:00
Jakob Unterwurzacher
e9bb8b800c
reverse: switch from GCM-SIV to AES-SIV
...
GCM-SIV is not yet finalized, and the reference implemenation is
painfully slow at about 2 MB/s. Switch to AES-SIV.
2016-09-26 23:25:13 +02:00
Jakob Unterwurzacher
abd61d968d
contentenc: rename constant "IVBitLen" to "DefaultIVBits" and clarify comment
...
128-bit IVs are NOT used everywhere.
2016-09-25 18:40:29 +02:00
Jakob Unterwurzacher
12808138ef
contentenc: add "ExternalNonce" mode
...
This will be used for strong symlink encryption in reverse mode.
2016-09-25 17:44:19 +02:00
Jakob Unterwurzacher
7bbf6ad6ea
reverse: derive file ID and block IVs from file paths
2016-09-25 16:43:17 +02:00
Jakob Unterwurzacher
5f726aaa9d
contentenc: add GCM-SIV support
...
Also add ReverseDummyNonce nonce generation.
2016-09-25 16:43:17 +02:00
Jakob Unterwurzacher
7f87ed78f2
cryptocore: add support for GCM-SIV
2016-09-25 16:43:17 +02:00
Jakob Unterwurzacher
e42ab3908d
reverse: print value in file offset panic
2016-09-25 16:43:17 +02:00
Jakob Unterwurzacher
5931eea387
contentenc: add helpers for reverse mode
...
Add the reverse variant of DecryptBlocks etc:
* EncryptBlocks
* JointPlaintextRange
* ExplodeCipherRange
2016-09-25 16:43:17 +02:00
Jakob Unterwurzacher
1d62086742
conentenc: handle zero-sized files in PlainSizeToCipherSize
...
Previously caused an integer underflow.
2016-09-25 16:43:17 +02:00
Jakob Unterwurzacher
e8a234f658
Add godoc comments to all internal packages
2016-07-06 21:51:25 +02:00
Jakob Unterwurzacher
04ad063515
fusefronted: move Truncate() and Allocate() to their own file
...
These are large complicated implementations that will share some
code.
2016-07-02 15:35:06 +02:00
Jakob Unterwurzacher
7b22b426b9
contentenc: rename PlaintextRange and CiphertextRange
...
The name could be misunderstood and actually caused a bug:
doWrite used to always preallocate 4128 instead of the actual
data length.
2016-07-02 00:12:36 +02:00
Jakob Unterwurzacher
f2b4d57068
fusefrontend: coalesce grows in Truncate()
...
We were growing the file block-by-block which was pretty
inefficient. We now coalesce all the grows into a single
Ftruncate. Also simplifies the code!
Simplistic benchmark: Before:
$ time truncate -s 1000M foo
real 0m0.568s
After:
$ time truncate -s 1000M foo
real 0m0.205s
2016-07-01 23:32:27 +02:00
Jakob Unterwurzacher
7e92ebe16a
Rename nametransform, contentenc source files
...
Let's have shorter names, and merge *_api.go into the "main"
file.
No code changes.
2016-06-16 19:02:47 +02:00
Jakob Unterwurzacher
6c3f97399a
Rename internal "toggledlog" package to "tlog"
...
tlog is used heavily everywhere and deserves a shorter name.
Renamed using sed magic, without any manual rework:
find * -type f -exec sed -i 's/toggledlog/tlog/g' {} +
2016-06-15 23:30:44 +02:00
Jakob Unterwurzacher
37a9b4c3ee
Enable openssl in tests to support old Go versions
...
Go 1.4 and older do not support 128-bit IVs which caused
the tests to panic.
2016-03-03 00:59:58 +01:00
Jakob Unterwurzacher
653d4a619c
longnames part II: Rename, Unlink, Rmdir, Mknod, Mkdir + tests
2016-02-07 14:02:09 +01:00
Jakob Unterwurzacher
c74772bc8d
Run go fmt
2016-02-06 20:23:36 +01:00
Jakob Unterwurzacher
b0ee5258b1
Fix tests - were broken by the refactoring
2016-02-06 20:22:45 +01:00
Jakob Unterwurzacher
2b8cbd9441
Major refactoring: Split up "cryptfs" into several internal packages
...
"git status" for reference:
deleted: cryptfs/cryptfs.go
deleted: cryptfs/names_core.go
modified: integration_tests/cli_test.go
modified: integration_tests/helpers.go
renamed: cryptfs/config_file.go -> internal/configfile/config_file.go
renamed: cryptfs/config_test.go -> internal/configfile/config_test.go
renamed: cryptfs/config_test/.gitignore -> internal/configfile/config_test/.gitignore
renamed: cryptfs/config_test/PlaintextNames.conf -> internal/configfile/config_test/PlaintextNames.conf
renamed: cryptfs/config_test/StrangeFeature.conf -> internal/configfile/config_test/StrangeFeature.conf
renamed: cryptfs/config_test/v1.conf -> internal/configfile/config_test/v1.conf
renamed: cryptfs/config_test/v2.conf -> internal/configfile/config_test/v2.conf
renamed: cryptfs/kdf.go -> internal/configfile/kdf.go
renamed: cryptfs/kdf_test.go -> internal/configfile/kdf_test.go
renamed: cryptfs/cryptfs_content.go -> internal/contentenc/content.go
new file: internal/contentenc/content_api.go
renamed: cryptfs/content_test.go -> internal/contentenc/content_test.go
renamed: cryptfs/file_header.go -> internal/contentenc/file_header.go
renamed: cryptfs/intrablock.go -> internal/contentenc/intrablock.go
renamed: cryptfs/address_translation.go -> internal/contentenc/offsets.go
new file: internal/cryptocore/crypto_api.go
renamed: cryptfs/gcm_go1.4.go -> internal/cryptocore/gcm_go1.4.go
renamed: cryptfs/gcm_go1.5.go -> internal/cryptocore/gcm_go1.5.go
renamed: cryptfs/nonce.go -> internal/cryptocore/nonce.go
renamed: cryptfs/openssl_aead.go -> internal/cryptocore/openssl_aead.go
renamed: cryptfs/openssl_benchmark.bash -> internal/cryptocore/openssl_benchmark.bash
renamed: cryptfs/openssl_test.go -> internal/cryptocore/openssl_test.go
new file: internal/nametransform/name_api.go
new file: internal/nametransform/names_core.go
renamed: cryptfs/names_diriv.go -> internal/nametransform/names_diriv.go
renamed: cryptfs/names_noiv.go -> internal/nametransform/names_noiv.go
renamed: cryptfs/names_test.go -> internal/nametransform/names_test.go
new file: internal/nametransform/pad16.go
renamed: cryptfs/log.go -> internal/toggledlog/log.go
renamed: cryptfs/log_go1.4.go -> internal/toggledlog/log_go1.4.go
renamed: cryptfs/log_go1.5.go -> internal/toggledlog/log_go1.5.go
modified: main.go
modified: masterkey.go
modified: pathfs_frontend/file.go
modified: pathfs_frontend/file_holes.go
modified: pathfs_frontend/fs.go
modified: pathfs_frontend/fs_dir.go
modified: pathfs_frontend/names.go
modified: test.bash
2016-02-06 19:22:35 +01:00