hardcoresushi/libgocryptfs
1
0
Fork 0
Code Issues Pull Requests Releases Activity
1,503 Commits 2 Branches 0 Tags 6 MiB
eeefddad6a
Commit Graph

55 Commits

Author SHA1 Message Date
Jakob Unterwurzacher
abc59fa968 contentenc: encryptBlocksParallel: explain why last part runs in new goroutine 
The result is counter-intuitive, so explain it here.
 2020-02-15 21:28:12 +01:00
Jakob Unterwurzacher
9aeb2a3df6 contentenc: move parallel encryption into encryptBlocksParallel 
Make the logic self-contained in the new helper function.
 2020-02-15 15:49:12 +01:00
Jakob Unterwurzacher
b3c88f573f contentenc: add explicit test for all-zero header 
https://github.com/rfjakob/gocryptfs/issues/363
 2019-10-06 19:37:51 +02:00
Jakob Unterwurzacher
179471b648 ParseHeader: print hexdump on error 
Should help debugging https://github.com/rfjakob/gocryptfs/issues/363
 2019-02-17 17:13:20 +01:00
Sebastian Lackner
07c486603c configfile: Explicitly wipe scrypt derived key after decrypting/encrypting master key. 
Further raises the bar for recovering keys from memory.
 2018-12-27 18:47:14 +01:00
Sebastian Lackner
874eaf9734 Assorted spelling fixes. 
Mostly detected with the 'codespell' utility, but also includes some
manual grammar fixes.
 2018-12-27 15:19:55 +01:00
Jakob Unterwurzacher
bcca323cb7 contentenc: reserve one extra block in pool plaintext buffers 
File holes and -fsck can cause unaligned read accesses, which means
we have to decrypt one extra plaintext block.

xfstests generic/083 manage to crash -fsck like this:

generic/083	2018/07/14 15:25:21 wrong len=266240, want=131072
panic: wrong len=266240, want=131072

goroutine 1 [running]:
log.Panicf(0x67fc00, 0x15, 0xc4204fec90, 0x2, 0x2)
	/usr/local/go/src/log/log.go:333 +0xda
github.com/rfjakob/gocryptfs/internal/contentenc.(*bPool).Put(0xc4200d4800, 0xc4202f2000, 0x21000, 0x41000)
	/home/jakob/go/src/github.com/rfjakob/gocryptfs/internal/contentenc/bpool.go:27 +0x15d
github.com/rfjakob/gocryptfs/internal/fusefrontend.(*File).doRead(0xc4200b4500, 0xc42019e000, 0x0, 0x20000, 0x28400, 0x20000, 0xc42019e000, 0xc4204ff008, 0x435164, 0xc420000180)
	/home/jakob/go/src/github.com/rfjakob/gocryptfs/internal/fusefrontend/file.go:227 +0xba9
github.com/rfjakob/gocryptfs/internal/fusefrontend.(*File).Read(0xc4200b4500, 0xc42019e000, 0x20000, 0x20000, 0x28400, 0x0, 0x0, 0x0)
	/home/jakob/go/src/github.com/rfjakob/gocryptfs/internal/fusefrontend/file.go:246 +0x23e
main.(*fsckObj).file(0xc420069320, 0xc42001a630, 0x21)
	/home/jakob/go/src/github.com/rfjakob/gocryptfs/fsck.go:126 +0x21f
main.(*fsckObj).dir(0xc420069320, 0xc420014dc0, 0x1d)
	/home/jakob/go/src/github.com/rfjakob/gocryptfs/fsck.go:76 +0x387
main.(*fsckObj).dir(0xc420069320, 0xc42021dae0, 0x19)
	/home/jakob/go/src/github.com/rfjakob/gocryptfs/fsck.go:74 +0x347
 2018-07-15 11:39:19 +02:00
Jakob Unterwurzacher
b6c8960b01 fsck: clean up log output 
Make sure we get only 1 warning output per
problem.

Also, add new corruption types to broken_fs_v1.4.
 2018-04-02 18:32:30 +02:00
Jakob Unterwurzacher
1ed3d51df1 fusefrontend: add xattr support 
At the moment, only for reverse mode.

https://github.com/rfjakob/gocryptfs/issues/217
 2018-03-25 21:06:10 +02:00
Jakob Unterwurzacher
29445c976d contentenc: reserve one additional block in CReqPool 
...to account for unaligned reads.

I have not seen this happen in the wild because the kernel
always seems to issue 4k-aligned requests. But the cost
of the additional block in the pool is low and prevents
a buffer overrun panic when an unaligned read does happen.
 2017-10-19 09:23:10 +02:00
Jakob Unterwurzacher
3009ec9852 fusefrontend: clamp oversized reads 
Our byte cache pools are sized acc. to MAX_KERNEL_WRITE, but the
running kernel may have a higher limit set. Clamp to what we can
handle.

Fixes a panic on a Synology NAS reported at
https://github.com/rfjakob/gocryptfs/issues/145
 2017-10-17 21:48:29 +02:00
Jakob Unterwurzacher
885fdcabda contentenc: deduplicate AD packing into new concatAD() func 
The encrypt and decrypt path both had a copy that were equivalent
but ordered differently, which was confusing.

Consolidate it in a new dedicated function.
 2017-09-17 11:21:48 +02:00
Jakob Unterwurzacher
4bd2c6736a contentenc: DecryptBlocks: give block number counter a clearer name 
Using firstBlockNo as the counter is confusing, create a
copy named "blockNo" and use that.
 2017-09-17 10:59:04 +02:00
Jakob Unterwurzacher
ed046aa359 Fix misspellings reported by goreportcard.com 
https://goreportcard.com/report/github.com/rfjakob/gocryptfs#misspell
 2017-08-21 21:06:05 +02:00
Jakob Unterwurzacher
ccf1a84e41 macos: make testing without openssl work properly 
On MacOS, building and testing without openssl is much easier.
The tests should skip tests that fail because of missing openssl
instead of aborting.

Fixes https://github.com/rfjakob/gocryptfs/issues/123
 2017-07-14 23:22:15 +02:00
Jakob Unterwurzacher
b6bda01c33 contentenc: MergeBlocks: short-circuit the trivial case 
Saves 3% for the tar extract benchmark because we skip the allocation.
 2017-07-02 16:23:24 +02:00
Jakob Unterwurzacher
12c0101a23 contentenc: add PReqPool and use it in DecryptBlocks 
This gets us a massive speed boost in streaming reads.
 2017-06-30 23:30:57 +02:00
Jakob Unterwurzacher
80676c685f contentenc: add safer "bPool" pool variant; add pBlockPool 
bPool verifies the lengths of slices going in and out.

Also, add a plaintext block pool - pBlockPool - and use
it for decryption.
 2017-06-29 23:44:32 +02:00
Jakob Unterwurzacher
3c6fe98eb1 contentenc: use sync.Pool memory pools for encryption 
We use two levels of buffers:

1) 4kiB+overhead for each ciphertext block
2) 128kiB+overhead for each FUSE write (32 ciphertext blocks)

This commit adds a sync.Pool for both levels.

The memory-efficiency for small writes could be improved,
as we now always use a 128kiB buffer.
 2017-06-20 21:22:00 +02:00
Jakob Unterwurzacher
e52594dae6 contentenc: parallelize encryption for 128kiB writes 
128kiB = 32 x 4kiB pages is the maximum we get from the kernel. Splitting
up smaller writes is probably not worth it.

Parallelism is limited to two for now.
 2017-06-11 21:56:16 +02:00
Jakob Unterwurzacher
294628b384 contentenc: move EncryptBlocks() loop into its own functions 
This allows easy parallelization in the future.
 2017-06-07 22:09:15 +02:00
Jakob Unterwurzacher
a24faa3ba5 fusefrontend: write: consolidate and move encryption to contentenc 
Collect all the plaintext and pass everything to contentenc in
one call.

This will allow easier parallization of the encryption.

https://github.com/rfjakob/gocryptfs/issues/116
 2017-06-01 22:19:27 +02:00
Jakob Unterwurzacher
f44902aaae Fix two comments 
One out-of-date and the other with a typo.
 2017-06-01 18:53:57 +02:00
Jakob Unterwurzacher
2ce269ec63 contenenc: reject all-zero file ID 
This should never happen in normal operation and is a sign of
data corruption. Catch it early.
 2017-05-25 14:20:27 +02:00
Jakob Unterwurzacher
c0e411f81d contentenc: better error reporting in ParseHeader 
Log the message ourselves and return EINVAL.

Before:

	gocryptfs[26962]: go-fuse: can't convert error type: ParseHeader: invalid version: got 0, want 2

After:

	gocryptfs[617]: ParseHeader: invalid version: want 2, got 0. Returning EINVAL.
 2017-05-25 14:18:44 +02:00
Jakob Unterwurzacher
1a89919d80 contentenc: downgrade "interrupted write?" warning to debug 
This can happen during normal operation, and is harmless since

14038a1644
"fusefrontend: readFileID: reject files that consist only of a header"

causes dormant header-only files to be rewritten on the next write.
 2017-05-01 18:44:18 +02:00
Jakob Unterwurzacher
3409ade272 forcedecode: tighten checks 
...and fix a few golint issues and print a scary warning message on mount.

Also, force the fs to ro,noexec.
 2017-04-24 00:25:02 +02:00
danim7
f1945c4daa Add -forcedecode 
Force decode of encrypted files even if the integrity check fails, instead of
failing with an IO error. Warning messages are still printed to syslog if corrupted
files are encountered.
It can be useful to recover files from disks with bad sectors or other corrupted
media.

Closes https://github.com/rfjakob/gocryptfs/pull/102 .
 2017-04-23 23:11:56 +02:00
Jakob Unterwurzacher
2f953fdb95 contentenc: catch integer underflow in file size calculation 
If you truncate a ciphertext file to 19 bytes, you could get the
impression that the plaintext is 18446744073709551585 bytes long,
as reported by "ls -l".

Fix it by clamping the value to zero.
 2017-03-07 20:56:50 +01:00
Jakob Unterwurzacher
d0bc7970f7 full stack: implement HKDF support 
...but keep it disabled by default for new filesystems.

We are still missing an example filesystem and CLI arguments
to explicitely enable and disable it.
 2017-03-05 21:59:55 +01:00
Jakob Unterwurzacher
b732881518 configfile: switch to 128-bit IVs for master key encryption 
There is no security reason for doing this, but it will allow
to consolidate the code once we drop compatibility with gocryptfs v1.2
(and earlier) filesystems.
 2017-03-05 18:03:03 +01:00
Jakob Unterwurzacher
c9f4400e6d Replace all calls to naked panic() with log.Panic() 
We want all panics to show up in the syslog.
 2016-12-10 11:54:36 +01:00
Jakob Unterwurzacher
012152f3d1 fusefrontend: I/O error instead of panic on all-zero nonce 
Running xfstests generic/075 on tmpfs often triggered a panic
for what seems to be a tmpfs bug.

Quoting from the email to lkml,
http://www.spinics.net/lists/kernel/msg2370127.html :

	tmpfs seems to be incorrectly returning 0-bytes when reading from
	a file that is concurrently being truncated.
 2016-10-28 21:18:36 +02:00
Valient Gough
b764917cd5 lint fixes 2016-10-04 23:18:33 +02:00
Jakob Unterwurzacher
a2510efe12 reverse: use per-purpose nonce generation 
Also pull all the deterministic nonce code into fusefrontend_reverse
to greatly simplify the normal code path.
 2016-09-29 21:56:49 +02:00
Jakob Unterwurzacher
e9bb8b800c reverse: switch from GCM-SIV to AES-SIV 
GCM-SIV is not yet finalized, and the reference implemenation is
painfully slow at about 2 MB/s. Switch to AES-SIV.
 2016-09-26 23:25:13 +02:00
Jakob Unterwurzacher
abd61d968d contentenc: rename constant "IVBitLen" to "DefaultIVBits" and clarify comment 
128-bit IVs are NOT used everywhere.
 2016-09-25 18:40:29 +02:00
Jakob Unterwurzacher
12808138ef contentenc: add "ExternalNonce" mode 
This will be used for strong symlink encryption in reverse mode.
 2016-09-25 17:44:19 +02:00
Jakob Unterwurzacher
7bbf6ad6ea reverse: derive file ID and block IVs from file paths 2016-09-25 16:43:17 +02:00
Jakob Unterwurzacher
5f726aaa9d contentenc: add GCM-SIV support 
Also add ReverseDummyNonce nonce generation.
 2016-09-25 16:43:17 +02:00
Jakob Unterwurzacher
7f87ed78f2 cryptocore: add support for GCM-SIV 2016-09-25 16:43:17 +02:00
Jakob Unterwurzacher
e42ab3908d reverse: print value in file offset panic 2016-09-25 16:43:17 +02:00
Jakob Unterwurzacher
5931eea387 contentenc: add helpers for reverse mode 
Add the reverse variant of DecryptBlocks etc:

* EncryptBlocks
* JointPlaintextRange
* ExplodeCipherRange
 2016-09-25 16:43:17 +02:00
Jakob Unterwurzacher
1d62086742 conentenc: handle zero-sized files in PlainSizeToCipherSize 
Previously caused an integer underflow.
 2016-09-25 16:43:17 +02:00
Jakob Unterwurzacher
e8a234f658 Add godoc comments to all internal packages 2016-07-06 21:51:25 +02:00
Jakob Unterwurzacher
04ad063515 fusefronted: move Truncate() and Allocate() to their own file 
These are large complicated implementations that will share some
code.
 2016-07-02 15:35:06 +02:00
Jakob Unterwurzacher
7b22b426b9 contentenc: rename PlaintextRange and CiphertextRange 
The name could be misunderstood and actually caused a bug:
doWrite used to always preallocate 4128 instead of the actual
data length.
 2016-07-02 00:12:36 +02:00
Jakob Unterwurzacher
f2b4d57068 fusefrontend: coalesce grows in Truncate() 
We were growing the file block-by-block which was pretty
inefficient. We now coalesce all the grows into a single
Ftruncate. Also simplifies the code!

Simplistic benchmark: Before:

  $ time truncate -s 1000M foo
  real	0m0.568s

After:

  $ time truncate -s 1000M foo
  real	0m0.205s
 2016-07-01 23:32:27 +02:00
Jakob Unterwurzacher
7e92ebe16a Rename nametransform, contentenc source files 
Let's have shorter names, and merge *_api.go into the "main"
file.

No code changes.
 2016-06-16 19:02:47 +02:00
Jakob Unterwurzacher
6c3f97399a Rename internal "toggledlog" package to "tlog" 
tlog is used heavily everywhere and deserves a shorter name.

Renamed using sed magic, without any manual rework:

   find * -type f -exec sed -i 's/toggledlog/tlog/g' {} +
 2016-06-15 23:30:44 +02:00