Commit Graph

1128 Commits

Author SHA1 Message Date
Jakob Unterwurzacher
52ab0462a4 fusefrontend: doRead: skip decryption for an empty read
Previously we ran through the decryption steps even for an empty
ciphertext slice. The functions handle it correctly, but returning
early skips all the extra calls.

Speeds up the tar extract benchmark by about 4%.
2017-07-02 16:02:13 +02:00
Jakob Unterwurzacher
ab787e18f0 README: update mac os x support status to "beta" 2017-07-01 11:51:02 +02:00
Jakob Unterwurzacher
b256c39271 profiling: add tar-extract helper
Extracts the linux-3.0.tar.gz tarball while capturing memory
and cpu profiles.
2017-07-01 11:42:52 +02:00
Jakob Unterwurzacher
52a8e14332 Travis CI: update to latest stable Go point releases 2017-07-01 11:39:33 +02:00
Jakob Unterwurzacher
9d10dcbd1c main: get rid of magic "MaxWrite: 1048576" constant
go-fuse caps MaxWrite at MAX_KERNEL_WRITE anyway, and we
actually depend on this behavoir now as the byte pools
are sized according to MAX_KERNEL_WRITE.

So let's use MAX_KERNEL_WRITE explicitely.
2017-07-01 11:32:17 +02:00
Jakob Unterwurzacher
b3e554acc2 Update performance.txt with new results
Massive speed boost for streaming reads.
2017-07-01 10:00:50 +02:00
Jakob Unterwurzacher
9f4bd76576 stupidgcm: add test for in-place Open
Adds a test for the optimization introduced in:

	stupidgcm: Open: if "dst" is big enough, use it as the output buffer
2017-07-01 09:56:05 +02:00
Jakob Unterwurzacher
12c0101a23 contentenc: add PReqPool and use it in DecryptBlocks
This gets us a massive speed boost in streaming reads.
2017-06-30 23:30:57 +02:00
Jakob Unterwurzacher
e4b5005bcc stupidgcm: Open: if "dst" is big enough, use it as the output buffer
This means we won't need any allocation for the plaintext.
2017-06-30 23:24:12 +02:00
Jakob Unterwurzacher
b2a23e94d1 fusefrontend: doRead: use CReqPool for ciphertext buffer
Easily saves lots of allocations.
2017-06-30 23:15:31 +02:00
Jakob Unterwurzacher
06398e82d9 fusefrontend: Read: use provided buffer
This will allow us to return internal buffers to a pool.
2017-06-30 23:11:38 +02:00
Jakob Unterwurzacher
2932a285aa Update performance.txt with new numbers 2017-06-29 23:45:39 +02:00
Jakob Unterwurzacher
80676c685f contentenc: add safer "bPool" pool variant; add pBlockPool
bPool verifies the lengths of slices going in and out.

Also, add a plaintext block pool - pBlockPool - and use
it for decryption.
2017-06-29 23:44:32 +02:00
Jakob Unterwurzacher
3d32bcd37b profiling: fix hardcoded path in hint 2017-06-29 19:20:34 +02:00
Jakob Unterwurzacher
dee88f3c4d Update performance.txt with new numbers 2017-06-29 19:00:16 +02:00
Jakob Unterwurzacher
0cc6f53496 stupidgcm: use "dst" as the output buffer it is big enough
This saves an allocation of the ciphertext block.
2017-06-29 18:52:33 +02:00
Jakob Unterwurzacher
c12a50d4c8 benchmarks: add streaming read benchmark 2017-06-27 00:04:58 +02:00
Jakob Unterwurzacher
b9b52854c3 profiling: add streaming read profiling helper
Reads 1GB of zeros while collecting memory and cpu profiles.
2017-06-24 15:52:24 +02:00
Jakob Unterwurzacher
5c7b5770ce Update performance numbers 2017-06-20 21:46:27 +02:00
Jakob Unterwurzacher
3c6fe98eb1 contentenc: use sync.Pool memory pools for encryption
We use two levels of buffers:

1) 4kiB+overhead for each ciphertext block
2) 128kiB+overhead for each FUSE write (32 ciphertext blocks)

This commit adds a sync.Pool for both levels.

The memory-efficiency for small writes could be improved,
as we now always use a 128kiB buffer.
2017-06-20 21:22:00 +02:00
Jakob Unterwurzacher
609343accf README: update changelog for v1.4 2017-06-20 19:56:38 +02:00
Jakob Unterwurzacher
bfe421b327 MANPAGE: reorder options to match "-hh" output; add "-hkdf", "-trace" 2017-06-20 19:49:18 +02:00
Jakob Unterwurzacher
f3965a4e4c build.bash: use plain "git describe" for go-fuse
go-fuse recently added a git tag - let's use it.
2017-06-20 18:59:48 +02:00
Jakob Unterwurzacher
c9c4bc0141 profiling: add streaming-write profiling helper
Writes 1GB of zeros to a gocryptfs mount while collecting
cpu and memory profiles.
2017-06-18 22:56:50 +02:00
Jakob Unterwurzacher
a4563e21ec main, syscallcompat: use Dup3 instead of Dup2
Dup2 is not implemented on linux/arm64.

Fixes https://github.com/rfjakob/gocryptfs/issues/121 .

Also adds cross-compilation to CI.
2017-06-18 15:43:22 +02:00
Jakob Unterwurzacher
afc3a8252b Add performance numbers for v1.3-69-ge52594d 2017-06-11 21:58:01 +02:00
Jakob Unterwurzacher
e52594dae6 contentenc: parallelize encryption for 128kiB writes
128kiB = 32 x 4kiB pages is the maximum we get from the kernel. Splitting
up smaller writes is probably not worth it.

Parallelism is limited to two for now.
2017-06-11 21:56:16 +02:00
Jakob Unterwurzacher
24a7b1b7b8 Add performance numbers for last change
Slight streaming write improvement.
2017-06-11 21:44:24 +02:00
Jakob Unterwurzacher
9837cb0ddc cryptocore: prefetch nonces in the background
Spawn a worker goroutine that reads the next 512-byte block
while the current one is being drained.

This should help reduce waiting times when /dev/urandom is very
slow (like on Linux 3.16 kernels).
2017-06-11 21:29:50 +02:00
Jakob Unterwurzacher
f351c3c1ec benchmark.bash: add dd-only mode, enable via "-dd"
Allows for quickly testing the streaming write throughput.
2017-06-11 12:05:59 +02:00
Jakob Unterwurzacher
b5358ea623 performance.txt: add numbers for latest change
Also, get rid of the half-empty line.
2017-06-09 22:13:23 +02:00
Jakob Unterwurzacher
80516ed335 cryptocore: prefetch nonces in 512-byte blocks
On my machine, reading 512-byte blocks from /dev/urandom
(same via getentropy syscall) is a lot faster in terms of
throughput:

Blocksize    Throughput
 16          28.18 MB/s
512          83.75 MB/s

For a single-threaded streaming write, this drops the CPU usage of
nonceGenerator.Get to almost 1/3:

        flat  flat%   sum%        cum   cum%
Before     0     0% 95.08%      0.35s  2.92%  github.com/rfjakob/gocryptfs/internal/cryptocore.(*nonceGenerator).Get
After  0.01s 0.092% 92.34%      0.13s  1.20%  github.com/rfjakob/gocryptfs/internal/cryptocore.(*nonceGenerator).Get

This change makes the nonce reading single-threaded, which may
hurt massively-parallel writes.
2017-06-09 22:05:14 +02:00
Charles Duffy
da1bd74246 Fix missing Owner coercion for already-open files (#117) 2017-06-09 22:04:56 +02:00
Jakob Unterwurzacher
d2be22a07f cryptocore: remove lastNonce check
This check would need locking to be multithreading-safe.
But as it is in the fastpath, just remove it.
rand.Read() already guarantees that the value is random.
2017-06-07 23:08:43 +02:00
Jakob Unterwurzacher
294628b384 contentenc: move EncryptBlocks() loop into its own functions
This allows easy parallelization in the future.
2017-06-07 22:09:15 +02:00
Jakob Unterwurzacher
71978ec88a Add "-trace" flag (record execution trace)
Uses the runtime/trace functionality.

TODO: add to man page.
2017-06-07 22:09:06 +02:00
Jakob Unterwurzacher
22820bcd76 main: reorder force_owner flag parsing
No functional changes, just keeping the profiling-related flags
together.
2017-06-07 22:07:56 +02:00
Jakob Unterwurzacher
0ac5e44137 Add "package-static.bash" helper script
Creates a tar.gz with a static build of gocryptfs and the man page.
2017-06-04 19:25:08 +02:00
Jakob Unterwurzacher
d8d7c5c4fa Remove obsolete TODO.md
This file has been obsolete for some time,
TODOs are no longer tracked here but on github.
2017-06-04 19:04:23 +02:00
Jakob Unterwurzacher
b3b10273b5 golint: get rid of underscore warnings for good.
We got another warning for force_other:

	cli_args.go:26:45: don't use underscores in Go names; struct field force_owner should be forceOwner

Use a broader grep.
2017-06-01 22:19:46 +02:00
Jakob Unterwurzacher
a24faa3ba5 fusefrontend: write: consolidate and move encryption to contentenc
Collect all the plaintext and pass everything to contentenc in
one call.

This will allow easier parallization of the encryption.

https://github.com/rfjakob/gocryptfs/issues/116
2017-06-01 22:19:27 +02:00
Jakob Unterwurzacher
53b7c17261 Don't cap GOMAXPROCS at 4.
Before Go 1.5, GOMAXPROCS defaulted to 1, hence it made
sense to unconditionally increase it to 4.

But since Go 1.5, GOMAXPROCS defaults to the number of cores,
so don't keep it from increasing above 4.

Also, update the performance numbers.
2017-06-01 20:55:13 +02:00
Jakob Unterwurzacher
f44902aaae Fix two comments
One out-of-date and the other with a typo.
2017-06-01 18:53:57 +02:00
Jakob Unterwurzacher
f91ce0b004 main: increase max write size to maximum
Previously, it was at the go-fuse default of 64KiB. Getting
bigger writes should increase throughput somewhat.

Testing on tmpfs shows an improvement from 112MiB/s to 120MiB/s.
2017-06-01 18:52:02 +02:00
Charles Duffy
cf1ded5236 Implement force_owner option to display ownership as a specific user. 2017-06-01 00:26:17 +02:00
Jakob Unterwurzacher
fc2a5f5ab0 pathiv: fix test failure on Go 1.6
Travis failed on Go 1.6.3 with this error:

	internal/pathiv/pathiv_test.go:20: no args in Error call

This change should solve the problem and provides a better error
message on (real) test failure.
2017-05-31 08:21:36 +02:00
Jakob Unterwurzacher
1e598e96fc main: add "-info" option
Pretty-prints the config while stripping out sensitive
(and uninteresting) data

https://github.com/rfjakob/gocryptfs/issues/111
2017-05-30 19:01:32 +02:00
Jakob Unterwurzacher
ca5f469ea3 README: update changelog with short help text 2017-05-30 18:01:03 +02:00
Jakob Unterwurzacher
df2f4b1c40 main: add short help text
We have accumulated so many options over time that they
no longer fit on the screen.

Display only a useful subset of options to the user unless
they pass "-hh".
2017-05-30 17:59:13 +02:00
Jakob Unterwurzacher
9a217ce786 pathiv: move block IV algorithm into this package
This was implemented in fusefrontend_reverse, but we need it
in fusefrontend as well. Move the algorithm into pathiv.BlockIV().
2017-05-30 17:04:46 +02:00