hardcoresushi/libgocryptfs
1
0
Fork 0
Code Issues Pull Requests Releases Activity
1,971 Commits 2 Branches 0 Tags 6 MiB
985d852343
Commit Graph

1971 Commits

This Branch
This Branch
All Branches
Author SHA1 Message Date
Jakob Unterwurzacher
12c0101a23 contentenc: add PReqPool and use it in DecryptBlocks 
This gets us a massive speed boost in streaming reads.
 2017-06-30 23:30:57 +02:00
Jakob Unterwurzacher
e4b5005bcc stupidgcm: Open: if "dst" is big enough, use it as the output buffer 
This means we won't need any allocation for the plaintext.
 2017-06-30 23:24:12 +02:00
Jakob Unterwurzacher
b2a23e94d1 fusefrontend: doRead: use CReqPool for ciphertext buffer 
Easily saves lots of allocations.
 2017-06-30 23:15:31 +02:00
Jakob Unterwurzacher
06398e82d9 fusefrontend: Read: use provided buffer 
This will allow us to return internal buffers to a pool.
 2017-06-30 23:11:38 +02:00
Jakob Unterwurzacher
2932a285aa Update performance.txt with new numbers 2017-06-29 23:45:39 +02:00
Jakob Unterwurzacher
80676c685f contentenc: add safer "bPool" pool variant; add pBlockPool 
bPool verifies the lengths of slices going in and out.

Also, add a plaintext block pool - pBlockPool - and use
it for decryption.
 2017-06-29 23:44:32 +02:00
Jakob Unterwurzacher
3d32bcd37b profiling: fix hardcoded path in hint 2017-06-29 19:20:34 +02:00
Jakob Unterwurzacher
dee88f3c4d Update performance.txt with new numbers 2017-06-29 19:00:16 +02:00
Jakob Unterwurzacher
0cc6f53496 stupidgcm: use "dst" as the output buffer it is big enough 
This saves an allocation of the ciphertext block.
 2017-06-29 18:52:33 +02:00
Jakob Unterwurzacher
c12a50d4c8 benchmarks: add streaming read benchmark 2017-06-27 00:04:58 +02:00
Jakob Unterwurzacher
b9b52854c3 profiling: add streaming read profiling helper 
Reads 1GB of zeros while collecting memory and cpu profiles.
 2017-06-24 15:52:24 +02:00
Jakob Unterwurzacher
5c7b5770ce Update performance numbers 2017-06-20 21:46:27 +02:00
Jakob Unterwurzacher
3c6fe98eb1 contentenc: use sync.Pool memory pools for encryption 
We use two levels of buffers:

1) 4kiB+overhead for each ciphertext block
2) 128kiB+overhead for each FUSE write (32 ciphertext blocks)

This commit adds a sync.Pool for both levels.

The memory-efficiency for small writes could be improved,
as we now always use a 128kiB buffer.
 2017-06-20 21:22:00 +02:00
Jakob Unterwurzacher
609343accf README: update changelog for v1.4 2017-06-20 19:56:38 +02:00
Jakob Unterwurzacher
bfe421b327 MANPAGE: reorder options to match "-hh" output; add "-hkdf", "-trace" 2017-06-20 19:49:18 +02:00
Jakob Unterwurzacher
f3965a4e4c build.bash: use plain "git describe" for go-fuse 
go-fuse recently added a git tag - let's use it.
 2017-06-20 18:59:48 +02:00
Jakob Unterwurzacher
c9c4bc0141 profiling: add streaming-write profiling helper 
Writes 1GB of zeros to a gocryptfs mount while collecting
cpu and memory profiles.
 2017-06-18 22:56:50 +02:00
Jakob Unterwurzacher
a4563e21ec main, syscallcompat: use Dup3 instead of Dup2 
Dup2 is not implemented on linux/arm64.

Fixes https://github.com/rfjakob/gocryptfs/issues/121 .

Also adds cross-compilation to CI.
 2017-06-18 15:43:22 +02:00
Jakob Unterwurzacher
afc3a8252b Add performance numbers for v1.3-69-ge52594d 2017-06-11 21:58:01 +02:00
Jakob Unterwurzacher
e52594dae6 contentenc: parallelize encryption for 128kiB writes 
128kiB = 32 x 4kiB pages is the maximum we get from the kernel. Splitting
up smaller writes is probably not worth it.

Parallelism is limited to two for now.
 2017-06-11 21:56:16 +02:00
Jakob Unterwurzacher
24a7b1b7b8 Add performance numbers for last change 
Slight streaming write improvement.
 2017-06-11 21:44:24 +02:00
Jakob Unterwurzacher
9837cb0ddc cryptocore: prefetch nonces in the background 
Spawn a worker goroutine that reads the next 512-byte block
while the current one is being drained.

This should help reduce waiting times when /dev/urandom is very
slow (like on Linux 3.16 kernels).
 2017-06-11 21:29:50 +02:00
Jakob Unterwurzacher
f351c3c1ec benchmark.bash: add dd-only mode, enable via "-dd" 
Allows for quickly testing the streaming write throughput.
 2017-06-11 12:05:59 +02:00
Jakob Unterwurzacher
b5358ea623 performance.txt: add numbers for latest change 
Also, get rid of the half-empty line.
 2017-06-09 22:13:23 +02:00
Jakob Unterwurzacher
80516ed335 cryptocore: prefetch nonces in 512-byte blocks 
On my machine, reading 512-byte blocks from /dev/urandom
(same via getentropy syscall) is a lot faster in terms of
throughput:

Blocksize    Throughput
 16          28.18 MB/s
512          83.75 MB/s

For a single-threaded streaming write, this drops the CPU usage of
nonceGenerator.Get to almost 1/3:

        flat  flat%   sum%        cum   cum%
Before     0     0% 95.08%      0.35s  2.92%  github.com/rfjakob/gocryptfs/internal/cryptocore.(*nonceGenerator).Get
After  0.01s 0.092% 92.34%      0.13s  1.20%  github.com/rfjakob/gocryptfs/internal/cryptocore.(*nonceGenerator).Get

This change makes the nonce reading single-threaded, which may
hurt massively-parallel writes.
 2017-06-09 22:05:14 +02:00
Charles Duffy
da1bd74246 Fix missing Owner coercion for already-open files (#117) 2017-06-09 22:04:56 +02:00
Jakob Unterwurzacher
d2be22a07f cryptocore: remove lastNonce check 
This check would need locking to be multithreading-safe.
But as it is in the fastpath, just remove it.
rand.Read() already guarantees that the value is random.
 2017-06-07 23:08:43 +02:00
Jakob Unterwurzacher
294628b384 contentenc: move EncryptBlocks() loop into its own functions 
This allows easy parallelization in the future.
 2017-06-07 22:09:15 +02:00
Jakob Unterwurzacher
71978ec88a Add "-trace" flag (record execution trace) 
Uses the runtime/trace functionality.

TODO: add to man page.
 2017-06-07 22:09:06 +02:00
Jakob Unterwurzacher
22820bcd76 main: reorder force_owner flag parsing 
No functional changes, just keeping the profiling-related flags
together.
 2017-06-07 22:07:56 +02:00
Jakob Unterwurzacher
0ac5e44137 Add "package-static.bash" helper script 
Creates a tar.gz with a static build of gocryptfs and the man page.
 2017-06-04 19:25:08 +02:00
Jakob Unterwurzacher
d8d7c5c4fa Remove obsolete TODO.md 
This file has been obsolete for some time,
TODOs are no longer tracked here but on github.
 2017-06-04 19:04:23 +02:00
Jakob Unterwurzacher
b3b10273b5 golint: get rid of underscore warnings for good. 
We got another warning for force_other:

	cli_args.go:26:45: don't use underscores in Go names; struct field force_owner should be forceOwner

Use a broader grep.
 2017-06-01 22:19:46 +02:00
Jakob Unterwurzacher
a24faa3ba5 fusefrontend: write: consolidate and move encryption to contentenc 
Collect all the plaintext and pass everything to contentenc in
one call.

This will allow easier parallization of the encryption.

https://github.com/rfjakob/gocryptfs/issues/116
 2017-06-01 22:19:27 +02:00
Jakob Unterwurzacher
53b7c17261 Don't cap GOMAXPROCS at 4. 
Before Go 1.5, GOMAXPROCS defaulted to 1, hence it made
sense to unconditionally increase it to 4.

But since Go 1.5, GOMAXPROCS defaults to the number of cores,
so don't keep it from increasing above 4.

Also, update the performance numbers.
 2017-06-01 20:55:13 +02:00
Jakob Unterwurzacher
f44902aaae Fix two comments 
One out-of-date and the other with a typo.
 2017-06-01 18:53:57 +02:00
Jakob Unterwurzacher
f91ce0b004 main: increase max write size to maximum 
Previously, it was at the go-fuse default of 64KiB. Getting
bigger writes should increase throughput somewhat.

Testing on tmpfs shows an improvement from 112MiB/s to 120MiB/s.
 2017-06-01 18:52:02 +02:00
Charles Duffy
cf1ded5236 Implement force_owner option to display ownership as a specific user. 2017-06-01 00:26:17 +02:00
Jakob Unterwurzacher
fc2a5f5ab0 pathiv: fix test failure on Go 1.6 
Travis failed on Go 1.6.3 with this error:

	internal/pathiv/pathiv_test.go:20: no args in Error call

This change should solve the problem and provides a better error
message on (real) test failure.
 2017-05-31 08:21:36 +02:00
Jakob Unterwurzacher
1e598e96fc main: add "-info" option 
Pretty-prints the config while stripping out sensitive
(and uninteresting) data

https://github.com/rfjakob/gocryptfs/issues/111
 2017-05-30 19:01:32 +02:00
Jakob Unterwurzacher
ca5f469ea3 README: update changelog with short help text 2017-05-30 18:01:03 +02:00
Jakob Unterwurzacher
df2f4b1c40 main: add short help text 
We have accumulated so many options over time that they
no longer fit on the screen.

Display only a useful subset of options to the user unless
they pass "-hh".
 2017-05-30 17:59:13 +02:00
Jakob Unterwurzacher
9a217ce786 pathiv: move block IV algorithm into this package 
This was implemented in fusefrontend_reverse, but we need it
in fusefrontend as well. Move the algorithm into pathiv.BlockIV().
 2017-05-30 17:04:46 +02:00
Jakob Unterwurzacher
e43eb36da3 tests: add v1.3-reverse example filesystem 
We check the md5 sum of the encrypted version of a file to make sure we don't
accidentially change the ciphertext generation.
 2017-05-30 17:04:46 +02:00
Jakob Unterwurzacher
d202a456f5 pathiv: move derivedIVContainer into the package 
...under the new name "FileIVs".

This will also be used by forward mode.
 2017-05-30 17:04:46 +02:00
Jakob Unterwurzacher
857507e8b1 fusefrontend_reverse: move pathiv to its own package 
We will also need it in forward mode.
 2017-05-30 17:04:46 +02:00
Jakob Unterwurzacher
4d2cc551cf package.bash: RHEL: strip brackets 
We now convert "CentOS release 5.11 (Final)"
to             "CentOS_release_5.11_Final"

https://github.com/rfjakob/gocryptfs/issues/113
 2017-05-29 08:42:01 +02:00
Jakob Unterwurzacher
d59e7da6a6 gocryptfs-xray: dumpmasterkey: disable "Reading password from stdin" 
...and also exit with the proper exit code when we get an error.
 2017-05-28 17:43:09 +02:00
Jakob Unterwurzacher
7f5ae5f843 gocryptfs-xray: add function to dump the master key 
Fixes https://github.com/rfjakob/gocryptfs/issues/83
 2017-05-28 17:11:50 +02:00
Jakob Unterwurzacher
c36a55e985 package.bash: make it work somewhat on RHEL 
RHEL and CentOS do not have /etc/os-release yet. Read from
/etc/redhat-release instead.

Fixes https://github.com/rfjakob/gocryptfs/issues/113
 2017-05-27 15:22:27 +02:00