hardcoresushi/libgocryptfs
1
0
Fork 0
Code Issues Pull Requests Releases Activity
1,886 Commits 2 Branches 0 Tags 6 MiB
b3d26b7264
Commit Graph

1886 Commits

This Branch
This Branch
All Branches
Author SHA1 Message Date
Jakob Unterwurzacher
7fc93ec093 main: fix USR1 race condition, fixes test flakiness 
We could have gotten the signal before the handler was ready,
which caused the process to wait indefinitely. This was the reason
test.bash sometimes hung.
 2016-11-26 18:18:14 +01:00
Jakob Unterwurzacher
3f5c48e058 tests: don't call t.Fatal in checkExampleFS 
Calling t.Fatal means that the parent test has no chance
to clean up.
 2016-11-26 15:20:54 +01:00
Jakob Unterwurzacher
a6006c0d2b test.bash: lock against multiple parallel incovations 
Running multiple test.bash in parallel causes all kinds of
mayham.
 2016-11-26 13:43:50 +01:00
Jakob Unterwurzacher
55ea345620 Add write coalescing and -noprealloc to changelog 2016-11-26 13:09:05 +01:00
Jakob Unterwurzacher
cb5426e8ee performance.txt: add numbers for current gocryptfs master and encfs v1.9.1 2016-11-26 12:46:58 +01:00
Jakob Unterwurzacher
10884603d8 benchmark.bash: double write length 
Writing 1000 128KB blocks takes only 1 second and yielded
inconsistent results. With 2000, things look saner.
 2016-11-26 12:36:55 +01:00
Jakob Unterwurzacher
0f8d3318a3 main, fusefrontend: add "-noprealloc" option 
Preallocation is very slow on hdds that run btrfs. Give the
user the option to disable it. This greatly speeds up small file
operations but reduces the robustness against out-of-space errors.

Also add the option to the man page.

More info: https://github.com/rfjakob/gocryptfs/issues/63
 2016-11-25 09:19:14 +01:00
Jakob Unterwurzacher
024511d9c7 fusefrontend: coalesce 4kB writes 
This improves performance on hdds running ext4, and improves
streaming write performance on hdds running btrfs. Tar extract
slows down on btrfs for some reason.

See https://github.com/rfjakob/gocryptfs/issues/63

Benchmarks:

encfs v1.9.1
============

$ ./benchmark.bash -encfs /mnt/hdd-ext4
Testing EncFS at /mnt/hdd-ext4/benchmark.bash.u0g
WRITE: 131072000 bytes (131 MB, 125 MiB) copied, 1,48354 s, 88,4 MB/s
UNTAR: 20.79
LS:    3.04
RM:    6.62

$ ./benchmark.bash -encfs /mnt/hdd-btrfs
Testing EncFS at /mnt/hdd-btrfs/benchmark.bash.h40
WRITE: 131072000 bytes (131 MB, 125 MiB) copied, 1,52552 s, 85,9 MB/s
UNTAR: 24.51
LS:    2.73
RM:    5.32

gocryptfs v1.1.1-26-g4a7f8ef
============================

$ ./benchmark.bash /mnt/hdd-ext4
Testing gocryptfs at /mnt/hdd-ext4/benchmark.bash.1KG
WRITE: 131072000 bytes (131 MB, 125 MiB) copied, 1,55782 s, 84,1 MB/s
UNTAR: 22.23
LS:    1.47
RM:    4.17

$ ./benchmark.bash /mnt/hdd-btrfs
Testing gocryptfs at /mnt/hdd-btrfs/benchmark.bash.2t8
WRITE: 131072000 bytes (131 MB, 125 MiB) copied, 6,87206 s, 19,1 MB/s
UNTAR: 69.87
LS:    1.52
RM:    5.33

gocryptfs v1.1.1-32
===================

$ ./benchmark.bash /mnt/hdd-ext4
Testing gocryptfs at /mnt/hdd-ext4/benchmark.bash.Qt3
WRITE: 131072000 bytes (131 MB, 125 MiB) copied, 1,22577 s, 107 MB/s
UNTAR: 23.46
LS:    1.46
RM:    4.67

$ ./benchmark.bash /mnt/hdd-btrfs/
Testing gocryptfs at /mnt/hdd-btrfs//benchmark.bash.XVk
WRITE: 131072000 bytes (131 MB, 125 MiB) copied, 3,68735 s, 35,5 MB/s
UNTAR: 116.87
LS:    1.84
RM:    6.34
 2016-11-25 09:03:32 +01:00
Jakob Unterwurzacher
80c50b9dbc main: give deferred functions a chance to run (fixes -cpuprofile) 2016-11-23 23:49:34 +01:00
Jakob Unterwurzacher
910fee244f benchmark.bash: allow to specify directory 2016-11-23 21:54:55 +01:00
Jakob Unterwurzacher
25d06fc809 Drop "benchmark-synthetic.bash" 
This scipt was broken for a long time and not very useful.
 2016-11-23 21:54:13 +01:00
Jakob Unterwurzacher
6f475da116 Fix golint warnings, add helper script 2016-11-17 23:40:03 +01:00
Jakob Unterwurzacher
36b4e325ca README: start list for v1.2 2016-11-17 23:34:16 +01:00
Jakob Unterwurzacher
41af353dcb tests: rename fsstress-loopback.bash to fsstress-gocryptfs.bash 
The primary use is testing gocryptfs, after all.
 2016-11-17 22:51:25 +01:00
Jakob Unterwurzacher
1bae06a16a tests: add OpenTruncateRead test 
This is a regression test for the issue that was fixed by the
last commit.
 2016-11-17 22:48:50 +01:00
Jakob Unterwurzacher
0489d08ae2 fusefrontend: get the file ID from the open files table 
This fixes the problem that a truncate can reset the file
ID without the other open FDs noticing it.
 2016-11-17 22:29:45 +01:00
Jakob Unterwurzacher
e04dc05012 fusefrontend: upgrade wlockMap to use device AND inode number 
If there are multiple filesystems backing the gocryptfs filesystems
inode numbers are not guaranteed to be unique.
 2016-11-17 20:32:19 +01:00
Jakob Unterwurzacher
081015aa74 MANPAGE: document ctlsock 2016-11-11 00:01:29 +01:00
Jakob Unterwurzacher
d8fb28a1c3 ctlsock: prevent panic on invalid decrypt request 2016-11-10 23:51:47 +01:00
Jakob Unterwurzacher
c2629bd9b5 tests: allow overriding hardcoded arguments in Mount helper 2016-11-10 23:47:04 +01:00
Jakob Unterwurzacher
c03fc46a51 ctlsock: implement EncryptPath for reverse mode, add tests 2016-11-10 23:32:51 +01:00
Jakob Unterwurzacher
d3764b7753 reverse: fix longname decoding bug 
This could have caused spurious ENOENT errors.

That it did not cause these errors all the time is interesting
and probably because an earlier readdir would place the entry
in the cache. This masks the bug.
 2016-11-10 23:30:30 +01:00
Jakob Unterwurzacher
e7f57695a6 Fix golint warnings 
$ golint ./... | grep -v underscore | grep -v ALL_CAPS
internal/fusefrontend_reverse/rfs.go:52:36: exported func NewFS returns unexported type *fusefrontend_reverse.reverseFS, which can be annoying to use
internal/nametransform/raw64_go1.5.go:10:2: exported const HaveRaw64 should have comment (or a comment on this block) or be unexported
 2016-11-10 00:38:01 +01:00
Jakob Unterwurzacher
75ebb28a62 ctlsock: add initial limited implementation 
At the moment, in forward mode you can only encrypt paths
and in reverse mode you can only decrypt paths.
 2016-11-10 00:27:08 +01:00
Jakob Unterwurzacher
df28fc5a11 nametransform: get rid of leading "./" 
Paths in the root directory were encrypted to this:

    foobar -> ./N9vPc0gXUY4PDSt0-muYXQ==
 2016-11-09 23:41:47 +01:00
Jakob Unterwurzacher
df1e3a10c4 nametransform: nicer error message on empty gocryptfs.diriv 
Old:

	Nov 06 13:34:38 brikett gocryptfs[16228]: ReadDirIVAt: Read failed: EOF
	Nov 06 13:34:38 brikett gocryptfs[16228]: go-fuse: can't convert error type: EOF

New:

	Nov 06 14:08:43 brikett gocryptfs[17361]: ReadDirIVAt: wanted 16 bytes, got 0. Returning EINVAL.
 2016-11-06 14:09:34 +01:00
Jakob Unterwurzacher
d15122d3d6 Add Go 1.4 compatibility layer for raw64 
Using raw64 will not work, but at least it will compile.
 2016-11-01 19:25:59 +01:00
Jakob Unterwurzacher
f4c367381e main: fix comments that reference "-f" 2016-11-01 19:04:49 +01:00
Jakob Unterwurzacher
b527e205e2 main: rename "-f" to "-fg" 
"-f" looks too much like "--force". The old variant is still
accepted for compatability.
 2016-11-01 19:00:45 +01:00
Jakob Unterwurzacher
d6678f73b4 MANPAGE: document -raw64 2016-11-01 18:53:42 +01:00
Jakob Unterwurzacher
2b991c9743 Add support for unpadded base64 filenames, "-raw64" 
Through base64.RawURLEncoding.

New command-line parameter "-raw64".
 2016-11-01 18:43:22 +01:00
Jakob Unterwurzacher
964e0e6b36 tests: add raw64 tests 
Also, use "%#v" instead of JSON for debug output.
This means we can unexport all fields.
 2016-11-01 18:42:10 +01:00
Jakob Unterwurzacher
d41492bcbc MANPAGE: update version field description 2016-11-01 18:30:32 +01:00
Jakob Unterwurzacher
8ce735e987 main: print "args" slightly prettier 2016-11-01 18:30:00 +01:00
Jakob Unterwurzacher
75b776cb3d MANPAGE: add reverse example, move "-o" into alphabetical list 
People will search for "-o" alphabetically, so put it into the
alphabetical option list, even if it is not a real option.
 2016-11-01 15:55:05 +01:00
Jakob Unterwurzacher
e993fd5fbc packakge.bash: include rendered man page in tarball 2016-11-01 15:43:33 +01:00
Jakob Unterwurzacher
944eaf2fb5 fusefronted: expire dir IV cache after one second 
The Back In Time backup tool (https://github.com/bit-team/backintime)
wants to write directly into the ciphertext dir.
This may cause the cached directory IV to become out-of-date.
Having an expiry time limits the inconstency to one second, like
attr_timeout does for the kernel getattr cache.
 2016-11-01 10:57:39 +01:00
Jakob Unterwurzacher
a9c7565b80 fusefrontend: refactor dirIVCache 
Simplify the code a bit.
 2016-11-01 10:34:41 +01:00
Jakob Unterwurzacher
fd88dbd687 README: fix "15#" typo 2016-10-30 16:57:22 +01:00
Jakob Unterwurzacher
1a847b8f00 Update README for v1.1.1 2016-10-30 16:48:02 +01:00
Jakob Unterwurzacher
c2192cfcad fusefrontend: drop atime workarounds 
The fix at https://github.com/hanwen/go-fuse/pull/131 has been merged.
Drop the workarounds and re-enable the tests.
 2016-10-30 16:29:36 +01:00
Jakob Unterwurzacher
4e95d394cf build.bash: rename "GOPATH2" to "GOPATH1" 
It's the 1st component of GOPATH, so call it like that.
 2016-10-30 16:17:39 +01:00
Jakob Unterwurzacher
c7385b3681 build.bash: abort if we hit a known-bad go-fuse version 
Also, standardize to "if [[ ]] ; then" style.
 2016-10-30 16:17:00 +01:00
Jakob Unterwurzacher
85f1fd0b0f fusefronted: more concise corrupt block log message 
Calculating the block offset is easy enough, even more now
that gocryptfs-xray exists.
 2016-10-28 21:18:36 +02:00
Jakob Unterwurzacher
012152f3d1 fusefrontend: I/O error instead of panic on all-zero nonce 
Running xfstests generic/075 on tmpfs often triggered a panic
for what seems to be a tmpfs bug.

Quoting from the email to lkml,
http://www.spinics.net/lists/kernel/msg2370127.html :

	tmpfs seems to be incorrectly returning 0-bytes when reading from
	a file that is concurrently being truncated.
 2016-10-28 21:18:36 +02:00
Jakob Unterwurzacher
4cc64c2c95 main: redirect panics to /tmp/gocryptfs_panic.NNNN 
Redirect stdout and stderr to /tmp/gocryptfs_paniclog.NNNNNN
instead of closing them so users have a chance to get the
backtrace on a panic.

This only applies if "-nosyslog" is NOT set. Panics will
go to terminal as usual if it is.
 2016-10-28 21:18:36 +02:00
Jakob Unterwurzacher
a08d55f42d fusefronted: optimize NFS streaming writes by saving one Stat() 
Stat() calls are expensive on NFS as they need a full network
round-trip. We detect when a write immediately follows the
last one and skip the Stat in this case because the write
cannot create a file hole.

On my (slow) NAS, this takes the write speed from 24MB/s to
41MB/s.
 2016-10-28 21:17:53 +02:00
Jakob Unterwurzacher
9b7135224b tests: fetch the example tarball from cdn.kernel.org 
www.kernel.org is painfully slow at times.
 2016-10-25 23:13:44 +02:00
Jakob Unterwurzacher
d64ccf7cf4 fusefrontend: move hole padding check out of Write() 
The details of the hole handling don't have to be in
Write, so move it away.
 2016-10-25 22:37:45 +02:00
Jakob Unterwurzacher
6538dc15af fusefrontend: rename "createsHole" to clearer "createsCiphertextHole" 
...and add comments for what is happening.
 2016-10-25 21:19:37 +02:00