hardcoresushi/libgocryptfs
1
0
Fork 0
Code Issues Pull Requests Releases Activity
632 Commits 2 Branches 0 Tags 6 MiB
cb5426e8ee
Commit Graph

168 Commits

Author SHA1 Message Date
Jakob Unterwurzacher
0f8d3318a3 main, fusefrontend: add "-noprealloc" option 
Preallocation is very slow on hdds that run btrfs. Give the
user the option to disable it. This greatly speeds up small file
operations but reduces the robustness against out-of-space errors.

Also add the option to the man page.

More info: https://github.com/rfjakob/gocryptfs/issues/63
 2016-11-25 09:19:14 +01:00
Jakob Unterwurzacher
024511d9c7 fusefrontend: coalesce 4kB writes 
This improves performance on hdds running ext4, and improves
streaming write performance on hdds running btrfs. Tar extract
slows down on btrfs for some reason.

See https://github.com/rfjakob/gocryptfs/issues/63

Benchmarks:

encfs v1.9.1
============

$ ./benchmark.bash -encfs /mnt/hdd-ext4
Testing EncFS at /mnt/hdd-ext4/benchmark.bash.u0g
WRITE: 131072000 bytes (131 MB, 125 MiB) copied, 1,48354 s, 88,4 MB/s
UNTAR: 20.79
LS:    3.04
RM:    6.62

$ ./benchmark.bash -encfs /mnt/hdd-btrfs
Testing EncFS at /mnt/hdd-btrfs/benchmark.bash.h40
WRITE: 131072000 bytes (131 MB, 125 MiB) copied, 1,52552 s, 85,9 MB/s
UNTAR: 24.51
LS:    2.73
RM:    5.32

gocryptfs v1.1.1-26-g4a7f8ef
============================

$ ./benchmark.bash /mnt/hdd-ext4
Testing gocryptfs at /mnt/hdd-ext4/benchmark.bash.1KG
WRITE: 131072000 bytes (131 MB, 125 MiB) copied, 1,55782 s, 84,1 MB/s
UNTAR: 22.23
LS:    1.47
RM:    4.17

$ ./benchmark.bash /mnt/hdd-btrfs
Testing gocryptfs at /mnt/hdd-btrfs/benchmark.bash.2t8
WRITE: 131072000 bytes (131 MB, 125 MiB) copied, 6,87206 s, 19,1 MB/s
UNTAR: 69.87
LS:    1.52
RM:    5.33

gocryptfs v1.1.1-32
===================

$ ./benchmark.bash /mnt/hdd-ext4
Testing gocryptfs at /mnt/hdd-ext4/benchmark.bash.Qt3
WRITE: 131072000 bytes (131 MB, 125 MiB) copied, 1,22577 s, 107 MB/s
UNTAR: 23.46
LS:    1.46
RM:    4.67

$ ./benchmark.bash /mnt/hdd-btrfs/
Testing gocryptfs at /mnt/hdd-btrfs//benchmark.bash.XVk
WRITE: 131072000 bytes (131 MB, 125 MiB) copied, 3,68735 s, 35,5 MB/s
UNTAR: 116.87
LS:    1.84
RM:    6.34
 2016-11-25 09:03:32 +01:00
Jakob Unterwurzacher
6f475da116 Fix golint warnings, add helper script 2016-11-17 23:40:03 +01:00
Jakob Unterwurzacher
0489d08ae2 fusefrontend: get the file ID from the open files table 
This fixes the problem that a truncate can reset the file
ID without the other open FDs noticing it.
 2016-11-17 22:29:45 +01:00
Jakob Unterwurzacher
e04dc05012 fusefrontend: upgrade wlockMap to use device AND inode number 
If there are multiple filesystems backing the gocryptfs filesystems
inode numbers are not guaranteed to be unique.
 2016-11-17 20:32:19 +01:00
Jakob Unterwurzacher
d8fb28a1c3 ctlsock: prevent panic on invalid decrypt request 2016-11-10 23:51:47 +01:00
Jakob Unterwurzacher
c03fc46a51 ctlsock: implement EncryptPath for reverse mode, add tests 2016-11-10 23:32:51 +01:00
Jakob Unterwurzacher
d3764b7753 reverse: fix longname decoding bug 
This could have caused spurious ENOENT errors.

That it did not cause these errors all the time is interesting
and probably because an earlier readdir would place the entry
in the cache. This masks the bug.
 2016-11-10 23:30:30 +01:00
Jakob Unterwurzacher
e7f57695a6 Fix golint warnings 
$ golint ./... | grep -v underscore | grep -v ALL_CAPS
internal/fusefrontend_reverse/rfs.go:52:36: exported func NewFS returns unexported type *fusefrontend_reverse.reverseFS, which can be annoying to use
internal/nametransform/raw64_go1.5.go:10:2: exported const HaveRaw64 should have comment (or a comment on this block) or be unexported
 2016-11-10 00:38:01 +01:00
Jakob Unterwurzacher
75ebb28a62 ctlsock: add initial limited implementation 
At the moment, in forward mode you can only encrypt paths
and in reverse mode you can only decrypt paths.
 2016-11-10 00:27:08 +01:00
Jakob Unterwurzacher
df28fc5a11 nametransform: get rid of leading "./" 
Paths in the root directory were encrypted to this:

    foobar -> ./N9vPc0gXUY4PDSt0-muYXQ==
 2016-11-09 23:41:47 +01:00
Jakob Unterwurzacher
df1e3a10c4 nametransform: nicer error message on empty gocryptfs.diriv 
Old:

	Nov 06 13:34:38 brikett gocryptfs[16228]: ReadDirIVAt: Read failed: EOF
	Nov 06 13:34:38 brikett gocryptfs[16228]: go-fuse: can't convert error type: EOF

New:

	Nov 06 14:08:43 brikett gocryptfs[17361]: ReadDirIVAt: wanted 16 bytes, got 0. Returning EINVAL.
 2016-11-06 14:09:34 +01:00
Jakob Unterwurzacher
d15122d3d6 Add Go 1.4 compatibility layer for raw64 
Using raw64 will not work, but at least it will compile.
 2016-11-01 19:25:59 +01:00
Jakob Unterwurzacher
2b991c9743 Add support for unpadded base64 filenames, "-raw64" 
Through base64.RawURLEncoding.

New command-line parameter "-raw64".
 2016-11-01 18:43:22 +01:00
Jakob Unterwurzacher
944eaf2fb5 fusefronted: expire dir IV cache after one second 
The Back In Time backup tool (https://github.com/bit-team/backintime)
wants to write directly into the ciphertext dir.
This may cause the cached directory IV to become out-of-date.
Having an expiry time limits the inconstency to one second, like
attr_timeout does for the kernel getattr cache.
 2016-11-01 10:57:39 +01:00
Jakob Unterwurzacher
a9c7565b80 fusefrontend: refactor dirIVCache 
Simplify the code a bit.
 2016-11-01 10:34:41 +01:00
Jakob Unterwurzacher
c2192cfcad fusefrontend: drop atime workarounds 
The fix at https://github.com/hanwen/go-fuse/pull/131 has been merged.
Drop the workarounds and re-enable the tests.
 2016-10-30 16:29:36 +01:00
Jakob Unterwurzacher
85f1fd0b0f fusefronted: more concise corrupt block log message 
Calculating the block offset is easy enough, even more now
that gocryptfs-xray exists.
 2016-10-28 21:18:36 +02:00
Jakob Unterwurzacher
012152f3d1 fusefrontend: I/O error instead of panic on all-zero nonce 
Running xfstests generic/075 on tmpfs often triggered a panic
for what seems to be a tmpfs bug.

Quoting from the email to lkml,
http://www.spinics.net/lists/kernel/msg2370127.html :

	tmpfs seems to be incorrectly returning 0-bytes when reading from
	a file that is concurrently being truncated.
 2016-10-28 21:18:36 +02:00
Jakob Unterwurzacher
a08d55f42d fusefronted: optimize NFS streaming writes by saving one Stat() 
Stat() calls are expensive on NFS as they need a full network
round-trip. We detect when a write immediately follows the
last one and skip the Stat in this case because the write
cannot create a file hole.

On my (slow) NAS, this takes the write speed from 24MB/s to
41MB/s.
 2016-10-28 21:17:53 +02:00
Jakob Unterwurzacher
d64ccf7cf4 fusefrontend: move hole padding check out of Write() 
The details of the hole handling don't have to be in
Write, so move it away.
 2016-10-25 22:37:45 +02:00
Jakob Unterwurzacher
6538dc15af fusefrontend: rename "createsHole" to clearer "createsCiphertextHole" 
...and add comments for what is happening.
 2016-10-25 21:19:37 +02:00
Jakob Unterwurzacher
aeda9721d0 Fix misspellings 
Close https://github.com/rfjakob/gocryptfs/issues/54
 2016-10-24 19:18:13 +02:00
Jakob Unterwurzacher
a74a7e7f54 configfile: add newline at the end 
For convenience for the user, add a newline at the end.
Reported as #51 .
 2016-10-21 08:43:39 +02:00
Jakob Unterwurzacher
589748548f tests: add 1980.tar.gz extract test 
Test that we get the right timestamp when extracting a tarball.

Also simplify the workaround in doTestUtimesNano() and fix the
fact that it was running no test at all.
 2016-10-19 22:25:54 +02:00
Jakob Unterwurzacher
600ceece35 lint fixes 2016-10-19 01:12:45 +02:00
Jakob Unterwurzacher
891a3b4c8a fusefrontend: Utimens: one more band-aid 
Revert once https://github.com/hanwen/go-fuse/pull/131 is merged.
 2016-10-16 20:20:00 +02:00
Jakob Unterwurzacher
c487e176bd main: allow password change with -masterkey 
Requested at https://github.com/rfjakob/gocryptfs/issues/28
 2016-10-16 18:17:28 +02:00
Jakob Unterwurzacher
b2d33028a6 readpassword: check extpass return code 
...and exit if it is not zero.
 2016-10-16 16:44:27 +02:00
Jakob Unterwurzacher
5144470e3d fusefrontend: Utimens: ugly band-aid for nil pointer crash in go-fuse 
Crash is described at https://github.com/rfjakob/gocryptfs/issues/48 .
Revert this once https://github.com/hanwen/go-fuse/pull/131 is merged.
 2016-10-16 15:08:05 +02:00
Jakob Unterwurzacher
35219d0022 fusefrontend: log missing gocryptfs.diriv 
This can happen during normal operation when the directory has
been deleted concurrently. But it can also mean that the
gocryptfs.diriv is missing due to an error, so log the event
at "info" level.
 2016-10-16 15:04:59 +02:00
Jakob Unterwurzacher
828f718483 fusefrontend: Also preserve the owner in Mkdir 
This already worked for files but was missing for dirs.
 2016-10-10 08:53:29 +02:00
Jakob Unterwurzacher
d3b78fea95 reverse: add panics against API abuse 
These should help prevent later programming errors.
 2016-10-09 17:05:12 +02:00
Jakob Unterwurzacher
d25fcc6a4b reverse: gocryptfs.conf was missing from the directory listings 
Fix the test for that and add checks in example_filesystems_test.
 2016-10-08 22:25:08 +02:00
Jakob Unterwurzacher
e47577834b reverse: merge config translation check into isTranslatedConfig 
Also get rid of useless isFiltered function.
 2016-10-08 21:14:16 +02:00
Jakob Unterwurzacher
f054353bd3 reverse: make gocryptfs.conf mapping plaintextnames-aware 
Only in plaintextnames-mode AND with the config file at the
default location it will be mapped into the mountpoint.

Also adds a test for that.
 2016-10-08 20:57:38 +02:00
Jakob Unterwurzacher
53257f4ee5 nametransform: better error code on invalid diriv length 
go-fuse translates errors unknown to it into "function not
implemented", which is wrong in this case.
 2016-10-07 22:40:30 +02:00
Jakob Unterwurzacher
ff48dc1aab reverse: initialize the longname cache only when reverse mode is used 
Gets rid of the idling longnameCacheCleaner thread in "normal" mode.
 2016-10-05 22:22:28 +02:00
Jakob Unterwurzacher
a4956fa6bf A few more lint fixes 2016-10-04 23:30:05 +02:00
Valient Gough
b764917cd5 lint fixes 2016-10-04 23:18:33 +02:00
Jakob Unterwurzacher
db5782028a tests: skip tests with -openssl=false on Go 1.4 and lower 
Go versions 1.4 and lower lack NewGCMWithNonceSize(), which causes
a panic in the test.
 2016-10-04 22:34:12 +02:00
Romain
db72a4489d fusefrontend_reverse cast Stat_t.Dev to uint64 (#44) 
The [Stat_t.Dev](https://golang.org/pkg/syscall/#Stat_t) docs say `Dev` is a `unit64`, but on [macOS amd64](https://golang.org/src/syscall/ztypes_darwin_amd64.go) it's an `int32`.
 2016-10-04 15:03:47 +02:00
Jakob Unterwurzacher
56c0b19612 without_openssl: support compiling completely without openssl 
Build helper script: build-without-openssl.bash
 2016-10-04 09:51:14 +02:00
Jakob Unterwurzacher
a00402cc47 cryptocore: rename "gcm" variable to generic "aeadCipher" 
As we now also support AES-SIV the old name is no longer
correct.
 2016-10-04 00:17:09 +02:00
Jakob Unterwurzacher
bb52937834 cryptocore: derive 512-bit key for AES-SIV 
AES-SIV uses 1/2 of the key for authentication, 1/2 for
encryption, so we need a 64-byte key for AES-256. Derive
it from the master key by hashing it with SHA-512.
 2016-10-04 00:16:56 +02:00
Jakob Unterwurzacher
46018785b4 siv_aead: Overhead is always 16 
Add a test for that.

Also test operations using a 64-byte key.
 2016-10-03 23:58:19 +02:00
Jakob Unterwurzacher
a2510efe12 reverse: use per-purpose nonce generation 
Also pull all the deterministic nonce code into fusefrontend_reverse
to greatly simplify the normal code path.
 2016-09-29 21:56:49 +02:00
Jakob Unterwurzacher
bce96b5095 reverse: move newDirIVFile into virtualfile.go 
After all, is's a virtual file.
 2016-09-28 23:30:13 +02:00
Jakob Unterwurzacher
e9bb8b800c reverse: switch from GCM-SIV to AES-SIV 
GCM-SIV is not yet finalized, and the reference implemenation is
painfully slow at about 2 MB/s. Switch to AES-SIV.
 2016-09-26 23:25:13 +02:00
Jakob Unterwurzacher
d9fc652df0 siv_aead: add AES-SIV AEAD wrapper 2016-09-26 23:06:40 +02:00