Commit Graph

623 Commits

Author SHA1 Message Date
Jakob Unterwurzacher
6d64dfe8f7 Only print masterkey once on -init
It is no longer printed at all when mounting a filesystem,
printing on -init can be disabled with -q.

https://github.com/rfjakob/gocryptfs/issues/76
2018-07-01 20:56:22 +02:00
Jakob Unterwurzacher
991891a5c4 trezor: add sanity checks for decrypted value
Check that the value has changed, is not all-zero
and has the right length.
2018-07-01 20:56:22 +02:00
Dmitry Yu Okunev
978f1f3f6d Implemented the support of Trezor devices. 2018-07-01 20:56:22 +02:00
Jakob Unterwurzacher
9a15dfa494 trezor: add TrezorPayload
TrezorPayload stores 32 random bytes used for unlocking
the master key using a Trezor security module. The randomness makes sure
that a unique unlock value is used for each gocryptfs filesystem.
2018-07-01 20:56:22 +02:00
Jakob Unterwurzacher
91de77943f configfile: reduce function name stutter
configfile.LoadConfFile()   -> configfile.Load()
configfile.CreateConfFile() -> configfile.Create()
2018-07-01 20:56:22 +02:00
Jakob Unterwurzacher
c6f6e8ec4d trezor: add skeleton for Trezor support
readpassword.Trezor() is not implemented yet and returns
a hardcoded dummy key.
2018-07-01 20:56:04 +02:00
Jakob Unterwurzacher
743c7705b2 configfile: use tlog.ColorYellow instead of hardcoded color code 2018-06-24 19:52:47 +02:00
Jakob Unterwurzacher
1bab400fca Fix three golint warnings
We are clean again.

Warnings were:

internal/fusefrontend/fs.go:443:14: should omit type string from declaration
of var cTarget; it will be inferred from the right-hand side
internal/fusefrontend/xattr.go:26:1: comment on exported method FS.GetXAttr
should be of the form "GetXAttr ..."
internal/syscallcompat/sys_common.go:9:7: exported const PATH_MAX should have
comment or be unexported
2018-06-19 20:16:21 +02:00
Jakob Unterwurzacher
bfa50517e9 xattr: return EOPNOTSUPP instead of ENODATA in GetXattr
Reading system.posix_acl_access and system.posix_acl_default
should return EOPNOTSUPP to inform user-space that we do not
support ACLs.

xftestest essientially does

	chacl -l | grep "Operation not supported"

to determine if the filesystem supports ACLs, and used to
wrongly believe that gocryptfs does.
2018-06-12 23:05:53 +02:00
Jakob Unterwurzacher
bde7ba57b0 darwin does not have PATH_MAX
Define our own, with the value from Linux.
2018-06-08 00:47:48 +02:00
Jakob Unterwurzacher
ae02ca1ded xattr: use LGet/LSet etc
Support has been merged into the xattr package
( https://github.com/pkg/xattr/pull/29 ), use it.
2018-05-27 20:09:48 +02:00
Jakob Unterwurzacher
7b00681807 xattr: return EOPNOTSUPP for unsupported attributes
mv is unhappy when we return EPERM when it tries to set
system.posix_acl_access:

   mv: preserving permissions for ‘b/x’: Operation not permitted

Now we return EOPNOTSUPP like tmpfs does and mv seems happy.
2018-05-15 23:00:47 +02:00
Bolshevik
a41ec2028c xattr: optimize storage, store as binary instead of bae64
Values a binary-safe, there is no need to base64-encode them.

Old, base64-encoded values are supported transparently
on reading. Writing xattr values now always writes them binary.
2018-05-10 23:25:49 +02:00
Jakob Unterwurzacher
a276321dea stupidgcm: return error on too short input instead of panicing
This is what Go GCM does as well.
2018-05-10 23:00:02 +02:00
Bolshevik
5ccc06d5cb xattr: added passing of a "flags" parameter
Pass the "flags" parameter to the lower layer syscall.
This makes Apple applications being able to successfully save data.
2018-05-07 21:45:40 +02:00
Jakob Unterwurzacher
fe3890688a fusefrontend: xattr: return ENOSYS on unsupported flags
We previously returned EPERM to prevent the kernel from
blacklisting our xattr support once we get an unsupported
flag, but this causes lots of trouble on MacOS:
Cannot save files from GUI apps, see
https://github.com/rfjakob/gocryptfs/issues/229

Returning ENOSYS triggers the dotfiles fallback on MacOS
and fixes the issue.
2018-05-01 23:30:53 +02:00
bolshevik
12b32aa06c Improved xattr handling on non-linux systems (#227)
* Fixed xattr filtering for MacOS. "system." and "user." prefixes are only relevant for Linux.
* Small cleanup and additional tests.
2018-04-17 20:33:04 +02:00
Jakob Unterwurzacher
bcc8378a2c Fix the easy golint warnings
Reported by https://goreportcard.com/report/github.com/rfjakob/gocryptfs
2018-04-08 20:26:25 +02:00
Jakob Unterwurzacher
4e5783591f fsck: report skipped corrupt files
OpenDir and ListXAttr skip over corrupt entries,
readFileID treats files the are too small as empty.
This improves usability in the face of corruption,
but hides the problem in a log message instead of
putting it in the return code.

Create a channel to report these corruptions to fsck
so it can report them to the user.

Also update the manpage and the changelog with the -fsck option.

Closes https://github.com/rfjakob/gocryptfs/issues/191
2018-04-03 21:24:48 +02:00
Jakob Unterwurzacher
4407ca3a4d fusefrontend: xattr: return ENODATA for security.* and system.*
"ls -l" queries security.selinux, system.posix_acl_access, system.posix_acl_default
and throws error messages if it gets something else than ENODATA.
2018-04-02 18:59:14 +02:00
Jakob Unterwurzacher
a0fd3eca98 fsck: test against example_filesystems 2018-04-02 18:43:50 +02:00
Jakob Unterwurzacher
b6c8960b01 fsck: clean up log output
Make sure we get only 1 warning output per
problem.

Also, add new corruption types to broken_fs_v1.4.
2018-04-02 18:32:30 +02:00
Jakob Unterwurzacher
f28d85fad5 fsck: add initial implementation
Most corruption cases except xattr should be covered.
With test filesystem.

The output is still pretty ugly. xattr support will
be added in the next commits.
2018-04-02 16:38:18 +02:00
Jakob Unterwurzacher
fb06c65ee9 fusefronted: reject oversized Read and Write requests
This should not happen via FUSE as the kernel caps the size,
but with fsck we have the first user that calls Read directly.
For symmetry, check it for Write as well.
2018-04-01 21:21:55 +02:00
Jakob Unterwurzacher
1a3d04ab87 Switch from private copy to pkg/xattr
Now that https://github.com/pkg/xattr/pull/24
has been merged there is no reason to keep
our private copy.

Switch to the upstream version.
2018-03-28 19:19:58 +02:00
Jakob Unterwurzacher
db778aae7d fusefrontend: handle empty xattrs efficiently
We handle empty files by storing an actual empty file
on disk. Handle xattrs similarily and encrypt the
empty value to the empty value.
2018-03-25 21:06:10 +02:00
Jakob Unterwurzacher
1ed3d51df1 fusefrontend: add xattr support
At the moment, only for reverse mode.

https://github.com/rfjakob/gocryptfs/issues/217
2018-03-25 21:06:10 +02:00
Jakob Unterwurzacher
3d54fc3a3a fusefrontend: create helpers for symlink encryption
These will be reused by the upcoming xattr support.
2018-03-24 21:40:11 +01:00
Jakob Unterwurzacher
9bc039a4ba Add -masterkey=stdin functionality
https://github.com/rfjakob/gocryptfs/issues/218
2018-03-22 00:02:10 +01:00
Jakob Unterwurzacher
4732e33a9a macos: tests: fix deleting of scratch dir
macos rm does not understand --one-file-system,
and it cannot handle unreadable directories.
2018-03-05 23:40:08 +01:00
Jakob Unterwurzacher
7db5395c53 macos: fix second TestEmulateSymlinkat test failure 2018-03-05 21:20:07 +01:00
Jakob Unterwurzacher
c5243fc79e MacOS: don't test symlinks longer than 1000 bytes
The limit is much lower than on Linux.

https://github.com/rfjakob/gocryptfs/issues/213
2018-02-28 20:40:08 +01:00
Jakob Unterwurzacher
90f2fea7fb MacOS: fix TestEmulateSymlinkat test failure
On MacOS, symlinks don't have their own permissions,
so don't check for them.
2018-02-28 20:19:31 +01:00
Jakob Unterwurzacher
b96e3ee271 tlog: stop embedding log.Logger to prevent mistakes
A few places have called tlog.Warn.Print, which directly
calls into log.Logger due to embedding, losing all features
of tlog.

Stop embedding log.Logger to make sure the internal functions
cannot be called accidentially and fix (several!) instances
that did.
2018-02-28 09:02:18 +01:00
Jakob Unterwurzacher
db45f27671 ctlsock: don't Warn() on closed socket
This Warn() is causing panics in the test suite
on MacOS: https://github.com/rfjakob/gocryptfs/issues/213
2018-02-27 09:58:14 +01:00
Jakob Unterwurzacher
5ad9bda206 cryptocore: make AEADTypeEnum values explicit
We now print the number in a debug message, so define
the numeric values explicitely instead of using iota.

This way you don't have to understand how iota works
to find out what the number means. Lack of understanding
of how iota works is also the reason why the numbers
start at 3 (to keep the current behavoir).
2018-02-18 16:20:38 +01:00
Jakob Unterwurzacher
6c6947126d cryptocore: zero derived keys
Zero the HKDF-derived keys when we don't need them
anymore, and let the variable run of of scope.

https://github.com/rfjakob/gocryptfs/issues/211
2018-02-18 16:07:09 +01:00
Jakob Unterwurzacher
344d7e0a6f siv_aead: create private key copy and implement wiping
Having a private copy relieves the caller from worrying about
whether he can zero his copy. The copy can be cleared by
calling Wipe().
2018-02-18 16:01:46 +01:00
Jakob Unterwurzacher
adf7d75d31 main: changePassword: zero masterkey
Overwrite the masterkey with zeros once we
have encrypted it, and let it run out of scope.

Also get rid of the password duplicate in
readpassword.Twice.
2018-02-18 15:36:14 +01:00
Jakob Unterwurzacher
3b8f5cbb17 readpassword: convert from string to []byte
This will allows us to overwrite the password
with zeros once we are done with it.

https://github.com/rfjakob/gocryptfs/issues/211
2018-02-18 14:26:54 +01:00
Jakob Unterwurzacher
bd78b44389 cryptocore, main: add two comments
While reading the code, I had to think about what it
does, so add a comment that explains it.
2018-02-18 12:41:11 +01:00
Jakob Unterwurzacher
0efd220d1e configfile: overwrite and let keys run out of scope
As soon as we don't need them anymore, overwrite
keys with zeros and make sure they run out of scope
so we don't create a risk of inadvertedly using all-zero
keys for encryption.

https://github.com/rfjakob/gocryptfs/issues/211
2018-02-18 12:39:44 +01:00
Jakob Unterwurzacher
72ddbae1e6 stupidgcm: create private copy of the key
Relieves the caller from worrying about whether they
can overwrite the key.
2018-02-18 12:35:51 +01:00
Jakob Unterwurzacher
18f6c6106c main: try to wipe cryptocore's secret keys on unmount
Raise the bar for recovering keys from memory.

https://github.com/rfjakob/gocryptfs/issues/211
2018-02-18 11:39:10 +01:00
Jakob Unterwurzacher
719693ec5d fusefrontend[_reverse]: move crypto init up to caller
Both fusefrontend and fusefrontend_reverse were doing
essentially the same thing, move it into main's
initFuseFrontend.

A side-effect is that we have a reference to cryptocore
in main, which will help with wiping the keys on exit
(https://github.com/rfjakob/gocryptfs/issues/211).
2018-02-18 11:21:58 +01:00
Jakob Unterwurzacher
eeed4b4bef stupidgcm: implement key wipe
Not bulletproof due to possible GC copies, but
still raises to bar for extracting the key.

https://github.com/rfjakob/gocryptfs/issues/211
2018-02-17 15:14:55 +01:00
Jakob Unterwurzacher
7e0fefe970 stupidgcm: switch to pointer receivers
What the key slice does not get copied around
will make it possible to check if the key has been wiped.
2018-02-17 15:02:01 +01:00
Jakob Unterwurzacher
8151222ada gccgo: skip emulateGetdents on linux
The test is known to fail on gccgo
(https://github.com/rfjakob/gocryptfs/issues/201), but
getdents emulation is not used on linux, so let's skip
the test and ignore the failure.
2018-02-04 21:14:12 +01:00
Felix Lechner
bf2f9640c4 Fix spelling (#205) 2018-02-04 20:38:22 +01:00
Felix Lechner
5b986288cf Fix assignment error in Unix2syscall by converting Timespec into Nsec (#203)
$ go.gcc build
# github.com/rfjakob/gocryptfs/internal/syscallcompat
internal/syscallcompat/unix2syscall_linux.go:32:13: error: incompatible types in assignment (cannot use type int64 as type syscall.Timespec_sec_t)
  s.Atim.Sec = u.Atim.Sec
             ^
2018-02-03 13:42:49 +01:00
Jakob Unterwurzacher
9f8d0d8e57 gccgo: replace syscall.NAME_MAX with unix.NAME_MAX
For some reason the syscall.NAME_MAX constant does not exist
on gccgo, and it does not hurt us to use unix.NAME_MAX instead.

https://github.com/rfjakob/gocryptfs/issues/201
2018-02-01 23:50:11 +01:00
Jakob Unterwurzacher
26ba8103bf syscallcompat: switch from syscall.Getdents to unix.Getdents
On mips64le, syscall.Getdents() and struct syscall.Dirent do
not fit together, causing our Getdents implementation to
return garbage ( https://github.com/rfjakob/gocryptfs/issues/200
and https://github.com/golang/go/issues/23624 ).

Switch to unix.Getdents which does not have this problem -
the next Go release with the syscall package fixes is too
far away, and will take time to trickle into distros.
2018-01-31 18:59:10 +01:00
Jakob Unterwurzacher
f3838c09d8 syscallcompat: hardcode maxReclen = 280 for all architectures
Due to padding between entries, it is 280 even on 32-bit architectures.
See https://github.com/rfjakob/gocryptfs/issues/197 for details.
2018-01-25 22:22:13 +01:00
Jakob Unterwurzacher
b318572312 syscallcompat: fix reversed warning output
We used to print somewhat strange messages:

	Getdents: corrupt entry #1: Reclen=276 > 280. Returning EBADR

Reported at https://github.com/rfjakob/gocryptfs/issues/197
2018-01-25 21:42:15 +01:00
Jakob Unterwurzacher
ea51837361 fusefrontend: drop unused haveGetdents warning
We don't actually print that warning anymore.
2018-01-25 09:06:06 +01:00
Jakob Unterwurzacher
de878a3346 syscallcompat: explain why we don't use syscall.ParseDirent()
syscall.ParseDirent only returns the NAMES, we want
everything.
2018-01-25 08:43:30 +01:00
Jakob Unterwurzacher
a2677bce2a fusefrontend_reverse: use OpenNofollow in virtualFile.GetAttr
Makes it robust against symlink races.

Final piece, closes https://github.com/rfjakob/gocryptfs/issues/165
2018-01-17 21:36:38 +01:00
Jakob Unterwurzacher
959e1fc1e2 fusefrontend_reverse: use OpenNofollow in findLongnameParent
Protects findLongnameParent against symlink races.

Also add comments to several functions along the way.

Reported at https://github.com/rfjakob/gocryptfs/issues/165
2018-01-17 20:54:05 +01:00
Jakob Unterwurzacher
8951eb2472 fusefronted: add PlaintextNames special-cases for Create & Rename
gocryptfs.longname.XXX files were considered magic in PlaintextNames
mode, which was wrong.

Fix that and add tests.

Fixes https://github.com/rfjakob/gocryptfs/issues/174
2018-01-17 00:25:36 +01:00
Jakob Unterwurzacher
36ffd813cd Run go fmt 2018-01-16 23:18:53 +01:00
Sebastian Lackner
a85dbcab38 fusefrontend: Use Linkat syscall to implement Link 2017-12-25 15:07:37 +01:00
Sebastian Lackner
a24342f656 fusefrontend: Handle PlaintextNames mode in Link
In PlaintextNames mode the "gocryptfs.longname." prefix does not have any
special meaning.

https://github.com/rfjakob/gocryptfs/issues/174
2017-12-25 15:07:37 +01:00
Sebastian Lackner
631974f9e0 fusefrontend_reverse: Use O_DIRECTORY in OpenDir implementation
Also get rid of the defer - it is not really necessary here.
2017-12-11 21:18:20 +01:00
Sebastian Lackner
96dc2ca709 fusefrontend_reverse: Reject access to device nodes in newFile function
Steps to reproduce:

* Create a regular reverse mount point
* Create a file "test" in the original directory
* Access the corresponding encrypted directory in the mount point (ls <encrypted dir>)
* Quickly delete the file in the original data - instead create a device node
* Access the file again, it will access the device node and attempt to read from it

Fixes https://github.com/rfjakob/gocryptfs/issues/187
2017-12-11 09:55:16 +01:00
Sebastian Lackner
3af51736f3 fusefrontend_reverse: Use openBackingDir in GetAttr
Also fixes 48bd59f388 - the directory FD should
also be closed in case of an error.
2017-12-07 23:36:11 +01:00
Sebastian Lackner
ad0f110191 fusefrontend_reverse: Use openBackingDir in Readlink 2017-12-07 23:36:11 +01:00
Jakob Unterwurzacher
48bd59f388 fusefrontend_reverse: fix fd leak in GetAttr
Fixes https://github.com/rfjakob/gocryptfs/issues/184
2017-12-07 09:01:12 +01:00
Jakob Unterwurzacher
87736eb833 fusefrontend_reverse: secure Access against symlink races (somewhat)
Unfortunately, faccessat in Linux ignores AT_SYMLINK_NOFOLLOW,
so this is not completely atomic.

Given that the information you get from access is not very
interesting, it seems good enough.

https://github.com/rfjakob/gocryptfs/issues/165
2017-12-07 00:11:35 +01:00
Jakob Unterwurzacher
2ceef01afe syscallcompat: add Faccessat
Add faccessat(2) with a hack for symlink, because the
kernel does not actually looks at the passed flags.

From man 2 faccessat:

   C library/kernel differences
       The  raw faccessat() system call takes only the first three argu‐
       ments.  The AT_EACCESS and AT_SYMLINK_NOFOLLOW flags are actually
       implemented  within  the  glibc wrapper function for faccessat().
2017-12-07 00:05:28 +01:00
Jakob Unterwurzacher
e042eb38fa fusefrontend_reverse: secure Readlink against symlink races
...by using Readlinkat.

Tracking ticket: https://github.com/rfjakob/gocryptfs/issues/165
2017-12-06 21:13:08 +01:00
Jakob Unterwurzacher
f97494e89b syscallcompat: add Readlinkat
We need readlinkat to implement Readlink
symlink-race-free.
2017-12-06 21:07:24 +01:00
Jakob Unterwurzacher
6beb45e5b7 syscallcompat: add Darwin version of unix2syscall
The "Atim" field is called "Atimespec" on Darwin,
same for Mtim and Ctim.
2017-12-06 00:18:38 +01:00
Jakob Unterwurzacher
a3bdc2bf2b fusefrontend_reverse: secure GetAttr against symlink races
...by using the OpenNofollow helper & Fstatat.

Also introduce a helper to convert from unix.Stat_t to
syscall.Stat_t.

Tracking ticket: https://github.com/rfjakob/gocryptfs/issues/165
2017-12-06 00:06:31 +01:00
Jakob Unterwurzacher
03bf604fc0 syscallcompat: OpenNofollow: use O_DIRECTORY flag
...when opening intermedia directories to give us an
extra layer of safety.

From the FreeBSD man page:

     This flag can be used to prevent applications with elevated
     privileges from opening files which are even unsafe to open with O_RDONLY,
     such as device nodes.
2017-12-05 23:31:07 +01:00
Jakob Unterwurzacher
926cb93b50 fusefrontend_reverse: secure OpenDir against symlink races
...by using the new OpenNofollow helper.

The benchmark shows a small but acceptable performance loss:

  $ ./benchmark-reverse.bash
  LS:  2.182
  CAT: 18.221

Tracking ticket: https://github.com/rfjakob/gocryptfs/issues/165
2017-12-05 23:14:12 +01:00
Jakob Unterwurzacher
e604ce6dea syscallcompat: OpenNofollow: fix relPath="" case
Sometimes want to open baseDir itself. This case
was broken, fix it.
2017-12-05 23:08:55 +01:00
Jakob Unterwurzacher
70bcf58a9b syscallcompat: convert Getdents to fd input, add emulation
Now that we have Fstatat we can use it in Getdents to
get rid of the path name.

Also, add an emulated version of getdents for MacOS. This allows
to drop the !HaveGetdents special cases from fusefrontend.

Modify the getdents test to test both native getdents and the emulated
version.
2017-12-03 19:33:26 +01:00
Jakob Unterwurzacher
e33593d30d syscallcompat: add Fstatat + emulation + test
Fstatat has recently been added to x/sys/unix. Make
it available for use in gocryptfs.
2017-12-03 19:32:59 +01:00
Jakob Unterwurzacher
441e796e70 fusefrontend_reverse: secure StatFs agains symlink races
...by ignoring the path that was passed in.

https://github.com/rfjakob/gocryptfs/issues/165
2017-12-02 21:36:07 +01:00
Jakob Unterwurzacher
316b916358 fusefrontend_reverse: secure Open against symlink races
...using the new syscallcompat.OpenNofollow helper.

This change secures Open() against symlink race attacks
as described in https://github.com/rfjakob/gocryptfs/issues/165
2017-12-02 21:07:56 +01:00
Jakob Unterwurzacher
91e042e2ba syscallcompat: add OpenNofollow helper
OpenNofollow = symlink-race-safe Open

Prepares fixing https://github.com/rfjakob/gocryptfs/issues/165
2017-12-02 20:35:44 +01:00
Jakob Unterwurzacher
1d28973611 syscallcompat: move test setup into its own file
The infrastructure will also be used by the upcoming
OpenNofollow tests.
2017-12-02 19:57:23 +01:00
Jakob Unterwurzacher
77191c3485 syscallcompat: use Unlinkat and Symlinkat from x/sys/unix
I'm unsure why I did not notice this earlier, but the
syscall wrappers provided by x/sys/unix seem to do just
fine.

Drop our own version.
2017-12-02 18:36:18 +01:00
Sebastian Lackner
616a468180 syscallcompat: Improve the Openat and Mknodat syscall emulation
This avoids the conversion to an absolute path.
2017-12-01 09:41:52 +01:00
Sebastian Lackner
f30522a0c1 syscallcompat: Fix syscall emulation for absolute paths
For absolute paths, the file descriptor should be ignored. In such a case
there is also no need to hold the lock or change the working directory.
2017-12-01 09:41:52 +01:00
Sebastian Lackner
9bcde0c09e fusefrontend: Improve documentation of mkdirWithIv and WriteDirIV
As requested in https://github.com/rfjakob/gocryptfs/pull/179
2017-12-01 09:41:52 +01:00
Jakob Unterwurzacher
e97c23e083 syscallcompat: check that we get NOFOLLOW wherever possible
...and fix the instances where the AT_SYMLINK_NOFOLLOW /
O_NOFOLLOW / O_EXCL flag was missing.
2017-11-30 19:40:53 +01:00
Jakob Unterwurzacher
22282aefe6 syscallcompat: add tests for emulated syscalls
Also fix the bug in emulateFchmodat that was found by the tests.
2017-11-30 19:10:21 +01:00
Jakob Unterwurzacher
bd79a8cd0d syscallcompat: build emulated syscalls under linux, too
This will allow to test them under linux as well.
2017-11-30 17:07:55 +01:00
Sebastian Lackner
614745ee57 fusefrontend: allow_other: close race between mkdir and chown
Fixes the same problem as described in 72b975867a,
except for directories instead of device nodes.
2017-11-29 13:28:04 +01:00
Sebastian Lackner
67bcbe81e8 fusefrontend: Use Fchownat to implement Chown 2017-11-29 13:05:46 +01:00
Sebastian Lackner
0162392a28 fusefrontend: Use Fchmodat to implement Chmod 2017-11-29 12:55:41 +01:00
Sebastian Lackner
0f44c617d0 syscallcompat: Introduce unlinkat syscall with flags argument 2017-11-29 12:41:23 +01:00
Sebastian Lackner
5d44a31b41 fusefrontend: Use openBackingPath in Unlink and simplify code 2017-11-28 09:28:06 +01:00
Sebastian Lackner
2591900b69 fusefrontend: Handle PlaintextNames mode in Unlink
In PlaintextNames mode the "gocryptfs.longname." prefix does not have any
special meaning. We should not attempt to delete any .name files.

Partially fixes https://github.com/rfjakob/gocryptfs/issues/174
2017-11-28 09:28:06 +01:00
Sebastian Lackner
eba49402e4 fusefrontend: Introduce a openBackingPath helper and use it to simplify Mknod and Symlink 2017-11-28 09:28:06 +01:00
Sebastian Lackner
ad2720e0f9 fusefrontend: allow_other: close race between symlink and chown
Fixes the same problem as described in 72b975867a,
except for symlinks instead of device nodes.
2017-11-28 09:28:06 +01:00
Sebastian Lackner
5a56810603 fusefrontend: Use the Symlinkat syscall for longname handling 2017-11-28 09:28:06 +01:00
Sebastian Lackner
295c4c2b85 fusefrontend: Set owner after symlink creation in PlaintextNames mode
This is already done in regular mode, but was missing when PlaintextNames mode
is enabled. As a result, symlinks created by non-root users were still owned
by root afterwards.

Fixes https://github.com/rfjakob/gocryptfs/issues/176
2017-11-28 09:28:06 +01:00
Sebastian Lackner
3f68b0c09a fusefrontend: Handle PlaintextNames mode in Mknod
In PlaintextNames mode the "gocryptfs.longname." prefix does not have any
special meaning. We should not attempt to read the directory IV or to
create special .name files.

Partially fixes https://github.com/rfjakob/gocryptfs/issues/174
2017-11-28 09:28:06 +01:00
Sebastian Lackner
8c5069c637 syscallcompat: Fix Fchownat syscall wrapper on darwin
* Acquire the lock before reading the current directory
* Fix a file descriptor leak
2017-11-28 09:28:06 +01:00
Jakob Unterwurzacher
72b975867a fusefronted: allow_other: close race between mknod and chown
If the user manages to replace the directory with
a symlink at just the right time, we could be tricked
into chown'ing the wrong file.

This change fixes the race by using fchownat, which
unfortunately is not available on darwin, hence a compat
wrapper is added.

Scenario, as described by @slackner at
https://github.com/rfjakob/gocryptfs/issues/177 :

1. Create a forward mount point with `plaintextnames` enabled
2. Mount as root user with `allow_other`
3. For testing purposes create a file `/tmp/file_owned_by_root`
   which is owned by the root user
4. As a regular user run inside of the GoCryptFS mount:

```
mkdir tempdir
mknod tempdir/file_owned_by_root p &
mv tempdir tempdir2
ln -s /tmp tempdir
```

When the steps are done fast enough and in the right order
(run in a loop!), the device file will be created in
`tempdir`, but the `lchown` will be executed by following
the symlink. As a result, the ownership of the file located
at `/tmp/file_owned_by_root` will be changed.
2017-11-27 21:04:45 +01:00
Jakob Unterwurzacher
1bb47b6796 reverse: reject too-long symlink target reads with ENAMETOOLONG
If the symlink target gets too long due to base64 encoding, we should
return ENAMETOOLONG instead of having the kernel reject the data and
returning an I/O error to the user.

Fixes https://github.com/rfjakob/gocryptfs/issues/167
2017-11-26 21:37:12 +01:00
Sebastian Lackner
90687215a4 fusefrontend_reverse: Do not mix up cache information for different directories
Fixes https://github.com/rfjakob/gocryptfs/issues/168

Steps to reproduce the problem:

* Create a regular reverse mount point
* Create files with the same very long name in multiple directories - so far
  everything works as expected, and it will appear with a different name each
  time, for example, gocryptfs.longname.A in directory A and
  gocryptfs.longname.B in directory B
* Try to access a path with A/gocryptfs.longname.B or B/gocryptfs.longname.A -
  this should fail, but it actually works.

The problem is that the longname cache only uses the path as key and not the
dir or divIV. Assume an attacker can directly interact with a reverse mount and
knows the relation longname path -> unencoded path in one directory, it allows
to test if the same unencoded filename appears in any other directory.
2017-11-25 16:20:48 +01:00
Sebastian Lackner
95870e841e fusefrontend: Skip gocryptfs.diriv handling when directory was deleted successfully
Fixes https://github.com/rfjakob/gocryptfs/issues/171

Steps to reproduce:

* Create a regular forward mount point
* Create a new directory in the mount point
* Manually delete the gocryptfs.diriv file from the corresponding ciphertext
  directory
* Attempt to delete the directory with 'rmdir <dirname>'

Although the code explicitly checks for empty directories, it will still attempt
to move the non-existent gocryptfs.diriv file and fails with:

    rmdir: failed to remove '<dirname>': No such file or directory
2017-11-25 16:20:00 +01:00
Sebastian Lackner
9f56b33e0c fusefrontend: Fix longname handling for renames with existing target
Fixes https://github.com/rfjakob/gocryptfs/issues/170

Steps to reproduce the problem:

* Create a regular forward mount point
* Create a file with a shortname and one with a long filename
* Try to run 'mv <shortname> <longname>'

This should actually work and replace the existing file, but instead it
fails with:

    mv: cannot move '<shortname>' to '<longname>': File exists

The problem is the creation of the .name file. If the target already exists
we can safely ignore the EEXIST error and just keep the existing .name file.
2017-11-25 16:19:09 +01:00
Sebastian Lackner
f80f19f589 fusefrontend_reverse: Add a missing Close() call 2017-11-22 23:42:49 +01:00
Sebastian Lackner
c547673529 nametransform: Return error if decrypted name is '.' or '..' 2017-11-22 23:42:08 +01:00
Sebastian Lackner
f3c777d5ea main: Add '-devrandom' commandline option
Allows to use /dev/random for generating the master key instead of the
default Go implementation. When the kernel random generator has been
properly initialized both are considered equally secure, however:

* Versions of Go prior to 1.9 just fall back to /dev/urandom if the
  getrandom() syscall would be blocking (Go Bug #19274)

* Kernel versions prior to 3.17 do not support getrandom(), and there
  is no check if the random generator has been properly initialized
  before reading from /dev/urandom

This is especially useful for embedded hardware with low-entroy. Please
note that generation of the master key might block indefinitely if the
kernel cannot harvest enough entropy.
2017-11-21 23:37:06 +01:00
Jakob Unterwurzacher
34547a6c39 tests: don't read /proc, the number of entries changes too quickly
This could lead to test failures like this:

  --- FAIL: TestGetdents (0.02s)
  	getdents_test.go:57: len(getdentsEntries)=362, len(readdirEntries)=360
  FAIL
2017-10-22 18:13:08 +02:00
Jakob Unterwurzacher
268e0484e2 Revert most of "fusefrontend: clamp oversized reads"
We cannot return less data than requested to the kernel!

From https://libfuse.github.io/doxygen/structfuse__operations.html:

  Read should return exactly the number of bytes
  requested except on EOF or error, otherwise the
  rest of the data will be substituted with
  zeroes.

Reverts commit 3009ec9852 minus
the formatting improvements we want to keep.

Fixes https://github.com/rfjakob/gocryptfs/issues/147
Reopens https://github.com/rfjakob/gocryptfs/issues/145
2017-10-21 17:43:21 +02:00
Jakob Unterwurzacher
29445c976d contentenc: reserve one additional block in CReqPool
...to account for unaligned reads.

I have not seen this happen in the wild because the kernel
always seems to issue 4k-aligned requests. But the cost
of the additional block in the pool is low and prevents
a buffer overrun panic when an unaligned read does happen.
2017-10-19 09:23:10 +02:00
Jakob Unterwurzacher
3009ec9852 fusefrontend: clamp oversized reads
Our byte cache pools are sized acc. to MAX_KERNEL_WRITE, but the
running kernel may have a higher limit set. Clamp to what we can
handle.

Fixes a panic on a Synology NAS reported at
https://github.com/rfjakob/gocryptfs/issues/145
2017-10-17 21:48:29 +02:00
Jakob Unterwurzacher
4da245c69d fusefrontend_reverse: fix 176-byte names
A file with a name of exactly 176 bytes length caused this error:

  ls: cannot access ./tmp/dsg/sXSGJLTuZuW1FarwIkJs0w/b6mGjdxIRpaeanTo0rbh0A/QjMRrQZC_4WLhmHI1UOBcA/gocryptfs.longname.QV-UipdDXeUVdl05WruoEzBNPrQCfpu6OzJL0_QnDKY: No such file or directory
  ls: cannot access ./tmp/dsg/sXSGJLTuZuW1FarwIkJs0w/b6mGjdxIRpaeanTo0rbh0A/QjMRrQZC_4WLhmHI1UOBcA/gocryptfs.longname.QV-UipdDXeUVdl05WruoEzBNPrQCfpu6OzJL0_QnDKY.name: No such file or directory
  -????????? ? ?     ?             ?            ? gocryptfs.longname.QV-UipdDXeUVdl05WruoEzBNPrQCfpu6OzJL0_QnDKY
  -????????? ? ?     ?             ?            ? gocryptfs.longname.QV-UipdDXeUVdl05WruoEzBNPrQCfpu6OzJL0_QnDKY.name

Root cause was a wrong shortNameMax constant that failed to
account for the obligatory padding byte.

Fix the constant and also expand the TestLongnameStat test case
to test ALL file name lengths from 1-255 bytes.

Fixes https://github.com/rfjakob/gocryptfs/issues/143 .
2017-10-01 13:50:25 +02:00
Jakob Unterwurzacher
0072a96f20 siv_aead: fix trivial typo in comment 2017-09-17 11:42:46 +02:00
Jakob Unterwurzacher
885fdcabda contentenc: deduplicate AD packing into new concatAD() func
The encrypt and decrypt path both had a copy that were equivalent
but ordered differently, which was confusing.

Consolidate it in a new dedicated function.
2017-09-17 11:21:48 +02:00
Jakob Unterwurzacher
4bd2c6736a contentenc: DecryptBlocks: give block number counter a clearer name
Using firstBlockNo as the counter is confusing, create a
copy named "blockNo" and use that.
2017-09-17 10:59:04 +02:00
Jakob Unterwurzacher
604b0779d4 macos: automatically remove .DS_Store on Rmdir
MacOS sprinkles .DS_Store files everywhere. This is hard to avoid for
users, so handle it transparently in Rmdir().

Mitigates https://github.com/rfjakob/gocryptfs/issues/140
2017-09-05 22:47:15 +02:00
Jakob Unterwurzacher
6f3b65d924 fusefrontend: reorder logic in Rmdir to get rid of one indentation level
Handle the errors first so that the normal code path is not indented.

This should not cause any behavoir changes.
2017-09-05 22:10:08 +02:00
Jakob Unterwurzacher
3a5a783b54 macos: don't throw IO errors because of .DS_Store files
MacOS creates lots of these files, and if the directory is otherwise
empty, we would throw an IO error to the unsuspecting user.

With this patch, we log a warning, but otherwise pretend we did not
see it.

Mitigates https://github.com/rfjakob/gocryptfs/issues/140
2017-09-05 21:47:05 +02:00
Jakob Unterwurzacher
538cae610c syscallcompat: Getdents: warn once if we get DT_UNKNOWN
...and if Getdents is not available at all.

Due to this warning I now know that SSHFS always returns DT_UNKNOWN:

    gocryptfs[8129]: Getdents: convertDType: received DT_UNKNOWN, falling back to Lstat

This behavoir is confirmed at http://ahefner.livejournal.com/16875.html:

    "With sshfs, I finally found that obscure case. The dtype is always set to DT_UNKNOWN [...]"
2017-09-03 15:05:54 +02:00
Jakob Unterwurzacher
276567eb13 fusefrontend: use DirIVCache in OpenDir()
Previously, OpenDir() did not use the cache at all, missing
an opportunity to speed up repeated directory reads.
2017-09-03 13:59:53 +02:00
Jakob Unterwurzacher
7da0e97c8b dirivcache: add better function comments + a sanity check on Store()
The comments were unclear on whether relative or absolute paths
have to be passed.
2017-09-03 13:53:50 +02:00
Jakob Unterwurzacher
ed046aa359 Fix misspellings reported by goreportcard.com
https://goreportcard.com/report/github.com/rfjakob/gocryptfs#misspell
2017-08-21 21:06:05 +02:00
Jakob Unterwurzacher
312ea32bb7 cryptocore: add urandom + randprefetch benchmarks
The benchmark that supported the decision for 512-byte
prefetching previously lived outside the repo.

Let's add it where it belongs so it cannot get lost.
2017-08-16 18:33:00 +02:00
Jakob Unterwurzacher
989b880989 fusefrontend: use Getdents if available
Getdents avoids calling Lstat on each file.
2017-08-15 19:04:02 +02:00
Jakob Unterwurzacher
e50a6a57e5 syscallcompat: implement Getdents()
The Readdir function provided by os is inherently slow because
it calls Lstat on all files.

Getdents gives us all the information we need, but does not have
a proper wrapper in the stdlib.

Implement the "Getdents()" wrapper function that calls
syscall.Getdents() and parses the returned byte blob to a
fuse.DirEntry slice.
2017-08-15 19:03:57 +02:00
Jakob Unterwurzacher
0c520845f3 main: purge masterkey from memory as soon as possible
Remove the "Masterkey" field from fusefrontend.Args because it
should not be stored longer than neccessary. Instead pass the
masterkey as a separate argument to the filesystem initializers.

Then overwrite it with zeros immediately so we don't have
to wait for garbage collection.

Note that the crypto implementation still stores at least a
masterkey-derived value, so this change makes it harder, but not
impossible, to extract the encryption keys from memory.

Suggested at https://github.com/rfjakob/gocryptfs/issues/137
2017-08-11 19:02:26 +02:00
Jakob Unterwurzacher
e80b5f2049 nametransform: extend diriv cache to 100 entries
* extend the diriv cache to 100 entries
* add special handling for the immutable root diriv

The better cache allows to shed some complexity from the path
encryption logic (parent-of-parent check).

Mitigates https://github.com/rfjakob/gocryptfs/issues/127
2017-08-09 22:00:53 +02:00
Jakob Unterwurzacher
75ec94a87a nametransform: add Dir() function
Dir is like filepath.Dir but returns "" instead of ".".
This was already implemented in fusefrontend_reverse as saneDir().

We will need it in nametransform for the improved diriv caching.
2017-08-06 23:14:39 +02:00
Jakob Unterwurzacher
5190cc09bb nametransform: move diriv cache into it's own package
Needs some space to grow.

renamed:    internal/nametransform/diriv_cache.go -> internal/nametransform/dirivcache/dirivcache.go
2017-08-06 21:59:15 +02:00
Jakob Unterwurzacher
32611ff97a nametransform: deduplicate code to encryptAndHashName()
This operation has been done three time by identical
sections of code. Create a function for it.
2017-08-06 21:23:42 +02:00
Jakob Unterwurzacher
d12aa57715 fusefronted_reverse: fix ino collision between .name and .diriv files
A directory with a long name has two associated virtual files:
the .name file and the .diriv files.

These used to get the same inode number:

  $ ls -di1  * */*
             33313535 gocryptfs.longname.2togDFouca9mrTwtfF1RNW5DZRAQY8alaR7wO_Xd5Zw
  1000000000033313535 gocryptfs.longname.2togDFouca9mrTwtfF1RNW5DZRAQY8alaR7wO_Xd5Zw/gocryptfs.diriv
  1000000000033313535 gocryptfs.longname.2togDFouca9mrTwtfF1RNW5DZRAQY8alaR7wO_Xd5Zw.name

With this change we use another prefix (2 instead of 1) for .name files.

  $ ls -di1 * */*
             33313535 gocryptfs.longname.2togDFouca9mrTwtfF1RNW5DZRAQY8alaR7wO_Xd5Zw
  1000000000033313535 gocryptfs.longname.2togDFouca9mrTwtfF1RNW5DZRAQY8alaR7wO_Xd5Zw/gocryptfs.diriv
  2000000000033313535 gocryptfs.longname.2togDFouca9mrTwtfF1RNW5DZRAQY8alaR7wO_Xd5Zw.name
2017-07-29 16:15:49 +02:00
Jakob Unterwurzacher
d5133ca5ac fusefrontend_reverse: return ENOENT for undecryptable names
This was working until DecryptName switched to returning
EBADMSG instead of EINVAL.

Add a test to catch the regression next time.
2017-07-27 20:31:22 +02:00
Jakob Unterwurzacher
ccf1a84e41 macos: make testing without openssl work properly
On MacOS, building and testing without openssl is much easier.
The tests should skip tests that fail because of missing openssl
instead of aborting.

Fixes https://github.com/rfjakob/gocryptfs/issues/123
2017-07-14 23:22:15 +02:00
Jakob Unterwurzacher
61e964457d stupidgcm: fix openssl 1.1 build failure
Fixed by including the correct header. Should work on older openssl
versions as well.

Error was:
locking.go:21: undefined reference to `CRYPTO_set_locking_callback'
2017-07-14 20:44:07 +02:00
Jakob Unterwurzacher
3062de6187 fusefronted: enable writing to write-only files
Due to RMW, we always need read permissions on the backing file. This is a
problem if the file permissions do not allow reading (i.e. 0200 permissions).
This patch works around that problem by chmod'ing the file, obtaining a fd,
and chmod'ing it back.

Test included.

Issue reported at: https://github.com/rfjakob/gocryptfs/issues/125
2017-07-11 23:19:58 +02:00
Jakob Unterwurzacher
b6bda01c33 contentenc: MergeBlocks: short-circuit the trivial case
Saves 3% for the tar extract benchmark because we skip the allocation.
2017-07-02 16:23:24 +02:00
Jakob Unterwurzacher
52ab0462a4 fusefrontend: doRead: skip decryption for an empty read
Previously we ran through the decryption steps even for an empty
ciphertext slice. The functions handle it correctly, but returning
early skips all the extra calls.

Speeds up the tar extract benchmark by about 4%.
2017-07-02 16:02:13 +02:00
Jakob Unterwurzacher
9f4bd76576 stupidgcm: add test for in-place Open
Adds a test for the optimization introduced in:

	stupidgcm: Open: if "dst" is big enough, use it as the output buffer
2017-07-01 09:56:05 +02:00
Jakob Unterwurzacher
12c0101a23 contentenc: add PReqPool and use it in DecryptBlocks
This gets us a massive speed boost in streaming reads.
2017-06-30 23:30:57 +02:00
Jakob Unterwurzacher
e4b5005bcc stupidgcm: Open: if "dst" is big enough, use it as the output buffer
This means we won't need any allocation for the plaintext.
2017-06-30 23:24:12 +02:00
Jakob Unterwurzacher
b2a23e94d1 fusefrontend: doRead: use CReqPool for ciphertext buffer
Easily saves lots of allocations.
2017-06-30 23:15:31 +02:00
Jakob Unterwurzacher
06398e82d9 fusefrontend: Read: use provided buffer
This will allow us to return internal buffers to a pool.
2017-06-30 23:11:38 +02:00
Jakob Unterwurzacher
80676c685f contentenc: add safer "bPool" pool variant; add pBlockPool
bPool verifies the lengths of slices going in and out.

Also, add a plaintext block pool - pBlockPool - and use
it for decryption.
2017-06-29 23:44:32 +02:00
Jakob Unterwurzacher
0cc6f53496 stupidgcm: use "dst" as the output buffer it is big enough
This saves an allocation of the ciphertext block.
2017-06-29 18:52:33 +02:00
Jakob Unterwurzacher
3c6fe98eb1 contentenc: use sync.Pool memory pools for encryption
We use two levels of buffers:

1) 4kiB+overhead for each ciphertext block
2) 128kiB+overhead for each FUSE write (32 ciphertext blocks)

This commit adds a sync.Pool for both levels.

The memory-efficiency for small writes could be improved,
as we now always use a 128kiB buffer.
2017-06-20 21:22:00 +02:00
Jakob Unterwurzacher
a4563e21ec main, syscallcompat: use Dup3 instead of Dup2
Dup2 is not implemented on linux/arm64.

Fixes https://github.com/rfjakob/gocryptfs/issues/121 .

Also adds cross-compilation to CI.
2017-06-18 15:43:22 +02:00
Jakob Unterwurzacher
e52594dae6 contentenc: parallelize encryption for 128kiB writes
128kiB = 32 x 4kiB pages is the maximum we get from the kernel. Splitting
up smaller writes is probably not worth it.

Parallelism is limited to two for now.
2017-06-11 21:56:16 +02:00
Jakob Unterwurzacher
9837cb0ddc cryptocore: prefetch nonces in the background
Spawn a worker goroutine that reads the next 512-byte block
while the current one is being drained.

This should help reduce waiting times when /dev/urandom is very
slow (like on Linux 3.16 kernels).
2017-06-11 21:29:50 +02:00
Jakob Unterwurzacher
80516ed335 cryptocore: prefetch nonces in 512-byte blocks
On my machine, reading 512-byte blocks from /dev/urandom
(same via getentropy syscall) is a lot faster in terms of
throughput:

Blocksize    Throughput
 16          28.18 MB/s
512          83.75 MB/s

For a single-threaded streaming write, this drops the CPU usage of
nonceGenerator.Get to almost 1/3:

        flat  flat%   sum%        cum   cum%
Before     0     0% 95.08%      0.35s  2.92%  github.com/rfjakob/gocryptfs/internal/cryptocore.(*nonceGenerator).Get
After  0.01s 0.092% 92.34%      0.13s  1.20%  github.com/rfjakob/gocryptfs/internal/cryptocore.(*nonceGenerator).Get

This change makes the nonce reading single-threaded, which may
hurt massively-parallel writes.
2017-06-09 22:05:14 +02:00
Charles Duffy
da1bd74246 Fix missing Owner coercion for already-open files (#117) 2017-06-09 22:04:56 +02:00
Jakob Unterwurzacher
d2be22a07f cryptocore: remove lastNonce check
This check would need locking to be multithreading-safe.
But as it is in the fastpath, just remove it.
rand.Read() already guarantees that the value is random.
2017-06-07 23:08:43 +02:00
Jakob Unterwurzacher
294628b384 contentenc: move EncryptBlocks() loop into its own functions
This allows easy parallelization in the future.
2017-06-07 22:09:15 +02:00
Jakob Unterwurzacher
71978ec88a Add "-trace" flag (record execution trace)
Uses the runtime/trace functionality.

TODO: add to man page.
2017-06-07 22:09:06 +02:00
Jakob Unterwurzacher
a24faa3ba5 fusefrontend: write: consolidate and move encryption to contentenc
Collect all the plaintext and pass everything to contentenc in
one call.

This will allow easier parallization of the encryption.

https://github.com/rfjakob/gocryptfs/issues/116
2017-06-01 22:19:27 +02:00
Jakob Unterwurzacher
f44902aaae Fix two comments
One out-of-date and the other with a typo.
2017-06-01 18:53:57 +02:00
Charles Duffy
cf1ded5236 Implement force_owner option to display ownership as a specific user. 2017-06-01 00:26:17 +02:00
Jakob Unterwurzacher
fc2a5f5ab0 pathiv: fix test failure on Go 1.6
Travis failed on Go 1.6.3 with this error:

	internal/pathiv/pathiv_test.go:20: no args in Error call

This change should solve the problem and provides a better error
message on (real) test failure.
2017-05-31 08:21:36 +02:00
Jakob Unterwurzacher
9a217ce786 pathiv: move block IV algorithm into this package
This was implemented in fusefrontend_reverse, but we need it
in fusefrontend as well. Move the algorithm into pathiv.BlockIV().
2017-05-30 17:04:46 +02:00
Jakob Unterwurzacher
d202a456f5 pathiv: move derivedIVContainer into the package
...under the new name "FileIVs".

This will also be used by forward mode.
2017-05-30 17:04:46 +02:00
Jakob Unterwurzacher
857507e8b1 fusefrontend_reverse: move pathiv to its own package
We will also need it in forward mode.
2017-05-30 17:04:46 +02:00
Jakob Unterwurzacher
d6ef283c3f cryptocore: improve comments and add tests for hkdfDerive
These should make it easier to re-implement the key derivation
that was enabled with the "HKDF" feature flag.
2017-05-27 14:41:20 +02:00
Jakob Unterwurzacher
9ecf2d1a3f fusefrontend_reverse: store derived values for hard-linked files
With hard links, the path to a file is not unique. This means
that the ciphertext data depends on the path that is used to access
the files.

Fix that by storing the derived values when we encounter a hard-linked
file. This means that the first path wins.
2017-05-25 21:33:16 +02:00
Jakob Unterwurzacher
9a3f9350fe nametransform: reject all-zero dirIV
This should never happen in normal operation and is a sign of
data corruption. Catch it early.
2017-05-25 14:21:55 +02:00
Jakob Unterwurzacher
2ce269ec63 contenenc: reject all-zero file ID
This should never happen in normal operation and is a sign of
data corruption. Catch it early.
2017-05-25 14:20:27 +02:00
Jakob Unterwurzacher
c0e411f81d contentenc: better error reporting in ParseHeader
Log the message ourselves and return EINVAL.

Before:

	gocryptfs[26962]: go-fuse: can't convert error type: ParseHeader: invalid version: got 0, want 2

After:

	gocryptfs[617]: ParseHeader: invalid version: want 2, got 0. Returning EINVAL.
2017-05-25 14:18:44 +02:00
Jakob Unterwurzacher
e827763f2e nametransform: harden name decryption against invalid input
This fixes a few issues I have found reviewing the code:

1) Limit the amount of data ReadLongName() will read. Previously,
you could send gocryptfs into out-of-memory by symlinking
gocryptfs.diriv to /dev/zero.

2) Handle the empty input case in unPad16() by returning an
error. Previously, it would panic with an out-of-bounds array
read. It is unclear to me if this could actually be triggered.

3) Reject empty names after base64-decoding in DecryptName().
An empty name crashes emeCipher.Decrypt().
It is unclear to me if B64.DecodeString() can actually return
a non-error empty result, but let's guard against it anyway.
2017-05-23 21:26:38 +02:00
Jakob Unterwurzacher
508fd9e1d6 main: downgrade panic log create failure from fatal error to warning
Exiting with a fatal error just pushes users to use "-nosyslog",
which is even worse than not having a paniclog.
2017-05-23 18:01:21 +02:00
Jakob Unterwurzacher
245b84c887 nametransform: diriv cache: fall back to the grandparent
When a user calls into a deep directory hierarchy, we often
get a sequence like this from the kernel:

LOOKUP a
LOOKUP a/b
LOOKUP a/b/c
LOOKUP a/b/c/d

The diriv cache was not effective for this pattern, because it
was designed for this:

LOOKUP a/a
LOOKUP a/b
LOOKUP a/c
LOOKUP a/d

By also using the cached entry of the grandparent we can avoid lots
of diriv reads.

This benchmark is against a large encrypted directory hosted on NFS:

Before:

  $ time ls -R nfs-backed-mount > /dev/null
  real	1m35.976s
  user	0m0.248s
  sys	0m0.281s

After:

  $ time ls -R nfs-backed-mount > /dev/null
  real	1m3.670s
  user	0m0.217s
  sys 	0m0.403s
2017-05-22 22:36:54 +02:00
Jakob Unterwurzacher
c44389d942 exitcodes: specific codes for failure to read or write gocryptfs.conf
New codes:
* OpenConf = 23
* WriteConf = 24
2017-05-14 14:30:50 +02:00
Jakob Unterwurzacher
2aea2d3d62 exitcodes: add code 22 for "password is empty"
Empty passwords are not allowed. Let's give the error
it's own exit code.
2017-05-14 14:02:08 +02:00
Jakob Unterwurzacher
8aabc54276 exitcodes: get rid of generic "Mount" exit code
Instead, create three new specific exit codes:
* FuseNewServer = 19
* CtlSock = 20
* PanicLogCreate = 21
2017-05-14 13:51:26 +02:00
Jakob Unterwurzacher
d5adde1eeb exitcodes: pull all exit code definitions into the package
This commit defines all exit codes in one place in the exitcodes
package.

Also, it adds a test to verify the exit code on incorrect
password, which is what SiriKali cares about the most.

Fixes https://github.com/rfjakob/gocryptfs/issues/77 .
2017-05-07 22:16:22 +02:00
Jakob Unterwurzacher
ad7942f434 fusefrontend: implement path decryption via ctlsock
Closes https://github.com/rfjakob/gocryptfs/issues/84 .
2017-05-07 21:01:39 +02:00
Jakob Unterwurzacher
26881538e1 nametranform, fusefrontend: better errors on invalid names
nametransform.DecryptName() now always returns syscall.EBADMSG if
the name was invalid.

fusefrontend.OpenDir error messages have been normalized.
2017-05-07 20:58:27 +02:00
Jakob Unterwurzacher
68387b470c Fix typos found by Misspell
Misspell Finds commonly misspelled English words
gocryptfs/internal/configfile/scrypt.go
Line 41: warning: "paramter" is a misspelling of "parameter" (misspell)
gocryptfs/internal/ctlsock/ctlsock_serve.go
Line 1: warning: "implementes" is a misspelling of "implements" (misspell)
gocryptfs/tests/test_helpers/helpers.go
Line 27: warning: "compatability" is a misspelling of "compatibility" (misspell)
2017-05-07 12:22:15 +02:00
Jakob Unterwurzacher
c52e1abc58 fusefrontend: log "too many open files" errors
This usually indicates that the open file limit for gocryptfs is
too low. We should report this to the user.
2017-05-03 23:46:52 +02:00
Jakob Unterwurzacher
fb3cc6ea40 openfiletable: rename WriteLock to ContentLock
...and IDLock to HeaderLock. This matches what the locks actually
protect.
2017-05-01 21:57:18 +02:00
Jakob Unterwurzacher
f322ee87e3 fusefrontend: rely on nodefs.defaultFile for no-op functions
Now that we embed nodefs.NewDefaultFile(), we can drop our own
no-ops.
2017-05-01 19:12:37 +02:00
Jakob Unterwurzacher
1a89919d80 contentenc: downgrade "interrupted write?" warning to debug
This can happen during normal operation, and is harmless since

14038a1644
"fusefrontend: readFileID: reject files that consist only of a header"

causes dormant header-only files to be rewritten on the next write.
2017-05-01 18:44:18 +02:00
Jakob Unterwurzacher
9ab11aa4d7 fusefrontend: drop writeOnly flag
We do not have to track the writeOnly status because the kernel
will not forward read requests on a write-only FD to us anyway.

I have verified this behavoir manually on a 4.10.8 kernel and also
added a testcase.
2017-05-01 17:49:37 +02:00
Jakob Unterwurzacher
514f515dd7 fusefronted, openfiletable: move the open file table to its own package
The open file table code needs some room to grow for the upcoming
FD multiplexing implementation.
2017-05-01 17:26:50 +02:00
Jakob Unterwurzacher
863c3ca36f fusefrontend: rename write_lock.go -> open_file_table.go
The data structure was originally called write lock table, but
is now simply called the open file table. Rename the file to
reflect that.
2017-04-29 22:24:38 +02:00
Jakob Unterwurzacher
b66e03486a fusefronted: drop unused file.String() function
This is a very old leftover.
2017-04-29 18:20:39 +02:00
Jakob Unterwurzacher
6e029a3799 readpassword: increase max password size to 2048
This is the value EncFS uses, so let's follow suit.
Suggested at https://github.com/rfjakob/gocryptfs/issues/77 .
2017-04-29 15:15:11 +02:00
Jakob Unterwurzacher
edb3e19cb5 fix golint complaints 2017-04-29 14:50:58 +02:00
Jakob Unterwurzacher
7d38f80a78 nametransform: WriteDirIV: replace ioutil.WriteFile
As reported at https://github.com/rfjakob/gocryptfs/issues/105 ,
the "ioutil.WriteFile(file, iv, 0400)" call causes "permissions denied"
errors on an NFSv4 setup.

"strace"ing diriv creation and gocryptfs.conf creation shows this:

conf (works on the user's NFSv4 mount):
openat(AT_FDCWD, "/tmp/a/gocryptfs.conf.tmp", O_WRONLY|O_CREAT|O_EXCL|O_CLOEXEC, 0400) = 3

diriv (fails):
openat(AT_FDCWD, "/tmp/a/gocryptfs.diriv", O_WRONLY|O_CREAT|O_TRUNC|O_CLOEXEC, 0400) = 3

This patch creates the diriv file with the same flags that are used for
creating the conf:
openat(AT_FDCWD, "/tmp/a/gocryptfs.diriv", O_WRONLY|O_CREAT|O_EXCL|O_CLOEXEC, 0400) = 3

Closes https://github.com/rfjakob/gocryptfs/issues/105
2017-04-29 14:15:13 +02:00
Jakob Unterwurzacher
3409ade272 forcedecode: tighten checks
...and fix a few golint issues and print a scary warning message on mount.

Also, force the fs to ro,noexec.
2017-04-24 00:25:02 +02:00
danim7
f1945c4daa Add -forcedecode
Force decode of encrypted files even if the integrity check fails, instead of
failing with an IO error. Warning messages are still printed to syslog if corrupted
files are encountered.
It can be useful to recover files from disks with bad sectors or other corrupted
media.

Closes https://github.com/rfjakob/gocryptfs/pull/102 .
2017-04-23 23:11:56 +02:00
Jakob Unterwurzacher
9777e4bf7e Fix Flock build breakage
go-fuse has added a new method to the nodefs.File interface that
caused this build error:

  internal/fusefrontend/file.go:75: cannot use file literal (type *file) as type nodefs.File in return argument:
  	*file does not implement nodefs.File (missing Flock method)

Fixes https://github.com/rfjakob/gocryptfs/issues/104 and
prevents the problem from happening again.
2017-04-23 00:06:56 +02:00
Jakob Unterwurzacher
778c955eea fusefrontend_reverse: switch to stable inode numbers
The volatile inode numbers that we used before cause "find" to complain and error out.
Virtual inode numbers are derived from their parent file inode number by adding 10^19,
which is hopefully large enough no never cause problems in practice.

If the backing directory contains inode numbers higher than that, stat() on these files
will return EOVERFLOW.

Example directory lising after this change:

  $ ls -i
               926473 gocryptfs.conf
  1000000000000926466 gocryptfs.diriv
               944878 gocryptfs.longname.hmZojMqC6ns47eyVxLlH2ailKjN9bxfosi3C-FR8mjA
  1000000000000944878 gocryptfs.longname.hmZojMqC6ns47eyVxLlH2ailKjN9bxfosi3C-FR8mjA.name
               934408 Tdfbf02CKsTaGVYnAsSypA
2017-04-01 17:19:15 +02:00
Jakob Unterwurzacher
e87aebb835 fusefrontend_reverse: drop unused dirIVAttr function
This has long been replaced by virtualFile.GetAttr().
2017-04-01 17:05:55 +02:00
Jakob Unterwurzacher
acb73ca436 fusefrontend_reverse: convert fmt.Printf calls to tlog
The fmt.Printfs output would end up in the paniclog.
2017-04-01 15:49:53 +02:00
Jakob Unterwurzacher
3cd18f288f fusefrontend_reverse: add comment to newVirtualFile
...and improve and comment variable naming in findLongnameParent.

No semantic changes.
2017-04-01 14:17:54 +02:00
danim7
fb1b8ced38 fusefrontend_reverse: consistent file owners for .diriv, .name files
This PR addresses the Issue #95, about "Confusing file owner for
longname files in reverse mode".

It affects only the reverse mode, and introduces two
modifications:

1) The "gocryptfs.longname.XXXX.name" files are assigned the owner and
group of the underlying plaintext file. Therefore it is consistent
with the file "gocryptfs.longname.XXXX" that has the encrypted
contents of the plaintext file.

2) The two virtual files mentioned above are given -r--r--r--
permissions. This is consistent with the behavior described in
function Access in internal/fusefrontend_reverse/rfs.go where all
virtual files are always readable. Behavior also observed in point
c) in #95 .

Issue #95 URL: https://github.com/rfjakob/gocryptfs/issues/95
Pull request URL: https://github.com/rfjakob/gocryptfs/pull/97
2017-03-28 22:58:03 +02:00
Jakob Unterwurzacher
c815554866 configfile: always validate all scrypt parameters
This makes sure we cannot get weak parameters passed through a
rougue gocryptfs.conf.
2017-03-25 19:36:16 +01:00
Jakob Unterwurzacher
2824218a25 readpassword: increase max password length to 2000
1000 was too low as at least one user had a password
that was longer.

Fixes https://github.com/rfjakob/gocryptfs/issues/93
2017-03-20 09:29:56 +01:00
Jakob Unterwurzacher
cb47f65212 fusefrontend: get rid of leftover debug output 2017-03-18 16:48:28 +01:00
Jakob Unterwurzacher
00df0771e3 serialize_reads: add read serialization logic
Due to kernel readahead, we usually get multiple read requests
at the same time. These get submitted to the backing storage in
random order, which is a problem if seeking is very expensive.

Details: https://github.com/rfjakob/gocryptfs/issues/92
2017-03-18 16:18:00 +01:00
Jakob Unterwurzacher
14038a1644 fusefrontend: readFileID: reject files that consist only of a header
A header-only file will be considered empty (this is not supposed to happen).
This makes File ID poisoning more difficult.
2017-03-12 21:11:02 +01:00
Jakob Unterwurzacher
d36d53c9bb fusefrontend: truncateGrowFile: avoid createHeader() call
...if doWrite() can do it for us. This avoids the situation
that the file only consists of a file header when calling
doWrite.

A later patch will check for this condition and warn about it,
as with this change it should no longer occour in normal operation.
2017-03-12 21:06:59 +01:00
Jakob Unterwurzacher
9a0808b1ee configfile: HKDF feature flag should also be set for "-plaintextnames" 2017-03-07 21:05:45 +01:00
Jakob Unterwurzacher
2f953fdb95 contentenc: catch integer underflow in file size calculation
If you truncate a ciphertext file to 19 bytes, you could get the
impression that the plaintext is 18446744073709551585 bytes long,
as reported by "ls -l".

Fix it by clamping the value to zero.
2017-03-07 20:56:50 +01:00
Jakob Unterwurzacher
9f17a78b4a configfile: enable HKDF and Raw64 feature flags by default
Also adds a test to verify that they are set in new config
files.
2017-03-07 20:56:50 +01:00
M. Vefa Bicakci
d48ccb3dda Report correct symbolic link dentry sizes
Prior to this commit, gocryptfs's reverse mode did not report correct
directory entry sizes for symbolic links, where the dentry size needs to
be the same as the length of a string containing the target path.

This commit corrects this issue and adds a test case to verify the
correctness of the implementation.

This issue was discovered during the use of a strict file copying program
on a reverse-mounted gocryptfs file system.
2017-03-07 20:46:58 +01:00
Jakob Unterwurzacher
6e9b6e17c3 tests: configfile: fix spurious test failure II
internal/configfile/config_test.go:67: c declared and not used
2017-03-05 23:24:47 +01:00
Jakob Unterwurzacher
b878306d2a tests: configfile: fix spurious test failure
This test fails because Raw64 has been disabled for now.
2017-03-05 23:15:50 +01:00
Jakob Unterwurzacher
a8fd8a2516 configfile: disable Raw64 for now
Raw64 is supported (but was disabled by default) since gocryptfs
v1.2. However, the implementation was buggy because it forgot
about long names and symlinks.

Disable it for now by default and enable it later, together
with HKDF.
2017-03-05 23:04:54 +01:00
Jakob Unterwurzacher
445b5019e3 nametransform: fix Raw64 not affecting symlink targets
The symlink functions incorrectly hardcoded the padded
base64 variant.
2017-03-05 22:59:25 +01:00
Jakob Unterwurzacher
5b54577d2e nametransform: fix Raw64 not affecting longnames
HashLongName() incorrectly hardcoded the call to base64.URLEncoding.
2017-03-05 22:25:41 +01:00
Jakob Unterwurzacher
d0bc7970f7 full stack: implement HKDF support
...but keep it disabled by default for new filesystems.

We are still missing an example filesystem and CLI arguments
to explicitely enable and disable it.
2017-03-05 21:59:55 +01:00
Jakob Unterwurzacher
4fadcbaf68 configfile: reject the "HKDF" flag for now
This will be re-enabled once it is implemented.
2017-03-05 18:16:49 +01:00
Jakob Unterwurzacher
decda6d255 configfile: switch on Raw64 by default
As we have dropped Go 1.4 compatibility already, and will add
a new feature flag for gocryptfs v1.3 anyway, this is a good
time to enable Raw64 as well.
2017-03-05 18:13:56 +01:00
Jakob Unterwurzacher
b732881518 configfile: switch to 128-bit IVs for master key encryption
There is no security reason for doing this, but it will allow
to consolidate the code once we drop compatibility with gocryptfs v1.2
(and earlier) filesystems.
2017-03-05 18:03:03 +01:00
Jakob Unterwurzacher
966308eeb7 Drop Go 1.4 compatability code everywhere
Yields a nice reduction in code size.
2017-03-05 17:44:14 +01:00
Jakob Unterwurzacher
874e4fb5e9 cryptocore: rename "BackendTypeEnum" -> "AEADTypeEnum"
There are two independent backends, one for name encryption,
the other one, AEAD, for file content.

"BackendTypeEnum" only applies to AEAD (file content), so make that
clear in the name.
2017-03-05 17:10:57 +01:00
Jakob Unterwurzacher
e032539e2c cryptocore: use eme v1.1 interface
Version 1.1 of the EME package (github.com/rfjakob/eme) added
a more convenient interface. Use it.

Note that you have to upgrade your EME package (go get -u)!
2017-03-05 13:58:24 +01:00
Jakob Unterwurzacher
6cc0aebd71 configfile: define HKDF flag 2017-03-05 12:08:12 +01:00
Jakob Unterwurzacher
b2f3dbb8bd fusefrontend: when chown'ing a directory, also chown its diriv
When filename encryption is active, every directory contains
a "gocryptfs.diriv" file. This file should also change the owner.

Fixes https://github.com/rfjakob/gocryptfs/issues/86
2017-03-02 19:12:21 +01:00
Jakob Unterwurzacher
427c6c1719 exitcodes: define code 12 for "password incorrect" 2017-02-26 19:25:23 +01:00
Jakob Unterwurzacher
57612a278b configfile: rename "kdf.go" -> "scrypt.go"
This really only handles scrypt and no other key-derivation functions.
Renaming the files prevents confusion once we introduce HKDF.

renamed:    internal/configfile/kdf.go -> internal/configfile/scrypt.go
renamed:    internal/configfile/kdf_test.go -> internal/configfile/scrypt_test.go
2017-02-25 18:51:17 +01:00
Jakob Unterwurzacher
a65965783a stupidgcm: drop only external dependecy
This makes it easier to use the package in external projects.

See https://github.com/rfjakob/gocryptfs/issues/79
2017-02-24 09:46:10 +01:00
Jakob Unterwurzacher
43d6aa6677 speed: add benchmark.bash helper 2017-02-24 09:38:50 +01:00
Jakob Unterwurzacher
477071d673 speed: fix build for Go 1.4 and lower
Old Go versions miss cipher.NewGCMWithNonceSize, which causes:

  internal/speed/speed.go:95: undefined: cipher.NewGCMWithNonceSize
2017-02-23 00:04:51 +01:00
Jakob Unterwurzacher
1e03e059fa Implement "gocryptfs -speed"
A crypto benchmark mode like "openssl speed".

Example run:

  $ ./gocryptfs -speed
  AES-GCM-256-OpenSSL 	 180.89 MB/s	(selected in auto mode)
  AES-GCM-256-Go      	  48.19 MB/s
  AES-SIV-512-Go      	  37.40 MB/s
2017-02-22 23:56:34 +01:00
Jakob Unterwurzacher
62e7eb7d04 tests: reverse: check Access() call 2017-02-16 21:20:29 +01:00
Jakob Unterwurzacher
45c1ea499e fusefrontend_reverse: handle .name files in Access()
These were currently passed to decryptPath() were it caused
a warning.
2017-02-16 21:16:42 +01:00
Jakob Unterwurzacher
55d0523dbe tests: configfile: add missing newlines in verbose output 2017-02-16 19:45:20 +01:00
Jakob Unterwurzacher
8bbc1038fe syscallcompat: OSX compat: fix variable warnings
As suggested by
https://github.com/rfjakob/gocryptfs/issues/15#issuecomment-279130217
2017-02-16 19:23:17 +01:00
Jakob Unterwurzacher
3784901fce readpassword: limit password length to 1000 bytes
This used to hang at 100% CPU:

    cat /dev/zero | gocryptfs -init a

...and would ultimately send the box into out-of-memory.

The number 1000 is chosen arbitrarily and seems big enough
given that the password must be one line.

Suggested by @mhogomchungu in https://github.com/rfjakob/gocryptfs/issues/77 .
2017-02-13 09:13:22 +01:00
Jakob Unterwurzacher
8adfbf2dc3 Check for trailing garbage after the password
From the comment:

// CheckTrailingGarbage tries to read one byte from stdin and exits with a
// fatal error if the read returns any data.
// This is meant to be called after reading the password, when there is no more
// data expected. This helps to catch problems with third-party tools that
// interface with gocryptfs.
2017-02-12 17:59:09 +01:00
Jakob Unterwurzacher
2dd90ac19c readpassword: improve comment on readPasswordStdin 2017-02-12 13:55:30 +01:00
Jakob Unterwurzacher
0f40afc832 ctlsock: handle non-canonical empty paths
We have to check if the input path is empty AFTER canonicalizing it,
too!
2017-02-05 18:17:30 +01:00
Jakob Unterwurzacher
8bcae63a5a ctlsock: sanitize: handle multiple leading slashes 2017-02-05 18:05:35 +01:00
Jakob Unterwurzacher
2bdd0ec802 golint comment fix
internal/ctlsock/ctlsock_serve.go:73:1: comment on exported const
ReadBufSize should be of the form "ReadBufSize ..."
2017-01-29 19:01:16 +01:00
Jakob Unterwurzacher
1273d7edae ctlsock: better error message for forward mode path decryption 2017-01-29 18:55:52 +01:00
Jakob Unterwurzacher
53fe6f5690 ctlsock: abort message processing on JSON error
The code was missing a "continue" in that branch.

Also improve the error messages a bit.
2017-01-29 18:25:46 +01:00
Jakob Unterwurzacher
532ef15417 ctlsock: interpret paths that point above CWD as ""
Paths that start with ".." were previously accepted as-is.
2017-01-29 15:24:47 +01:00
Jakob Unterwurzacher
6166dad05c readpassword: support spaces in "-passfile" filename
...and while we are at it, also filenames starting with "-".
2017-01-29 00:34:12 +01:00
Jakob Unterwurzacher
39eca53677 main: make sure the ctlsock file is deleted on incorrect password
Otherwise the next try to mount ends in
"ctlsock: listen unix ctl.sock: bind: address already in use"
2017-01-26 21:32:08 +01:00
Jakob Unterwurzacher
a7c7588deb fusefrontend: fix hard-linking with long name
This used to incorrectly try to link twice and return EEXIST.
2017-01-26 20:56:42 +01:00
Jakob Unterwurzacher
de87a256ab tlog: better comment for the terminal colors 2017-01-26 20:01:45 +01:00
Jakob Unterwurzacher
fe1705c55b reverse: add single-entry path cache
Speeds up the "ls -lR" benchmark from 2.6 seconds to 2.0.
2017-01-03 18:14:01 +01:00
Jakob Unterwurzacher
ef089c9f80 reverse: factor out rDecryptName
This prepares the code for the introduction of a path cache.
2017-01-03 17:46:11 +01:00
Jakob Unterwurzacher
8be54bf3f8 prefer_openssl: default to Go GCM on OSX 2016-12-10 21:04:17 +01:00
Jakob Unterwurzacher
12374be9c5 ctlsock: abort the connection if the request is too big
Reading partial JSON would cause a mess. Just kill the connection.

Also, stop using syscall.PathMax that is not defined on Darwin
( https://github.com/rfjakob/gocryptfs/issues/15#issuecomment-264253024 )
2016-12-10 20:43:08 +01:00
Jakob Unterwurzacher
8945f4db95 ctlsock: add a note about Accept() throwing errors on Close() 2016-12-10 15:01:00 +01:00
Jakob Unterwurzacher
6af725ff09 ctlsock: exit early if socket cannot be created; delete on exit
Both are achieved by opening the socket from main and passing
it to the ctlsock package instead of passing the path.
2016-12-10 14:54:06 +01:00
Jakob Unterwurzacher
967e7bd125 stupidgcm: Fix without_openssl build failure
Also, always call build-without-openssl.bash from test.bash.

Failure was:

  internal/stupidgcm/without_openssl.go:29: missing return at end of function
2016-12-10 13:15:19 +01:00