Commit Graph

837 Commits

Author SHA1 Message Date
Jakob Unterwurzacher
23180794fe v2api: implement Readlink 2020-07-04 20:32:02 +02:00
Jakob Unterwurzacher
1618fbbac5 v2api: implement Unlink 2020-06-21 14:08:53 +02:00
Jakob Unterwurzacher
cc0b94a3c5 v2api: implement Rmdir 2020-06-21 13:57:04 +02:00
Jakob Unterwurzacher
192a29075a v2api: implement Mkdir 2020-06-21 13:46:08 +02:00
Jakob Unterwurzacher
f6ded09e36 v2api: implement Create 2020-06-21 13:25:12 +02:00
Jakob Unterwurzacher
74a4accf0c v2api: collect RootNode code in root_node.go 2020-06-21 12:44:32 +02:00
Jakob Unterwurzacher
ebdf58b9eb v2api: implement GetAttr and Readdir 2020-06-21 12:42:18 +02:00
Jakob Unterwurzacher
6aa9f5636f v2api: implement Lookup()
Compiles, but untested otherwise. No caching.
2020-06-21 12:01:34 +02:00
Jakob Unterwurzacher
9b8ce55383 inomap: clarify TranslateStat function comment
Also, fix the namespaceMap comment.
2020-06-21 12:01:34 +02:00
Jakob Unterwurzacher
2aad58f9ec v2api (go-fuse v2 api): initial noop implementation
Compiles and mounts but does nothing useful.
2020-06-21 12:01:24 +02:00
Jakob Unterwurzacher
261cf12829 badname: stop trying to decrypt at length 22
At least 16 bytes due to AES --> at least 22 characters in base64
2020-06-06 12:49:45 +02:00
DerDonut
a8230d271f Added auto decryption of invalid file names
Changed invalid file decoding and decryption. Function
DecryptName now shortens the filename until the filename is
decodable and decryptable. Will work with valid **and**
invalid Base64URL delimiter (valid delimiter [0-9a-zA-z_\\-].
If the filename is not decryptable at all, it returns the
original cipher name with flag suffix Changed cli tests to
generate decryptable and undecryptable file names with correct
encrypted content. Replacing #474, extends #393
2020-06-06 12:20:31 +02:00
Jakob Unterwurzacher
71c0481f0e Revert "fusefrontend: don't always clear the dircache in Rename"
As noticed by @slackner in
cb8872577d (commitcomment-39405233) ,
this is not safe.

This reverts commit cb8872577d.
2020-05-24 23:36:11 +02:00
Jakob Unterwurzacher
15ff79bf14 syscallcompat: warn when Getdents truncates data
On CIFS mounts, unix.Getdents can return sudden ENOENT
in the middle of data. This will not be reported as an error
by user space tools, so return EIO instead.

Also log it as a warning.

https://github.com/rfjakob/gocryptfs/issues/483
2020-05-24 23:30:25 +02:00
Jakob Unterwurzacher
cb8872577d fusefrontend: don't always clear the dircache in Rename
When filename encryption is on, we do know when we
overwrite a directory, and can clear only in this case.

sshfs-benchmark.bash:    sshfs  gocryptfs-on-sshfs
git init                  1.74                7.80
rsync                     6.19               11.63
2020-05-24 20:19:27 +02:00
Jakob Unterwurzacher
2a9b99a0ef fusefrontend: don't clear dircache on Mkdir
Mkdir can not cause existing entries in the cache to go
stale. So don't clear it. Benchmark results:

sshfs-benchmark.bash:    sshfs  gocryptfs-on-sshfs
git init                  1.65                8.74
rsync                     6.09               17.54
2020-05-24 15:30:14 +02:00
Jakob Unterwurzacher
25f1727de9 syscallcompat: getdents: retry on EINTR
Fixes: https://github.com/rfjakob/gocryptfs/issues/483
Related: https://github.com/golang/go/issues/38836
2020-05-23 22:54:23 +02:00
Jakob Unterwurzacher
f8ad2ac3e2 dircache: increase cache size & lifetime
Looking at the dircache debug output, we see
that a "git status" workload has a very bad
cache hit rate because the entries expire or
get evicted before they can be reused.

Increase both cache size and lifetime for
a 4x speedup:

Before: 75s
After:  17s

https://github.com/rfjakob/gocryptfs/issues/410
2020-05-17 21:37:36 +02:00
Jakob Unterwurzacher
f6088e5008 dircache: improve debug messages
Before:

Lookup "errno.html/1/2/3/4/5": miss
Store: "errno.html/1/2/3/4/5" fd=26 iv=21be6e083d60dcabfe7368264d5082b7
Lookup "errno.html": hit 25 6d68a16d217978915036a3bd55428ae7
Lookup "errno.html/1": hit 25 932a464c299b3430c5e55c924f98ac4d
Lookup "errno.html/1/2": hit 25 7d53348b1692d537f017bf86b3cf5feb
Lookup "errno.html/1/2/3": hit 25 2aef1c9d1ab2b55b163215053fefe703
Lookup "errno.html/1/2/3/4": hit 25 cb802be53721c46a46247c5e4e0f4ce6
Lookup "errno.html/1/2/3/4": hit 25 cb802be53721c46a46247c5e4e0f4ce6
Lookup "errno.html": hit 25 6d68a16d217978915036a3bd55428ae7

After:

Lookup "earlyoom/.git/refs"                     hit fd=10 dup=17 iv=6ae2cecd269a25e8d946aff6afe9b8b8
Lookup "earlyoom/.git/refs/remotes"             hit fd=19 dup=17 iv=f04c2d2a5bcc33ebdeaca664859c980d
Lookup "earlyoom/.git/refs/remotes/origin"      miss
Store  "earlyoom/.git/refs/remotes/origin"      fd=17 iv=834a64a1697c9f5705455ba6dbed22b5
Lookup "earlyoom"                               hit fd=7 dup=25 iv=2303a892d6e2357c483574a8070b7679
Lookup "earlyoom/.git"                          hit fd=11 dup=25 iv=d43ca4aff23720c57789c9f62f0aee00
Lookup "earlyoom/.git"                          hit fd=11 dup=25 iv=d43ca4aff23720c57789c9f62f0aee00
Lookup "earlyoom/.git/refs"                     hit fd=10 dup=25 iv=6ae2cecd269a25e8d946aff6afe9b8b8
Lookup "earlyoom/.git/refs/heads"               hit fd=13 dup=25 iv=f9245e7c066b9adc768a1a666da9fbc8
2020-05-17 21:26:56 +02:00
Jakob Unterwurzacher
416080203b main: accept multiple -passfile options
Each file will be read and then concatenated
for the effictive password. This can be used as a
kind of multi-factor authenticiton.

Fixes https://github.com/rfjakob/gocryptfs/issues/288
2020-05-17 19:31:04 +02:00
Jakob Unterwurzacher
ec74d1d2f4 Update go-fuse import path to github.com/hanwen/go-fuse/v2
We need
fd7328faf9
to fix a crash reported in https://github.com/rfjakob/gocryptfs/issues/430 :

  2019/10/30 17:14:16 Unknown opcode 2016
  panic: runtime error: invalid memory address or nil pointer dereference
  [signal SIGSEGV: segmentation violation code=0x1 addr=0x20 pc=0x508d38]

This patch is only in the v2.x.x branch. Upgrade to v2, as the
old API is also supported there.

Running

  git grep hanwen/go-fuse | grep -v hanwen/go-fuse/v2

to check for forgotten references comes back clean.
2020-05-17 14:23:47 +02:00
Jakob Unterwurzacher
ead7008a08 Fix spelling mistakes found by misspell
https://github.com/client9/misspell
2020-05-10 00:25:49 +02:00
Jakob Unterwurzacher
16221facb9 ctlsock: create exported ctlsock client library
The former interal ctlsock server package is renamed
to ctlsocksrv.
2020-05-09 17:36:41 +02:00
Jakob Unterwurzacher
feaeee90e2 inomap: fix TestSpill bit check
Wrong bit operator was used.
2020-05-03 20:28:26 +02:00
Jakob Unterwurzacher
8c9c68fb72 inomap: fix spillBit not set on 2nd hit
Also add a test for this.

Thanks @slackner for the comment.
2020-05-03 20:21:11 +02:00
Jakob Unterwurzacher
91f5c242a8 inomap: remove leftover debug output
This was committed by accident.
2020-05-03 20:01:12 +02:00
Jakob Unterwurzacher
518771e4e2 fusefrontend_reverse: use inomap for inode number translation
Gets rid of static inode number value limitations.

Fixes https://github.com/rfjakob/gocryptfs/issues/457
2020-05-03 15:22:10 +02:00
Jakob Unterwurzacher
483054efaa inomap: comment constants 2020-04-19 22:09:21 +02:00
Jakob Unterwurzacher
9f9d59ded9 inomap: rework logic to efficiently support flags
Adding flags allows to use inomap in reverse mode,
replacing the clunky inoBaseDirIV/inoBaseNameFile
logic that causes problems with high underlying
inode numbers ( https://github.com/rfjakob/gocryptfs/issues/457 )

Microbenchmarks (values below) show that the "SingleDev"
case is now much slower due to an extra map lookup,
but this has no visible effects in ./test.bash results,
so there was no time spent optimizing the case further.

$ go test -bench=.
goos: linux
goarch: amd64
pkg: github.com/rfjakob/gocryptfs/internal/inomap
BenchmarkTranslateSingleDev-4   	18757510	        61.5 ns/op
BenchmarkTranslateManyDevs-4    	18061515	        64.5 ns/op
PASS
ok  	github.com/rfjakob/gocryptfs/internal/inomap	2.467s
2020-04-19 22:00:56 +02:00
Jakob Unterwurzacher
fcdeb52390 inomap: add benchmark
$ go test -bench=.
goos: linux
goarch: amd64
pkg: github.com/rfjakob/gocryptfs/internal/inomap
BenchmarkTranslateSingleDev-4   	202479382	         5.88 ns/op
BenchmarkTranslateManyDevs-4    	16095795	        71.9 ns/op
PASS
ok  	github.com/rfjakob/gocryptfs/internal/inomap	3.039s
2020-04-19 21:35:06 +02:00
Jakob Unterwurzacher
f6b1c680b3 nametransform: update comment & simplify tests
The comment still mentioned CBC, which has been removed
a long time ago.

The test definition can be rewritten using slice literals,
saving sume stuttering.
2020-04-18 16:14:48 +02:00
Jakob Unterwurzacher
8f5c2a613d Prefer Go stdlib aes-gcm on arm64 with aes instructions
We used to prefer openssl in this situation, which
used to make sense, but now Go gained an optimized
assembly implementation for aes-gcm on arm64 with
aes instructions:

  root@q1:~/go/src/github.com/rfjakob/gocryptfs# ./gocryptfs -speed
  gocryptfs v1.7.1-46-g73436d9; go-fuse v1.0.1-0.20190319092520-161a16484456; 2020-04-13 go1.14.2 linux/arm64
  AES-GCM-256-OpenSSL      212.30 MB/s    (selected in auto mode)
  AES-GCM-256-Go           452.30 MB/s
  AES-SIV-512-Go           100.25 MB/s
  XChaCha20-Poly1305-Go    137.35 MB/s

https://github.com/rfjakob/gocryptfs/issues/452
2020-04-13 22:34:07 +02:00
Jakob Unterwurzacher
488111ce39 inomap: split into separate package
inomap will also be used by fusefrontend_reverse
in the future. Split if off openfiletable to make
it independent.
2020-04-13 14:54:04 +02:00
Jakob Unterwurzacher
194030f18a speed: add XChaCha20-Poly1305-Go
https://github.com/rfjakob/gocryptfs/issues/452
2020-04-13 14:54:04 +02:00
Jakob Unterwurzacher
f82b9caa9c speed: add code comments 2020-02-29 21:26:28 +01:00
Jakob Unterwurzacher
ca9e912a28 fusefrontend: drop xattr user namespace restriction
We used to restrict setting xattrs to the "user."
namespace. I don't see a real reason for this
anymore, and it causes trouble for users who are using
acls.

Tests will be added in the next commit.

https://github.com/rfjakob/gocryptfs/issues/453
2020-02-29 20:12:43 +01:00
orcas
9ec042f2f6 Show undecryptable filenames if they match supplied glob
Resolves https://github.com/rfjakob/gocryptfs/issues/393
2020-02-28 22:17:59 +01:00
Jakob Unterwurzacher
abc59fa968 contentenc: encryptBlocksParallel: explain why last part runs in new goroutine
The result is counter-intuitive, so explain it here.
2020-02-15 21:28:12 +01:00
Jakob Unterwurzacher
d5ce340c02 merge prefer_openssl package into stupidgcm
Now that I have discovered golang.org/x/sys/cpu and that Go
versions below 1.6 are uncommon, there was not much useful
code left in prefer_openssl.

Merge the remains into stupidgcm.
2020-02-15 17:21:30 +01:00
Jakob Unterwurzacher
9aeb2a3df6 contentenc: move parallel encryption into encryptBlocksParallel
Make the logic self-contained in the new helper function.
2020-02-15 15:49:12 +01:00
Pavol Rusnak
1364b44ae3 remove Trezor support 2019-12-28 19:50:49 +01:00
Jakob Unterwurzacher
8ec804d825 fs: add uint64 cast to fix Darwin build
Error from Travis CI was:

 +GOOS=darwin
 +GOARCH=amd64
 +go build -tags without_openssl
 # github.com/rfjakob/gocryptfs/internal/fusefrontend
 internal/fusefrontend/fs.go:88:45: cannot use st.Dev (type int32) as type uint64 in argument to openfiletable.NewInumMap

Add uint64 to fix it.
2019-11-16 23:21:37 +01:00
Jakob Unterwurzacher
bb6155a51f fusefrontend: use inummap
translate inode numbers on different devices to fix
collisions.

Fixes https://github.com/rfjakob/gocryptfs/issues/435
2019-11-16 21:36:27 +01:00
Jakob Unterwurzacher
e5d5ab3973 openfiletable: add inummap
Generates unique inode numbers for files on different
devices.

https://github.com/rfjakob/gocryptfs/issues/435
2019-11-16 21:35:26 +01:00
Jakob Unterwurzacher
e4148028af fusefrontend: warn about missing diriv
The comment is outdated, at this point, we should
really not get any errors from ReadDirIVAt.

The change is best seen when running the fsck tests. Before:

  fsck: error opening dir "missing_diriv": 2=no such file or directory

After:

  OpenDir "K2m0E6qzIfoLkVZJanoUiQ": could not read gocryptfs.diriv: no such file or directory
  fsck: error opening dir "missing_diriv": 5=input/output error

See https://github.com/rfjakob/gocryptfs/issues/403 , where
the extra info would have been helpful.
2019-11-03 20:38:15 +01:00
Jakob Unterwurzacher
93939961f5 fusefrontend: don't return EIO on directory with corrupt file names
This was meant as a way to inform the user that
something is very wrong, however, users are hitting
the condition on MacOS due to ".DS_Store" files, and
also on NFS due to ".nfsXXX" files.

Drop the whole thing as it seems to cause more pain
than gain.

Fixes https://github.com/rfjakob/gocryptfs/issues/431
2019-11-03 20:12:05 +01:00
Jakob Unterwurzacher
00af4f4864 fusefrontend: unregister from openfiletable before closing the fd
Closing the fd means the inode number may be reused immediately
by a new file, so we have to get the old fileID out of the table
beforehand!

Hopefully fixes https://github.com/rfjakob/gocryptfs/issues/363
2019-10-06 21:47:36 +02:00
Jakob Unterwurzacher
43265940fa fusefrontend: print file hexdump on header error
This should help debugging https://github.com/rfjakob/gocryptfs/issues/363 ,
but does no harm in normal operation as it only prints ciphertext to the log.
2019-10-06 19:37:51 +02:00
Jakob Unterwurzacher
b3c88f573f contentenc: add explicit test for all-zero header
https://github.com/rfjakob/gocryptfs/issues/363
2019-10-06 19:37:51 +02:00
Jakob Unterwurzacher
56eb85651b serialize_reads: drop superflous "continue"
Found while debugging https://github.com/rfjakob/gocryptfs/pull/413
2019-09-08 21:19:44 +02:00
Jakob Unterwurzacher
0a4db7d9e9 Fix -idle unmounting despite activity
Fixes https://github.com/rfjakob/gocryptfs/issues/421
2019-09-08 16:29:20 +02:00
Jakob Unterwurzacher
b1468a732f Fix unix2syscall_darwin.go build failure
Error was

 +GOOS=darwin
 +GOARCH=amd64
 +go build -tags without_openssl
 # github.com/rfjakob/gocryptfs/internal/syscallcompat
 internal/syscallcompat/unix2syscall_darwin.go:22:32: u.Atimespec undefined (type unix.Stat_t has no field or method Atimespec)
 internal/syscallcompat/unix2syscall_darwin.go:23:32: u.Mtimespec undefined (type unix.Stat_t has no field or method Mtimespec)
 internal/syscallcompat/unix2syscall_darwin.go:24:32: u.Ctimespec undefined (type unix.Stat_t has no field or method Ctimespec)

caused by 87c872767d (diff-4913a9178621eadcdf191db17915fbcb)
2019-05-19 21:04:33 +02:00
Sebastian Lackner
a97d14c42d syscallcompat: fetch supplementary groups for OpenatUser & friends
Handled the same way in GlusterFS, disorderfs, libfuse.
Fixes https://github.com/rfjakob/gocryptfs/issues/394
2019-05-01 17:52:07 +02:00
Jakob Unterwurzacher
3ac9872230 tests: split testParentDir by UID
When we run tests as root, they will leave root-owned files
in testParentDir, which causes trouble when we run tests as
a normal user later on. Split by UID.
2019-05-01 13:12:44 +02:00
Jakob Unterwurzacher
7d81494a64 fusefrontend: get rid of last hardcoded "gocryptfs.diriv" instances
Makes it easier to change the name (as some people want to):
https://github.com/rfjakob/gocryptfs/issues/37
2019-04-09 20:51:33 +02:00
Jakob Unterwurzacher
fe06e9f456 readpassword: delete CheckTrailingGarbage
CheckTrailingGarbage was called even when "-passfile" was
used, which is stupid, and causes false positives:

https://github.com/rfjakob/gocryptfs/issues/391
(false error "Received trailing garbage after the password"
when using -passfile in .bash_profile)

Instead of trying to improve the logic to handle that case
and make everything even more complicated, delete the function.

It is unclear if actually helps in some cases, and it definitely
harms as shown by the above bug report.
2019-04-08 20:18:45 +02:00
Jakob Unterwurzacher
8459bb15c1 configfile: fall back to sync() if fsync() fails
This can happen on network drives: FRITZ.NAS mounted on MacOS returns
"operation not supported": https://github.com/rfjakob/gocryptfs/issues/390
2019-04-08 19:54:54 +02:00
Jakob Unterwurzacher
ec17445b99 forward mode: create gocryptfs.diriv files with 0440 permissions
Makes it easier to share an encrypted folder via a network drive.

https://github.com/rfjakob/gocryptfs/issues/387
2019-03-30 20:06:40 +01:00
Jakob Unterwurzacher
fba9b2b995 reverse: don't show gocryptfs.conf if a custom config path was passed
GetAttr checks for this, but OpenDir did not.

https://github.com/rfjakob/gocryptfs/issues/385
2019-03-26 20:59:52 +01:00
Eduardo M KALINOWSKI
3bc100aeb3 reverse mode: support wildcard exclude (--exclude-wildcard)
This adds support for gitignore-like wildcards and exclude patters in
reverse mode. It (somewhat) fixes #273: no regexp support, but the
syntax should be powerful enough to satisfy most needs.

Also, since adding a lot of --exclude options can be tedious, it adds
the --exclude-from option to read patterns from a file (or files).
2019-03-26 20:56:37 +01:00
Jakob Unterwurzacher
cf27037f20 Allow multiple -extpass arguments
To support arguments containing spaces, -extpass can now
be passed multiple times.

https://github.com/rfjakob/gocryptfs/issues/289
2019-03-03 13:25:30 +01:00
Jakob Unterwurzacher
179471b648 ParseHeader: print hexdump on error
Should help debugging https://github.com/rfjakob/gocryptfs/issues/363
2019-02-17 17:13:20 +01:00
Jakob Unterwurzacher
19cb6d046a nametransform: reject names longer than 255 chars
Looks like we allowed creating longer names by accident.
Fix that, and add a test that verifies it.
2019-02-17 17:05:05 +01:00
Jakob Unterwurzacher
16fac26c57 Disable Trezor support
Trezor support has been broken since Sept 2018
( https://github.com/rfjakob/gocryptfs/issues/261 ).

Disable trezor.go by renaming to trezor.go.broken.
This keeps "dep" from having to pull in A LOT OF dependencies:

Before:

  $ du -sh vendor/
  49M	vendor/

After:

  $ du -sh vendor/
  16M	vendor/
2019-02-04 22:08:03 +01:00
Jakob Unterwurzacher
b08cbce5c1 fusefronted: comment why Access() does not check context.Uid
https://github.com/rfjakob/gocryptfs/issues/347
2019-01-20 14:37:36 +01:00
Jakob Unterwurzacher
962c523644 fusefrontend: ensure directories without W or X perms can be deleted
This fixed the "Permission denied" bug, but still has the problem that
the directory may be replaced behind our back. Mitigated by the fact
that we skip the workaround when running as root with -allow_other.

https://github.com/rfjakob/gocryptfs/issues/354
2019-01-20 14:29:28 +01:00
Jakob Unterwurzacher
3d6b2685fb Revert "syscallcompat: drop Faccessat AT_SYMLINK_NOFOLLOW helper"
Breaks mounting on MacOS: unix.Faccessat on Darwin does NOT (yet)
support AT_SYMLINK_NOFOLLOW. See d44fe89ba4 .

This reverts commit 0805a63df1.
2019-01-20 13:10:59 +01:00
Jakob Unterwurzacher
0805a63df1 syscallcompat: drop Faccessat AT_SYMLINK_NOFOLLOW helper
unix.Faccessat has added support for AT_SYMLINK_NOFOLLOW in July 2018,
bd9dbc187b (diff-341484dbbe3180cd7a31ef2ad2d679b6)
which means we no longer need our own helper.

Closes https://github.com/rfjakob/gocryptfs/issues/347
2019-01-20 12:59:59 +01:00
Jakob Unterwurzacher
8c09df03aa ctlsock: fix buffer truncation of JSON unmarshal error
In the error case, buf was not restored to the original
capacity. Instead of truncating "buf" and restoring (or forgetting to restore)
later, introduce the "data" slice.

Fixes https://github.com/rfjakob/gocryptfs/issues/356
2019-01-20 12:13:49 +01:00
Sebastian Lackner
452b8b00f4 fusefrontend: Always use intFd() method instead of int(f.fd.Fd()). 2019-01-16 20:55:20 +01:00
Sebastian Lackner
682e642cfa fusefrontend: Rework the Utimens handling on macOS.
For Linux, everything effectively stays the same. For both path-based and
fd-based Utimens() calls, we use unix.UtimesNanoAt(). To avoid introducing
a separate syscall wrapper for futimens() (as done in go-fuse, for example),
we instead use the /proc/self/fd - trick.

On macOS, this changes quite a lot:

* Path-based Utimens() calls were previously completely broken, since
  unix.UtimensNanoAt() ignores the passed file descriptor. Note that this
  cannot be fixed easily since there IS no appropriate syscall available on
  macOS prior to High Sierra (10.13). We emulate this case by using
  Fchdir() + setattrlist().

* Fd-based Utimens() calls were previously translated to f.GetAttr() (to
  fill any empty parameters) and syscall.Futimes(), which does not does
  support nanosecond precision. Both issues can be fixed by switching to
  fsetattrlist().

Fixes https://github.com/rfjakob/gocryptfs/issues/350
2019-01-16 20:55:20 +01:00
Sebastian Lackner
8f33145651 fusefrontend: Print 'too many open files' warning for both short and long names.
While we're at it, also replace os.* constants with syscall.* constants.
2019-01-15 22:07:37 +01:00
Sebastian Lackner
d8bb223dd3 fusefrontend_reverse: Delete leftover debug statement. 2019-01-15 22:07:37 +01:00
Jakob Unterwurzacher
20140e24ed tests: reduce noise on MacOS
This should get rid of

    Openat: O_NOFOLLOW missing: flags = 0x0
    Fchmodat: adding missing AT_SYMLINK_NOFOLLOW flag
    sys_common_test.go:203: chmod on symlink should have failed, but did not. New mode=0333
    UnmountErr: "[...]/057376762.mnt" was not found in MountInfo, cannot check for FD leak

and add some context to

    --- FAIL: TestUtimesNano (0.00s)
    matrix_test.go:628: no such file or directory

See https://github.com/rfjakob/gocryptfs/pull/343#issuecomment-453888006
for full test output
2019-01-14 22:11:15 +01:00
Jakob Unterwurzacher
6542ddd2f9 syscallcompat: fix FchmodatNofollow tests
FchmodatNofollow dropped the flags parameter.
2019-01-14 21:57:24 +01:00
Jakob Unterwurzacher
a7d59032d3 syscallcompat: rework Fchmodat to FchmodatNofollow
We never want Fchmodat to follow symlinks, so follow what
Qemu does, and call our function FchmodatNofollow.
2019-01-14 21:54:16 +01:00
Sebastian Lackner
a9d8eb49ef syscallcompat: Drop Fstatat emulation on macOS. 2019-01-14 21:27:28 +01:00
Sebastian Lackner
4134ff7570 syscallcompat: Drop Mkdirat emulation on macOS. 2019-01-14 21:27:28 +01:00
Sebastian Lackner
7b0d56fe98 syscallcompat: Drop Symlinkat emulation on macOS. 2019-01-14 21:27:28 +01:00
Sebastian Lackner
92110628ee syscallcompat: Drop Fchownat emulation on macOS. 2019-01-14 21:27:28 +01:00
Sebastian Lackner
0345cc0830 syscallcompat: Drop Fchmodat emulation on macOS.
On macOS the function has a flags argument, so we don't need the
/proc/self/fd trick used on Linux.
2019-01-14 21:27:28 +01:00
Sebastian Lackner
229a9da74b syscallcompat: Drop Unlinkat emulation on macOS. 2019-01-14 21:27:28 +01:00
Sebastian Lackner
42bf6d1c68 syscallcompat: Drop Renameat emulation on macOS. 2019-01-14 21:27:28 +01:00
Sebastian Lackner
da557702d7 syscallcompat: Drop Openat emulation on macOS. 2019-01-14 21:27:28 +01:00
Sebastian Lackner
d7be766851 syscallcompat: Use pthread_setugid_np() to implement *User() functions on macOS.
Fixes -allow_other mode on macOS.
2019-01-14 21:27:28 +01:00
Jakob Unterwurzacher
e885f08746 fusefrontend: drop last remaining call into loopbackFileSystem
The only call forwarded to loopbackFileSystem was Statfs,
which is trivial to implement.

Implement it and drop loopbackFileSystem, as having it carries the
risk that a coding error bypasses the usual encryption/decryption
chain.
2019-01-13 20:27:35 +01:00
Sebastian Lackner
d44fe89ba4 fusefrontend: Do not pass unsupported flags to Faccessat on macOS.
Fixes mounting of forward mounts on macOS High Sierra.
2019-01-13 14:10:34 +01:00
Jakob Unterwurzacher
711ef81bfb macos: filter SUID and SGID bits in OpenatUser, MknodatUser, MkdiratUser
When gocryptfs runs as root, we don't want to allow people to create
SUID root files.
2019-01-13 14:05:03 +01:00
Sebastian Lackner
99e8b6d288 fusefrontend: Preserve SUID/SGID/sticky-bits in openWriteOnlyFile and Rmdir.
Fixes https://github.com/rfjakob/gocryptfs/issues/336 and
https://github.com/rfjakob/gocryptfs/issues/337.
2019-01-12 21:24:50 +01:00
Sebastian Lackner
efc280330c fusefrontend: -allow_other: Use SymlinkatUser in Symlink FUSE call.
Instead of manually adjusting the user after creating the symlink,
adjust effective permissions and let the kernel deal with it.

Related to https://github.com/rfjakob/gocryptfs/issues/338.
2019-01-12 21:22:58 +01:00
Sebastian Lackner
1fbe7798cf fusefrontend: -allow_other: Use MknodatUser in Mknod FUSE call.
Instead of manually adjusting the user and mode after creating the
device file, adjust effective permissions and let the kernel deal
with it.

Related to https://github.com/rfjakob/gocryptfs/issues/338.
2019-01-12 21:20:16 +01:00
Sebastian Lackner
a525e33eaa fusefrontend: -allow_other: Use MkdiratUser in Mkdir FUSE call.
Revert commit fcaca5fc94.

Instead of manually adjusting the user and mode after creating the
directory, adjust effective permissions and let the kernel deal with it.

Related to https://github.com/rfjakob/gocryptfs/issues/338.
2019-01-12 21:20:07 +01:00
Sebastian Lackner
03b9d65cce fusefrontend: -allow_other: Use OpenatUser in Create FUSE call.
Revert commit b22cc03c75.

Instead of manually adjusting the user and mode after creating the
file, adjust effective permissions and let the kernel deal with it.

Related to https://github.com/rfjakob/gocryptfs/issues/338.
2019-01-12 20:54:39 +01:00
Sebastian Lackner
669322482a fusefrontend: Don't chown gocryptfs.diriv files.
The current code has a risk of race-conditions, since we pass a path
containing "/" to Fchownat. We could fix this by opening a file descriptor,
however, this does not seem worth the effort. We also don't chown *.name files.
2019-01-12 20:35:50 +01:00
Sebastian Lackner
fcaca5fc94 fusefrontend: -allow_other: set file mode after chown in Mkdir().
Make sure that the directory belongs to the correct owner before users
can access it. For directories with SUID/SGID mode, there is a risk of
race-conditions when files are created before the correct owner is set.
They will then inherit the wrong user and/or group.

See https://github.com/rfjakob/gocryptfs/issues/327 for more details.
2019-01-09 20:48:00 +01:00
Sebastian Lackner
aae45b4d77 nametransform: Create *.name files with 0400 permission.
Similar to gocryptfs.iv files they are never modified.
2019-01-09 20:42:18 +01:00
Jakob Unterwurzacher
b22cc03c75 fusefrontend: -allow_other: set file mode *after* chown in Create()
Reported by @slackner at https://github.com/rfjakob/gocryptfs/issues/327 :

Possible race-conditions between file creation and Fchownat

* Assume a system contains a gocryptfs mount as root user
  with -allow_other
* As a regular user create a new file with mode containing
  the SUID flag and write access for other users
* Before gocryptfs executes the Fchownat call, try to open
  the file again, write some exploit code to it, and try to run it.

For a short time, the file is owned by root and has the SUID flag, so
this is pretty dangerous.
2019-01-08 21:50:10 +01:00
Sebastian Lackner
4170ef00f3 syscallcompat: Implement workaround for Fchmodat with AT_SYMLINK_NOFOLLOW.
Fixes https://github.com/rfjakob/gocryptfs/issues/259
2019-01-07 23:07:53 +01:00
Jakob Unterwurzacher
8253c55386 tests: add Fchmodat test
Test that we handle symlinks correctly.
2019-01-07 23:07:53 +01:00
Sebastian Lackner
57a52d6aef fusefrontend: Clarify access mode check related to O_WRONLY handling.
Use O_ACCMODE mask in openWriteOnlyFile for improved readability.
2019-01-07 21:25:23 +01:00
Sebastian Lackner
2332462e78 fusefrontend: Filter O_CREAT in mangleOpenFlags. 2019-01-07 21:19:13 +01:00
Sebastian Lackner
aa2fa24c42 fusefrontend: Check result of Fchmod syscall.
Fixes https://github.com/rfjakob/gocryptfs/issues/328
2019-01-06 21:00:17 +01:00
Sebastian Lackner
40d2427fd7 fusefrontend: Fix computation of cipherSz in Allocate FUSE call.
Do not use PlainSizeToCipherSize() since this adds the 18 bytes file header.

Partially fixes https://github.com/rfjakob/gocryptfs/issues/311
2019-01-06 20:56:59 +01:00
Sebastian Lackner
8310dd95be fusefrontend: Properly convert plaintext <-> ciphertext offsets in SeekData().
Fixes https://github.com/rfjakob/gocryptfs/issues/304
2019-01-06 12:21:54 +01:00
Sebastian Lackner
7e05e809b7 main: Run 'ensure fds' code early during the program startup.
The files are apparently processed in alphabetic order, so cli_args.go is
processed before main.go. In order to run before the go-fuse imports, put
the 'ensure fds' code in a separate package. Debug messages are omitted
to avoid additional imports (that might contain other code messing up our
file descriptors).
2019-01-05 16:12:16 +01:00
Jakob Unterwurzacher
ad15ad9985 main: ensure fds 0,1,2 are always open
The Go stdlib, as well as the gocryptfs code, relies on the fact
that fds 0,1,2 are always open.

See https://github.com/rfjakob/gocryptfs/issues/320 for details.
2019-01-05 14:17:51 +01:00
Sebastian Lackner
5055f39bd5 fusefrontend: Allow to set/remove xattr on directory without read permission.
Setting/removing extended attributes on directories was partially fixed with
commit eff35e60b6. However, on most file systems
it is also possible to do these operations without read access (see tests).

Since we cannot open a write-access fd to a directory, we have to use the
/proc/self/fd trick (already used for ListXAttr) for the other operations aswell.
For simplicity, let's separate the Linux and Darwin code again (basically revert
commit f320b76fd1), and always use the
/proc/self/fd trick on Linux. On Darwin we use the best-effort approach with
openBackingFile() as a fallback.

More discussion about the available options is available in
https://github.com/rfjakob/gocryptfs/issues/308.
2019-01-05 12:34:40 +01:00
Sebastian Lackner
f17721c364 A few more spelling fixes. 2019-01-05 12:27:55 +01:00
Jakob Unterwurzacher
65eded4a22 tests: bump maxCacheFds to 3
As the dirCache now has 3 entries, the tests should accept
up to 3 extra fds without declaring an fd leak.
2019-01-04 23:50:01 +01:00
Jakob Unterwurzacher
c32066c5b0 fusefrontend: fix fd leak in dirCache
The missing break meant that we may find a second
hit in the cache, Dup() a second fd, and leak the first
one.

Thanks @slackner for finding this.
2019-01-04 23:35:48 +01:00
Jakob Unterwurzacher
8074f12beb nametransform: ReadDirIVAt: return raw syscall error
Otherwise this can happen, as triggered by xfstests generic/011:

  go-fuse: can't convert error type: openat failed: too many open files

The app then gets a misleading "Function not implemented" error.
2019-01-04 23:21:27 +01:00
Jakob Unterwurzacher
3473a84963 fusefrontend: print warning when Create() runs out of file descriptors
We alread have this warning in Open(), but xfstests generic/488
causes "too many open files" via Create. Add the same message so
the user sees what is going on.
2019-01-04 23:15:04 +01:00
Jakob Unterwurzacher
eff35e60b6 fusefrontend: fix setting xattrs on directories
Directories cannot be opened read-write. Retry with RDONLY.
2019-01-04 22:22:24 +01:00
Jakob Unterwurzacher
3365cfc02b fusefrontend: disable dirCache stats printing
This was inadvertedly kept enabled after benchmarking.
2019-01-04 22:07:02 +01:00
Jakob Unterwurzacher
c0a7a14cde fusefrontend: upgrade to three-entry dirCache
3 entries should work well for up to three parallel users.
It works well for extractloop.bash (two parallel tar extracts).
2019-01-04 21:45:03 +01:00
Sebastian Lackner
117dc3f2cc fusefrontend_reverse: Fix redeclaration of 'entries' variable.
Go version go1.10.7 linux/amd64 complains with:

 internal/fusefrontend_reverse/rfs.go:333: declaration of "entries" shadows
 declaration at internal/fusefrontend_reverse/rfs.go:327
2019-01-04 20:11:45 +01:00
Jakob Unterwurzacher
6b94f5ef51 reverse mode: -exclude: filter out excluded .name files
Fixes https://github.com/rfjakob/gocryptfs/issues/286 :

While the actual file is properly excluded, the * .name file is still leaked in the directory listing:

```
drwxr-xr-x 2 sebastian sebastian 4,0K Dez 17 14:58 .
drwxr-xr-x 7 sebastian sebastian 4,0K Dez 17 14:45 ..
-r-------- 1 sebastian sebastian  408 Dez 17 14:56 gocryptfs.conf
-r--r--r-- 1 sebastian sebastian   16 Dez 17 14:58 gocryptfs.diriv
-r--r--r-- 1 sebastian sebastian  320 Dez 17 14:58 gocryptfs.longname.3vZ_r3eDPb1_fL3j5VA4rd_bcKWLKT9eaxOVIGK5HFA.name
```
2019-01-04 17:59:00 +01:00
Jakob Unterwurzacher
75a3e2c2ee reverse mode: fix "-exclude" in "-plaintextnames" dir listings
Excluded files showed up in directory listing like this:

 drwxr-xr-x 2 sebastian sebastian 4,0K Dez 17 14:48 .
 drwxr-xr-x 7 sebastian sebastian 4,0K Dez 17 14:45 ..
 -????????? ? ?         ?            ?            ? abcd
 -r-------- 1 sebastian sebastian  366 Dez 17 14:45 gocryptfs.conf

Fixes https://github.com/rfjakob/gocryptfs/issues/285
2019-01-04 17:36:06 +01:00
Sebastian Lackner
acf7e52022 fusefrontend: Allow to create sparse file of size 4096.
When the old size is zero, there are no existing blocks to merge the
new data with. Directly use Ftruncate if the size is block-aligned.

Fixes https://github.com/rfjakob/gocryptfs/issues/305
2019-01-04 01:38:47 +01:00
Sebastian Lackner
ab169443fd A few more spelling fixes.
Found with the 'codespell' utility.
2019-01-04 01:23:44 +01:00
Sebastian Lackner
a1ba4b6576 Omit syscall.O_RDONLY flag when passing O_PATH.
When O_PATH is specified in flags, flag bits other than O_CLOEXEC, O_DIRECTORY,
and O_NOFOLLOW are ignored.
2019-01-03 18:24:05 +01:00
Sebastian Lackner
0414ef2572 fusefrontend: Use appropriate flags in decryptPathAt. 2019-01-03 18:24:05 +01:00
Sebastian Lackner
c0640ff3ef fusefrontend: Open directory with syscall.O_DIRECTORY in OpenDir. 2019-01-03 18:24:05 +01:00
Sebastian Lackner
078a431493 fusefrontend: Open directory with syscall.O_DIRECTORY in Rmdir. 2019-01-03 18:24:05 +01:00
Sebastian Lackner
885a341df6 fusefrontend: Do not Clear cache at end of Rmdir function.
We already do 'defer fs.dirCache.Clear()', so this is no longer required.
2019-01-03 18:24:05 +01:00
Sebastian Lackner
61241b0588 nametransform: Add implicit syscall.O_RDONLY flag. 2019-01-03 18:24:05 +01:00
Sebastian Lackner
927b3ce4cf syscallcompat: Use O_PATH to open base directory.
Also remove some unnecessary flags: When O_PATH is specified in flags, flag
bits other than O_CLOEXEC, O_DIRECTORY, and O_NOFOLLOW are ignored.
2019-01-03 18:24:05 +01:00
Sebastian Lackner
d86f9914ac fusefrontend: Remove debug code.
This code was accidentially added in 4f66d66755.
2019-01-03 18:24:05 +01:00
Jakob Unterwurzacher
fcdb4bec09 fusefronted: dirCache: fix bug handling ""
Bug looked like this:

  $ ls -l .
  total 0
  drwxrwxr-x. 2 jakob jakob 60 Jan  3 15:42 foo
  -rw-rw-r--. 1 jakob jakob  0 Jan  3 15:46 x

  $ ls -l .
  ls: cannot access '.': No such file or directory

(only happened when "" was in the dirCache)
2019-01-03 15:59:54 +01:00
Jakob Unterwurzacher
4f66d66755 fusefrontend: add dirCache 2019-01-03 15:31:13 +01:00
Jakob Unterwurzacher
f6dad8d0fa nametransform: simplify WriteDirIV to WriteDirIVAt
Un-spaghettify the function and let the callers open
the directory.
2019-01-03 15:31:13 +01:00
Jakob Unterwurzacher
0fd7637624 fusefrontend: use O_RDONLY in the ListXAttr fallback path
Copy-paste error.

https://github.com/rfjakob/gocryptfs/issues/308
2019-01-02 22:20:44 +01:00
Jakob Unterwurzacher
352f3147c5 fusefrontend: move openBackingDir into its own file
This function is in all fastpaths, will get a cache, and needs
its own file.
renamed:    internal/fusefrontend/names.go -> internal/fusefrontend/openbackingdir.go
renamed:    internal/fusefrontend/names_test.go -> internal/fusefrontend/openbackingdir_test.go
2019-01-02 21:52:52 +01:00
Jakob Unterwurzacher
2b12bba274 fusefronted: make EncryptPath symlink-safe
Finally allows us to delete EncryptPathDirIV.
2019-01-02 21:45:40 +01:00
Jakob Unterwurzacher
b214be5e3f fusefrontend: xattr: fix operations on files without read permissions
* listxattr is fixed via the /proc/self/fd trick
* setxattr,removexattr are fixed by opening the file O_WRONLY

Fixes https://github.com/rfjakob/gocryptfs/issues/308
2019-01-02 20:48:46 +01:00
Jakob Unterwurzacher
bb7f919674 fusefrontend: don't downgrade type needlessly 2019-01-02 20:45:08 +01:00
Jakob Unterwurzacher
f320b76fd1 fusefrontend: use Fsetxattr/Fgetxattr/etc on all platforms
Darwin now also has these functions, use them. Simplifies
the code and makes it symlink-safe on Darwin as well.
2019-01-02 16:58:48 +01:00
Jakob Unterwurzacher
7995a8358e syscallcompat: add Fgetxattr / Fsetxattr wrappers
These take care of buffer sizing and parsing.
2019-01-02 16:56:23 +01:00
Jakob Unterwurzacher
5aa1755cbc fusefrontend: openBackingDir: fix fd leak in error path
Reported by @slackner at
932efbd459 (r31813373)
thanks!
2019-01-02 00:14:12 +01:00
Jakob Unterwurzacher
d99a0480f7 nametransform: fix possible incomplete read in ReadLongNameAt
Pread() needs retry logic, so instead of implementing it ourselves,
use os.File.

Reported by @slackner at
c09bf1f228 (r31813394)
2019-01-02 00:09:17 +01:00
Jakob Unterwurzacher
cd0ec342b9 fusefrontend: fix fd leak in error path 2019-01-01 20:49:56 +01:00
Jakob Unterwurzacher
77c3df48ef fusefrontend: fix fd leak in Access()
Thanks @slackner!

Fixes https://github.com/rfjakob/gocryptfs/issues/306
2019-01-01 20:10:17 +01:00
Jakob Unterwurzacher
60e7a0ca9f fusefrontend: xattr: fix hang on FIFOs
An Open() a fifo blocks until it is opened for writing.
This meant that xattr operations on FIFOs would block.
Pass O_NONBLOCK to fix that, and add a test.
2019-01-01 16:24:25 +01:00
Jakob Unterwurzacher
1d5500c3db fusefrontend: only compile getBackingPath() on Darwin
This function is NOT symlink-safe. Darwin needs it because it lacks
fgetxattr(2) and friends.
2019-01-01 16:24:25 +01:00
Jakob Unterwurzacher
c3adf9729d fusefrontend: make ListXAttr symlink-safe on Linux
Uses /proc/self/fd.
2019-01-01 16:24:25 +01:00
Jakob Unterwurzacher
d3ae87fa2b fusefrontend: make RemoveXAttr() symlink-safe
Uses /proc/self/fd on Linux.
2019-01-01 16:24:25 +01:00
Jakob Unterwurzacher
810d2a8b47 fusefrontend: make SetXAttr() symlink-safe on Linux
Uses the /proc/self/fd trick.
2019-01-01 16:24:25 +01:00
Jakob Unterwurzacher
2286372603 fusefrontend: make GetXAttr() symlink-safe on Linux
Uses the /proc/self/fd trick, which does not work
on Darwin.
2019-01-01 16:24:25 +01:00
Jakob Unterwurzacher
a355670ca2 fusefrontend: make Utimens symlink-safe
unix.UtimesNanoAt now also exists on Darwin, yay!
2019-01-01 16:24:25 +01:00
Jakob Unterwurzacher
abbdaa8ea4 fusefrontend: fix compile failure on Darwin
Failure was:

 + GOOS=darwin
 + GOARCH=amd64
 + go build -tags without_openssl
 # github.com/rfjakob/gocryptfs/internal/fusefrontend
 internal/fusefrontend/fs_dir.go:159:60: cannot use origMode | 448 (type uint16) as type uint32 in argument to syscallcompat.Fchmodat
 internal/fusefrontend/fs_dir.go:170:33: cannot use origMode (type uint16) as type uint32 in argument to syscallcompat.Fchmodat
2019-01-01 16:24:25 +01:00
Jakob Unterwurzacher
d4b7f42c3b fusefrontend: mark Truncate, Unlink, Symlink symlink-safe
No changes needed.
2019-01-01 16:24:25 +01:00
Jakob Unterwurzacher
436f918c21 fusefrontend: make Rmdir symlink-safe
Now uses Unlinkat.
2019-01-01 16:24:25 +01:00
Jakob Unterwurzacher
2de3851abd nametransform: rename WriteLongName() -> WriteLongNameAt()
And also rename DeleteLongName() -> DeleteLongNameAt(). The
naming follow the names open the openat() etc syscalls.
2019-01-01 16:24:25 +01:00
Jakob Unterwurzacher
4fae240153 fusefrontend: make Readlink() symlink-safe
Now symlink-safe through Readlinkat().
2019-01-01 16:24:25 +01:00
Jakob Unterwurzacher
21f1f858b9 fusefrontend: make OpenDir() symlink-safe
Interestingly, little or no performance impact:

$ ./benchmark.bash
Testing gocryptfs at /tmp/benchmark.bash.39W: gocryptfs v1.6-42-g30c2349-dirty; go-fuse v20170619-66-g6df8ddc; 2018-11-04 go1.11
Downloading linux-3.0.tar.gz
/tmp/linux-3.0.tar.gz                  100%[=========================================================================>]  92.20M  2.93MB/s    in 31s
2018-11-04 21:44:44 URL:https://cdn.kernel.org/pub/linux/kernel/v3.0/linux-3.0.tar.gz [96675825/96675825] -> "/tmp/linux-3.0.tar.gz" [1]
WRITE: 262144000 bytes (262 MB, 250 MiB) copied, 1.1808 s, 222 MB/s
READ:  262144000 bytes (262 MB, 250 MiB) copied, 0.866438 s, 303 MB/s
UNTAR: 24.745
MD5:   12.050
LS:    3.525
RM:    9.544

Note: kernel has been updated:

$ uname -a
Linux brikett 4.18.16-200.fc28.x86_64 #1 SMP Sat Oct 20 23:53:47 UTC 2018 x86_64 x86_64 x86_64 GNU/Linux
2019-01-01 16:24:25 +01:00
Jakob Unterwurzacher
de3a2c1895 fusefrontend: mark a few more functions as symlink-safe / unsafe 2019-01-01 16:24:25 +01:00
Jakob Unterwurzacher
8586a83825 fusefrontend: use openBackingDir in ctlsock interface
Instead of calling syscall.Open() ourselves, rely on
openBackingDir().
2019-01-01 16:24:20 +01:00
Jakob Unterwurzacher
0c1ceed1fa fusefrontend: make GetAttr() symlink-safe
Use openBackingDir() and Fstatat().

High performance impact, though part of it should be
mitigated by adding DirIV caching to the new code paths.

$ ./benchmark.bash
Testing gocryptfs at /tmp/benchmark.bash.Eou: gocryptfs v1.6-37-ge3914b3-dirty; go-fuse v20170619-66-g6df8ddc; 2018-10-14 go1.11
WRITE: 262144000 bytes (262 MB, 250 MiB) copied, 1.2289 s, 213 MB/s
READ:  262144000 bytes (262 MB, 250 MiB) copied, 1.02616 s, 255 MB/s
UNTAR: 24.490
MD5:   13.120
LS:    3.368
RM:    9.232
2019-01-01 16:24:09 +01:00
Jakob Unterwurzacher
932efbd459 fusefrontend: make openBackingDir() symlink-safe
openBackingDir() used encryptPath(), which is not symlink-safe
itself. Drop encryptPath() and implement our own directory walk.

Adds three seconds to untar and two seconds to rm:

$ ./benchmark.bash
Testing gocryptfs at /tmp/benchmark.bash.MzG: gocryptfs v1.6-36-g8fb3c2f-dirty; go-fuse v20170619-66-g6df8ddc; 2018-10-14 go1.11
WRITE: 262144000 bytes (262 MB, 250 MiB) copied, 1.25078 s, 210 MB/s
READ:  262144000 bytes (262 MB, 250 MiB) copied, 1.0318 s, 254 MB/s
UNTAR: 20.941
MD5:   11.568
LS:    1.638
RM:    5.337
2019-01-01 16:24:09 +01:00
Jakob Unterwurzacher
0e2e7c13cf fusefrontend: mark symlink-safe FUSE calls
Document which FUSE calls are already symlink-safe in
the function comment.
2019-01-01 16:24:09 +01:00
Jakob Unterwurzacher
c09bf1f228 fusefrontend: make DecryptPath() symlink-safe
DecryptPath is now symlink-safe through the use of *at()
functions.
2019-01-01 16:24:09 +01:00
Jakob Unterwurzacher
ed6ed513d7 fusefrontend: make Access() symlink-safe.
Make Access() symlink-safe through use of faccessat.
2019-01-01 16:24:09 +01:00
Jakob Unterwurzacher
545a03da24 nametransform: comments: directly link to ioutil.WriteFile fix
So the reader does not have to read through the whole ticket.
The commit message has a nice summary of the problem.
2019-01-01 16:23:28 +01:00
Sebastian Lackner
5713154468 fusefrontend: Fix debug message in doWrite() method. 2019-01-01 16:12:42 +01:00
Sebastian Lackner
9ed60678e5 fusefrontend: Fix order of arguments in debug message for Read() FUSE call. 2019-01-01 16:12:05 +01:00
Sebastian Lackner
87ced5f95d nametransform: Delete incomplete longname files on error. 2019-01-01 16:09:57 +01:00
Sebastian Lackner
24594d99bf configfile: Fix a copy&paste error in validateParams method. 2018-12-28 09:58:46 +01:00
Sebastian Lackner
07c486603c configfile: Explicitly wipe scrypt derived key after decrypting/encrypting master key.
Further raises the bar for recovering keys from memory.
2018-12-27 18:47:14 +01:00
Sebastian Lackner
874eaf9734 Assorted spelling fixes.
Mostly detected with the 'codespell' utility, but also includes some
manual grammar fixes.
2018-12-27 15:19:55 +01:00
Sebastian Lackner
4c2ff26457 fusefrontend: Remove unnecessary check in doRead function.
The same condition is already checked a few lines above, and 'err' is not
changed inbetween.
2018-12-27 15:18:03 +01:00
Sebastian Lackner
1ced0b192e fusefrontend: Don't treat Fchownat error as failure in Mkdir.
The directory was already created, so return success even if Fchownat fails.
The same error handling is already used if fs.args.PlaintextNames is false.
2018-12-27 15:16:00 +01:00
Sebastian Lackner
5918884926 fusefrontend: Check the correct 'err' variable. 2018-12-27 15:11:23 +01:00
Jakob Unterwurzacher
a55e53c196 tests: fix TestPassfileNewline
Due to a copy-paste error, we ran the wrong test in the
subprocess.

Thanks @slackner for noticing at
295d432175 (r31690478) !
2018-12-16 12:33:25 +01:00
Jakob Unterwurzacher
295d432175 passfile: directly read file instead of invoking cat
Allows better error handling, gets rid of the call to an
external program, and fixes https://github.com/rfjakob/gocryptfs/issues/278 .
2018-12-15 17:09:38 +01:00
Jakob Unterwurzacher
e665df7179 syscallcompat: downgrade DT_UNKNOWN message level on XFS
Old XFS filesystems always return DT_UNKNOWN. Downgrade the message
to "info" level if we are on XFS.

Using the "warning" level means that users on old XFS filesystems
cannot run the test suite as it intentionally aborts on any
warnings.

Fixes https://github.com/rfjakob/gocryptfs/issues/267
2018-11-17 17:44:21 +01:00
Jakob Unterwurzacher
1ed08c7384 tlog: disable color codes when switching to syslog
When gocryptfs was started on a terminal and later
daemonized, the color codes stayed active in the syslog
output.

The codes are not visible in "journalctl -f", which is why
I have not noticed it yet, but they do show up in normal
syslog as the usual "#033[33m" crap.
2018-10-17 22:34:30 +02:00
Jakob Unterwurzacher
4cdf6b9af9 fusefronted: log more details on WriteAt failures
Also log inode number, fd number, offset and length.

Maybe help debugging https://github.com/rfjakob/gocryptfs/issues/269 .
2018-10-17 22:18:07 +02:00
Jesse Dunietz
87d3ed9187 Add option for autounmount
Even though filesystem notifications aren't implemented for FUSE, I decided to
try my hand at implementing the autounmount feature (#128). I based it on the
EncFS autounmount code, which records filesystem accesses and checks every X
seconds whether it's idled long enough to unmount.

I've tested the feature locally, but I haven't added any tests for this flag.
I also haven't worked with Go before. So please let me know if there's
anything that should be done differently.

One particular concern: I worked from the assumption that the open files table
is unique per-filesystem. If that's not true, I'll need to add an open file
count and associated lock to the Filesystem type instead.

https://github.com/rfjakob/gocryptfs/pull/265
2018-10-11 20:16:45 +02:00
Jakob Unterwurzacher
57a5a8791f tests: syscallcompat: allow failure for symlinks > 1000
MacOS and old XFS versions do not support very long symlinks,
but let's not make the tests fail because of that.

https://github.com/rfjakob/gocryptfs/issues/267
2018-10-11 19:45:47 +02:00
Jakob Unterwurzacher
e4f1a32a9a fusefrontend: Fix uint16 build failure on Darwin
Error was:

  # github.com/rfjakob/gocryptfs/internal/fusefrontend
  internal/fusefrontend/fs.go:179: cannot use perms | 256 (type uint16) as type uint32 in argument to syscall.Fchmod
  internal/fusefrontend/fs.go:185: cannot use perms (type uint16) as type uint32 in argument to syscall.Fchmod
2018-09-23 12:17:59 +02:00
Jakob Unterwurzacher
a1fb456618 fusefrontend: make Rename() symlink-safe
Use Openat() and the openBackingDir() helper so we
never follow symlinks.
2018-09-23 12:17:59 +02:00
Jakob Unterwurzacher
897bb8924f fusefrontend: make Create() symlink-safe
Use Openat() and the openBackingDir() helper so we
never follow symlinks.
2018-09-23 12:17:59 +02:00
Jakob Unterwurzacher
63762b33af fusefrontend: Open(): fix dirfd leak
Close was missing.
2018-09-23 12:17:59 +02:00
Jakob Unterwurzacher
bead82c9fb fusefrontend: add named parameters to openBackingDir
Named parameters make using the function easier.
2018-09-23 12:17:59 +02:00
Jakob Unterwurzacher
c270b21efc fusefrontend: get rid of os.File* wrapping
Directly use int file descriptors for the dirfd
and get rid of one level of indirection.
2018-09-23 12:17:26 +02:00
Jakob Unterwurzacher
22fba4ac3e fusefrontent: make Open() symlink-safe 2018-09-23 12:17:26 +02:00
Jakob Unterwurzacher
2d01d5f2d4 tlog: always trim trailing newlines
The messages we print through tlog sometimes do, sometimes do
not contain a trailing newline. The stdlib logger usually drops
trailing newlines automatically, but tlog postfixes ColorReset to
the caller's message, so the logger logic does not work when we
print colored output.

Drop the newlines on our own, and add a test.

Fixes the blank lines in fsck output:

~/go/src/github.com/rfjakob/gocryptfs/tests/fsck$ ./run_fsck.bash
Reading password from extpass program
Decrypting master key
OpenDir "": invalid entry "invalid_file_name.3": illegal base64 data at input byte 17
OpenDir "": invalid entry "invalid_file_name_2": bad message
fsck: corrupt entry in dir "": "invalid_file_name.3"
fsck: corrupt entry in dir "": "invalid_file_name_2"
OpenDir "": invalid entry "invalid_file_name____1": bad message
fsck: corrupt entry in dir "": "invalid_file_name____1"
doRead 4327225: corrupt block #0: stupidgcm: message authentication failed
fsck: error reading file "corrupt_file" (inum 4327225): 5=input/output error
cipherSize 40 < overhead 50: corrupt file

doRead 4327074: corrupt header: ParseHeader: invalid version, want=2 have=22616
cipherSize 40 < overhead 50: corrupt file

fsck: error reading file "corrupt_file_2" (inum 4327074): 5=input/output error
Readlink "s-P7PcQDUcVkoeMDnC3EYA": decrypting target failed: stupidgcm: message authentication failed
fsck: error reading symlink "corrupt_symlink": 5=input/output error
Readlink "iI0MtUdzELPeOAZYwYZFee169hpGgd3l2PXQBcc9sl4": decrypting target failed: illegal base64 data at input byte 0
fsck: error reading symlink "corrupt_symlink_2": 5=input/output error
OpenDir "yrwcjj2qoC4IYvhw9sbfRg": could not read gocryptfs.diriv: wanted 16 bytes, got 17
fsck: error opening dir "diriv_too_long": 5=input/output error
OpenDir "trqecbMNXdzLqzpk7fSfKw": could not read gocryptfs.diriv: wanted 16 bytes, got 3
fsck: error opening dir "diriv_too_short": 5=input/output error
cipherSize 8 < header size 18: corrupt file

readFileID 4327049: incomplete file, got 8 instead of 19 bytes
fsck: corrupt file "incomplete_file_1" (inode 4327049)
readFileID 4327038: incomplete file, got 18 instead of 19 bytes
fsck: corrupt file "incomplete_file_2" (inode 4327038)
cipherSize 1 < header size 18: corrupt file

readFileID 4327063: incomplete file, got 1 instead of 19 bytes
fsck: corrupt file "incomplete_file_3" (inode 4327063)
fsck: error opening dir "missing_diriv": 2=no such file or directory
ListXAttr: invalid xattr name "user.gocryptfs.0a5e7yWl0SGUGeWB0Sy2K0": bad message
fsck: corrupt xattr name on file "xattr_corrupt_name": "user.gocryptfs.0a5e7yWl0SGUGeWB0Sy2K0"
GetXAttr: stupidgcm: message authentication failed
fsck: error reading xattr "user.foo" from "xattr_corrupt_value": 5=input/output error
fsck summary: 15 corrupt files
2018-09-23 11:28:49 +02:00
Jakob Unterwurzacher
737a2f2012 syscallcompat: untangle Openat flag check
Check for O_NWFOLLOW and O_EXCL separately to
make the logic clearer.
2018-09-22 19:38:47 +02:00
Jakob Unterwurzacher
e8d8ae54d3 fusefrontend: use OpenDirNofollow in openBackingDir
Rename openBackingPath to openBackingDir and use OpenDirNofollow
to be safe against symlink races. Note that openBackingDir is
not used in several important code paths like Create().

But it is used in Unlink, and the performance impact in the RM benchmark
to be acceptable:

Before

	$ ./benchmark.bash
	Testing gocryptfs at /tmp/benchmark.bash.bYO: gocryptfs v1.6-12-g930c37e-dirty; go-fuse v20170619-49-gb11e293; 2018-09-08 go1.10.3
	WRITE: 262144000 bytes (262 MB, 250 MiB) copied, 1.07979 s, 243 MB/s
	READ:  262144000 bytes (262 MB, 250 MiB) copied, 0.882413 s, 297 MB/s
	UNTAR: 16.703
	MD5:   7.606
	LS:    1.349
	RM:    3.237

After

	$ ./benchmark.bash
	Testing gocryptfs at /tmp/benchmark.bash.jK3: gocryptfs v1.6-13-g84d6faf-dirty; go-fuse v20170619-49-gb11e293; 2018-09-08 go1.10.3
	WRITE: 262144000 bytes (262 MB, 250 MiB) copied, 1.06261 s, 247 MB/s
	READ:  262144000 bytes (262 MB, 250 MiB) copied, 0.947228 s, 277 MB/s
	UNTAR: 17.197
	MD5:   7.540
	LS:    1.364
	RM:    3.410
2018-09-08 19:27:33 +02:00
Jakob Unterwurzacher
930c37e03d syscallcompat: use O_PATH in OpenDirNofollow
This fixes the "0100 directory" problem in reverse mode,
and should be slightly faster.
2018-09-08 18:06:33 +02:00
Jakob Unterwurzacher
9ec9d0c49c syscallcompat: untangle OpenNofollow and rename to OpenDirNofollow
The function used to do two things:

1) Walk the directory tree in a manner safe from symlink attacks
2) Open the final component in the mode requested by the caller

This change drops (2), which was only used once, and lets the caller
handle it. This simplifies the function and makes it fit for reuse in
forward mode in openBackingPath(), and for using O_PATH on Linux.
2018-09-08 17:41:17 +02:00
Jakob Unterwurzacher
2bdf7d5172 configfile: add LoadAndDecrypt wrapper
Callers that do not want to decrypt the masterkey should
call plain Load().

https://github.com/rfjakob/gocryptfs/issues/258
2018-09-08 12:40:29 +02:00
Jakob Unterwurzacher
09d28c293e configfile: split off masterkey decryption
Preparation for fixing https://github.com/rfjakob/gocryptfs/issues/258
2018-09-08 12:19:19 +02:00
Jakob Unterwurzacher
21eaa8f164 configfile: return specific error on empty input
Report the actual problem instead of a generic
"unexpected end of JSON input".

https://github.com/rfjakob/gocryptfs/issues/258
2018-09-08 12:18:26 +02:00
Jakob Unterwurzacher
658cc4aebb syscallcompat: drop Fchmodat flags
These were silently ignored until now (!) but
are rejected by Go 1.11 stdlib.

Drop the flags so the tests work again, until
we figure out a better solution.

https://github.com/golang/go/issues/20130
2018-08-26 13:04:01 +02:00
Jakob Unterwurzacher
bd054e70ef trezor: show support in version string
Show enable_trezor in the version string if we were compiled
with `-tags enable_trezor`. And hide the `-trezor` flag from
the help output if we were not.
2018-08-15 23:31:37 +02:00
Jakob Unterwurzacher
dbd400d930 fusefrontend: truncateGrowFile: pass zeroPad error to caller
Errors from zeroPad were ignored until now, as discovered
using xfstests generic/083.
2018-08-15 17:25:22 +02:00
Jakob Unterwurzacher
7a02f71fc2 fusefrontend_reverse: reject excludes for the root directory ""
This is most likely a mistake by the user. Reject it.
2018-08-15 12:28:29 +02:00
Jakob Unterwurzacher
ec2fdc19cf reverse mode: add --exclude option
https://github.com/rfjakob/gocryptfs/issues/235
2018-08-11 23:26:49 +02:00
Jakob Unterwurzacher
f4a972ddf1 configfile: drop superflous Printf
Before:

  $ gocryptfs -fsck .
  LoadConfFile: ReadFile: &os.PathError{Op:"open", Path:"/var/tmp/check-gocryptfs/scratchdev/gocryptfs.conf", Err:0xd}
  Cannot open config file: open /var/tmp/check-gocryptfs/scratchdev/gocryptfs.conf: permission denied

After:

  $ gocryptfs -fsck .
  Cannot open config file: open /var/tmp/check-gocryptfs/scratchdev/gocryptfs.conf: permission denied
2018-07-23 22:25:40 +02:00
Jakob Unterwurzacher
f316f1b2df fusefronted: disallow writes running concurrently with reads
As uncovered by xfstests generic/465, concurrent reads and writes
could lead to this,

  doRead 3015532: corrupt block #1039: stupidgcm: message authentication failed,

as the read could pick up a block that has not yet been completely written -
write() is not atomic!

Now writes take ContentLock exclusively, while reads take it shared,
meaning that multiple reads can run in parallel with each other, but
not with a write.

This also simplifies the file header locking.
2018-07-22 22:29:22 +02:00
Jakob Unterwurzacher
c70df522d2 fusefrontend: doWrite: delete file header if first write fails
xfstests generic/083 fills the filesystem almost completely while
running fsstress in parallel. In fsck, these would show up:

  readFileID 2580: incomplete file, got 18 instead of 19 bytes

This could happen when writing the file header works, but writing
the actual data fails.

Now we kill the header again by truncating the file to zero.
2018-07-15 15:12:55 +02:00
Jakob Unterwurzacher
55bb22bad6 fusefrontend: doWrite: no need to take HeaderLock.RLock()
Other writers are blocked by ContentLock already.
2018-07-15 12:40:23 +02:00
Jakob Unterwurzacher
bbf5b72fff WriteDirIV: delete incomplete gocryptfs.diriv file if write fails
If the underlying filesystem is full, writing to gocryptfs.diriv may
fail, and later fsck show this:

	OpenDir "xyz": could not read gocryptfs.diriv: wanted 16 bytes, got 0

Uncovered by xfstests generic/083.

Also fixes a fd leak in the error path.
2018-07-15 12:02:39 +02:00
Jakob Unterwurzacher
bcca323cb7 contentenc: reserve one extra block in pool plaintext buffers
File holes and -fsck can cause unaligned read accesses, which means
we have to decrypt one extra plaintext block.

xfstests generic/083 manage to crash -fsck like this:

generic/083	2018/07/14 15:25:21 wrong len=266240, want=131072
panic: wrong len=266240, want=131072

goroutine 1 [running]:
log.Panicf(0x67fc00, 0x15, 0xc4204fec90, 0x2, 0x2)
	/usr/local/go/src/log/log.go:333 +0xda
github.com/rfjakob/gocryptfs/internal/contentenc.(*bPool).Put(0xc4200d4800, 0xc4202f2000, 0x21000, 0x41000)
	/home/jakob/go/src/github.com/rfjakob/gocryptfs/internal/contentenc/bpool.go:27 +0x15d
github.com/rfjakob/gocryptfs/internal/fusefrontend.(*File).doRead(0xc4200b4500, 0xc42019e000, 0x0, 0x20000, 0x28400, 0x20000, 0xc42019e000, 0xc4204ff008, 0x435164, 0xc420000180)
	/home/jakob/go/src/github.com/rfjakob/gocryptfs/internal/fusefrontend/file.go:227 +0xba9
github.com/rfjakob/gocryptfs/internal/fusefrontend.(*File).Read(0xc4200b4500, 0xc42019e000, 0x20000, 0x20000, 0x28400, 0x0, 0x0, 0x0)
	/home/jakob/go/src/github.com/rfjakob/gocryptfs/internal/fusefrontend/file.go:246 +0x23e
main.(*fsckObj).file(0xc420069320, 0xc42001a630, 0x21)
	/home/jakob/go/src/github.com/rfjakob/gocryptfs/fsck.go:126 +0x21f
main.(*fsckObj).dir(0xc420069320, 0xc420014dc0, 0x1d)
	/home/jakob/go/src/github.com/rfjakob/gocryptfs/fsck.go:76 +0x387
main.(*fsckObj).dir(0xc420069320, 0xc42021dae0, 0x19)
	/home/jakob/go/src/github.com/rfjakob/gocryptfs/fsck.go:74 +0x347
2018-07-15 11:39:19 +02:00
Jakob Unterwurzacher
95b93db35f fusefrontend: log prealloc failures at Info level
If the underlying filesystem is full, it is normal get ENOSPC here.
Log at Info level instead of Warning.

Fixes xfstests generic/015 and generic/027, which complained about
the extra output.
2018-07-14 15:18:27 +02:00
Jakob Unterwurzacher
53f7e1a0f0 macos: fix O_DIRECT build failure
O_DIRECT has no direct equivalent on MacOS
(check out https://github.com/libuv/libuv/issues/1600 for details).

Just define it to zero there.
2018-07-04 09:04:00 +02:00
Jakob Unterwurzacher
893e41149e fusefrontend: disallow O_DIRECT and fall back to buffered IO
O_DIRECT accesses must be aligned in both offset and length. Due to our
crypto header, alignment will be off, even if userspace makes aligned
accesses. Running xfstests generic/013 on ext4 used to trigger lots of
EINVAL errors due to missing alignment. Just fall back to buffered IO.
2018-07-02 23:54:37 +02:00
Jakob Unterwurzacher
c51fc9e07d fusefronted: downgrade fallocate message severity
The message causes output mismatches in xfstests generic/112.
Downgrade the severity to Info so it gets disabled when using "-q".
2018-07-02 23:03:43 +02:00
Jakob Unterwurzacher
01a078e7c0 Fix golint warnings 2018-07-01 22:00:06 +02:00
Jakob Unterwurzacher
5243cd0e0d trezor: hide behind compile tag
The trezor libraries are not yet stable enough to build
gocryptfs with trezor support by default.

It does not even compile at the moment:

  $ ./build.bash -tags enable_trezor
  # github.com/conejoninja/tesoro/vendor/github.com/trezor/usbhid
  ../../conejoninja/tesoro/vendor/github.com/trezor/usbhid/hid.go:32:11: fatal error: os/threads_posix.c: No such file or directory
    #include "os/threads_posix.c"
           ^~~~~~~~~~~~~~~~~~~~
  compilation terminated.

https://github.com/conejoninja/tesoro/issues/9
2018-07-01 21:48:51 +02:00
Jakob Unterwurzacher
e951043084 fusefrontend: add File.SeekData() function
This function will enable "gocryptfs -fsck" to handle
sparse files efficiently.
2018-07-01 20:56:22 +02:00
Jakob Unterwurzacher
a2af1fb5da fusefrontend: export "File" type
"gocryptfs -fsck" will need access to helper functions,
and to get that, it will need to cast a gofuse.File to a
fusefrontend.File. Make fusefrontend.File exported to make
this work.
2018-07-01 20:56:22 +02:00
Jakob Unterwurzacher
1a18d8e609 fsck: rename "CorruptItems" channel to "MitigatedCorruptions"
Make it clear that this channel is only used to report corruptions
that are transparently mitigated and do not return an error to
the user.
2018-07-01 20:56:22 +02:00
Jakob Unterwurzacher
6d64dfe8f7 Only print masterkey once on -init
It is no longer printed at all when mounting a filesystem,
printing on -init can be disabled with -q.

https://github.com/rfjakob/gocryptfs/issues/76
2018-07-01 20:56:22 +02:00
Jakob Unterwurzacher
991891a5c4 trezor: add sanity checks for decrypted value
Check that the value has changed, is not all-zero
and has the right length.
2018-07-01 20:56:22 +02:00
Dmitry Yu Okunev
978f1f3f6d Implemented the support of Trezor devices. 2018-07-01 20:56:22 +02:00
Jakob Unterwurzacher
9a15dfa494 trezor: add TrezorPayload
TrezorPayload stores 32 random bytes used for unlocking
the master key using a Trezor security module. The randomness makes sure
that a unique unlock value is used for each gocryptfs filesystem.
2018-07-01 20:56:22 +02:00
Jakob Unterwurzacher
91de77943f configfile: reduce function name stutter
configfile.LoadConfFile()   -> configfile.Load()
configfile.CreateConfFile() -> configfile.Create()
2018-07-01 20:56:22 +02:00
Jakob Unterwurzacher
c6f6e8ec4d trezor: add skeleton for Trezor support
readpassword.Trezor() is not implemented yet and returns
a hardcoded dummy key.
2018-07-01 20:56:04 +02:00
Jakob Unterwurzacher
743c7705b2 configfile: use tlog.ColorYellow instead of hardcoded color code 2018-06-24 19:52:47 +02:00
Jakob Unterwurzacher
1bab400fca Fix three golint warnings
We are clean again.

Warnings were:

internal/fusefrontend/fs.go:443:14: should omit type string from declaration
of var cTarget; it will be inferred from the right-hand side
internal/fusefrontend/xattr.go:26:1: comment on exported method FS.GetXAttr
should be of the form "GetXAttr ..."
internal/syscallcompat/sys_common.go:9:7: exported const PATH_MAX should have
comment or be unexported
2018-06-19 20:16:21 +02:00
Jakob Unterwurzacher
bfa50517e9 xattr: return EOPNOTSUPP instead of ENODATA in GetXattr
Reading system.posix_acl_access and system.posix_acl_default
should return EOPNOTSUPP to inform user-space that we do not
support ACLs.

xftestest essientially does

	chacl -l | grep "Operation not supported"

to determine if the filesystem supports ACLs, and used to
wrongly believe that gocryptfs does.
2018-06-12 23:05:53 +02:00
Jakob Unterwurzacher
bde7ba57b0 darwin does not have PATH_MAX
Define our own, with the value from Linux.
2018-06-08 00:47:48 +02:00
Jakob Unterwurzacher
ae02ca1ded xattr: use LGet/LSet etc
Support has been merged into the xattr package
( https://github.com/pkg/xattr/pull/29 ), use it.
2018-05-27 20:09:48 +02:00
Jakob Unterwurzacher
7b00681807 xattr: return EOPNOTSUPP for unsupported attributes
mv is unhappy when we return EPERM when it tries to set
system.posix_acl_access:

   mv: preserving permissions for ‘b/x’: Operation not permitted

Now we return EOPNOTSUPP like tmpfs does and mv seems happy.
2018-05-15 23:00:47 +02:00
Bolshevik
a41ec2028c xattr: optimize storage, store as binary instead of bae64
Values a binary-safe, there is no need to base64-encode them.

Old, base64-encoded values are supported transparently
on reading. Writing xattr values now always writes them binary.
2018-05-10 23:25:49 +02:00
Jakob Unterwurzacher
a276321dea stupidgcm: return error on too short input instead of panicing
This is what Go GCM does as well.
2018-05-10 23:00:02 +02:00
Bolshevik
5ccc06d5cb xattr: added passing of a "flags" parameter
Pass the "flags" parameter to the lower layer syscall.
This makes Apple applications being able to successfully save data.
2018-05-07 21:45:40 +02:00
Jakob Unterwurzacher
fe3890688a fusefrontend: xattr: return ENOSYS on unsupported flags
We previously returned EPERM to prevent the kernel from
blacklisting our xattr support once we get an unsupported
flag, but this causes lots of trouble on MacOS:
Cannot save files from GUI apps, see
https://github.com/rfjakob/gocryptfs/issues/229

Returning ENOSYS triggers the dotfiles fallback on MacOS
and fixes the issue.
2018-05-01 23:30:53 +02:00
bolshevik
12b32aa06c Improved xattr handling on non-linux systems (#227)
* Fixed xattr filtering for MacOS. "system." and "user." prefixes are only relevant for Linux.
* Small cleanup and additional tests.
2018-04-17 20:33:04 +02:00
Jakob Unterwurzacher
bcc8378a2c Fix the easy golint warnings
Reported by https://goreportcard.com/report/github.com/rfjakob/gocryptfs
2018-04-08 20:26:25 +02:00
Jakob Unterwurzacher
4e5783591f fsck: report skipped corrupt files
OpenDir and ListXAttr skip over corrupt entries,
readFileID treats files the are too small as empty.
This improves usability in the face of corruption,
but hides the problem in a log message instead of
putting it in the return code.

Create a channel to report these corruptions to fsck
so it can report them to the user.

Also update the manpage and the changelog with the -fsck option.

Closes https://github.com/rfjakob/gocryptfs/issues/191
2018-04-03 21:24:48 +02:00
Jakob Unterwurzacher
4407ca3a4d fusefrontend: xattr: return ENODATA for security.* and system.*
"ls -l" queries security.selinux, system.posix_acl_access, system.posix_acl_default
and throws error messages if it gets something else than ENODATA.
2018-04-02 18:59:14 +02:00
Jakob Unterwurzacher
a0fd3eca98 fsck: test against example_filesystems 2018-04-02 18:43:50 +02:00
Jakob Unterwurzacher
b6c8960b01 fsck: clean up log output
Make sure we get only 1 warning output per
problem.

Also, add new corruption types to broken_fs_v1.4.
2018-04-02 18:32:30 +02:00
Jakob Unterwurzacher
f28d85fad5 fsck: add initial implementation
Most corruption cases except xattr should be covered.
With test filesystem.

The output is still pretty ugly. xattr support will
be added in the next commits.
2018-04-02 16:38:18 +02:00
Jakob Unterwurzacher
fb06c65ee9 fusefronted: reject oversized Read and Write requests
This should not happen via FUSE as the kernel caps the size,
but with fsck we have the first user that calls Read directly.
For symmetry, check it for Write as well.
2018-04-01 21:21:55 +02:00
Jakob Unterwurzacher
1a3d04ab87 Switch from private copy to pkg/xattr
Now that https://github.com/pkg/xattr/pull/24
has been merged there is no reason to keep
our private copy.

Switch to the upstream version.
2018-03-28 19:19:58 +02:00
Jakob Unterwurzacher
db778aae7d fusefrontend: handle empty xattrs efficiently
We handle empty files by storing an actual empty file
on disk. Handle xattrs similarily and encrypt the
empty value to the empty value.
2018-03-25 21:06:10 +02:00
Jakob Unterwurzacher
1ed3d51df1 fusefrontend: add xattr support
At the moment, only for reverse mode.

https://github.com/rfjakob/gocryptfs/issues/217
2018-03-25 21:06:10 +02:00
Jakob Unterwurzacher
3d54fc3a3a fusefrontend: create helpers for symlink encryption
These will be reused by the upcoming xattr support.
2018-03-24 21:40:11 +01:00
Jakob Unterwurzacher
9bc039a4ba Add -masterkey=stdin functionality
https://github.com/rfjakob/gocryptfs/issues/218
2018-03-22 00:02:10 +01:00
Jakob Unterwurzacher
4732e33a9a macos: tests: fix deleting of scratch dir
macos rm does not understand --one-file-system,
and it cannot handle unreadable directories.
2018-03-05 23:40:08 +01:00
Jakob Unterwurzacher
7db5395c53 macos: fix second TestEmulateSymlinkat test failure 2018-03-05 21:20:07 +01:00
Jakob Unterwurzacher
c5243fc79e MacOS: don't test symlinks longer than 1000 bytes
The limit is much lower than on Linux.

https://github.com/rfjakob/gocryptfs/issues/213
2018-02-28 20:40:08 +01:00
Jakob Unterwurzacher
90f2fea7fb MacOS: fix TestEmulateSymlinkat test failure
On MacOS, symlinks don't have their own permissions,
so don't check for them.
2018-02-28 20:19:31 +01:00
Jakob Unterwurzacher
b96e3ee271 tlog: stop embedding log.Logger to prevent mistakes
A few places have called tlog.Warn.Print, which directly
calls into log.Logger due to embedding, losing all features
of tlog.

Stop embedding log.Logger to make sure the internal functions
cannot be called accidentially and fix (several!) instances
that did.
2018-02-28 09:02:18 +01:00
Jakob Unterwurzacher
db45f27671 ctlsock: don't Warn() on closed socket
This Warn() is causing panics in the test suite
on MacOS: https://github.com/rfjakob/gocryptfs/issues/213
2018-02-27 09:58:14 +01:00
Jakob Unterwurzacher
5ad9bda206 cryptocore: make AEADTypeEnum values explicit
We now print the number in a debug message, so define
the numeric values explicitely instead of using iota.

This way you don't have to understand how iota works
to find out what the number means. Lack of understanding
of how iota works is also the reason why the numbers
start at 3 (to keep the current behavoir).
2018-02-18 16:20:38 +01:00