Commit Graph

369 Commits

Author SHA1 Message Date
Jakob Unterwurzacher
2b991c9743 Add support for unpadded base64 filenames, "-raw64"
Through base64.RawURLEncoding.

New command-line parameter "-raw64".
2016-11-01 18:43:22 +01:00
Jakob Unterwurzacher
c2192cfcad fusefrontend: drop atime workarounds
The fix at https://github.com/hanwen/go-fuse/pull/131 has been merged.
Drop the workarounds and re-enable the tests.
2016-10-30 16:29:36 +01:00
Jakob Unterwurzacher
85f1fd0b0f fusefronted: more concise corrupt block log message
Calculating the block offset is easy enough, even more now
that gocryptfs-xray exists.
2016-10-28 21:18:36 +02:00
Jakob Unterwurzacher
a08d55f42d fusefronted: optimize NFS streaming writes by saving one Stat()
Stat() calls are expensive on NFS as they need a full network
round-trip. We detect when a write immediately follows the
last one and skip the Stat in this case because the write
cannot create a file hole.

On my (slow) NAS, this takes the write speed from 24MB/s to
41MB/s.
2016-10-28 21:17:53 +02:00
Jakob Unterwurzacher
d64ccf7cf4 fusefrontend: move hole padding check out of Write()
The details of the hole handling don't have to be in
Write, so move it away.
2016-10-25 22:37:45 +02:00
Jakob Unterwurzacher
6538dc15af fusefrontend: rename "createsHole" to clearer "createsCiphertextHole"
...and add comments for what is happening.
2016-10-25 21:19:37 +02:00
Jakob Unterwurzacher
aeda9721d0 Fix misspellings
Close https://github.com/rfjakob/gocryptfs/issues/54
2016-10-24 19:18:13 +02:00
Jakob Unterwurzacher
589748548f tests: add 1980.tar.gz extract test
Test that we get the right timestamp when extracting a tarball.

Also simplify the workaround in doTestUtimesNano() and fix the
fact that it was running no test at all.
2016-10-19 22:25:54 +02:00
Jakob Unterwurzacher
600ceece35 lint fixes 2016-10-19 01:12:45 +02:00
Jakob Unterwurzacher
891a3b4c8a fusefrontend: Utimens: one more band-aid
Revert once https://github.com/hanwen/go-fuse/pull/131 is merged.
2016-10-16 20:20:00 +02:00
Jakob Unterwurzacher
5144470e3d fusefrontend: Utimens: ugly band-aid for nil pointer crash in go-fuse
Crash is described at https://github.com/rfjakob/gocryptfs/issues/48 .
Revert this once https://github.com/hanwen/go-fuse/pull/131 is merged.
2016-10-16 15:08:05 +02:00
Jakob Unterwurzacher
35219d0022 fusefrontend: log missing gocryptfs.diriv
This can happen during normal operation when the directory has
been deleted concurrently. But it can also mean that the
gocryptfs.diriv is missing due to an error, so log the event
at "info" level.
2016-10-16 15:04:59 +02:00
Jakob Unterwurzacher
828f718483 fusefrontend: Also preserve the owner in Mkdir
This already worked for files but was missing for dirs.
2016-10-10 08:53:29 +02:00
Jakob Unterwurzacher
f054353bd3 reverse: make gocryptfs.conf mapping plaintextnames-aware
Only in plaintextnames-mode AND with the config file at the
default location it will be mapped into the mountpoint.

Also adds a test for that.
2016-10-08 20:57:38 +02:00
Valient Gough
b764917cd5 lint fixes 2016-10-04 23:18:33 +02:00
Jakob Unterwurzacher
a2510efe12 reverse: use per-purpose nonce generation
Also pull all the deterministic nonce code into fusefrontend_reverse
to greatly simplify the normal code path.
2016-09-29 21:56:49 +02:00
Jakob Unterwurzacher
5f4b16c00f Implement changes proposed by gosimple.
Also delete the unused "dirIVNameStruct", found by deadcode.
2016-09-25 19:48:21 +02:00
Jakob Unterwurzacher
c7b3150afc nametransform: delete unused function DecryptPathDirIV 2016-09-25 18:56:23 +02:00
Jakob Unterwurzacher
abd61d968d contentenc: rename constant "IVBitLen" to "DefaultIVBits" and clarify comment
128-bit IVs are NOT used everywhere.
2016-09-25 18:40:29 +02:00
Jakob Unterwurzacher
12808138ef contentenc: add "ExternalNonce" mode
This will be used for strong symlink encryption in reverse mode.
2016-09-25 17:44:19 +02:00
Jakob Unterwurzacher
32e55261ca fusefrontend: handle Readlink directly
Calling into go-fuse's loopbackFileSystem does not add
any value here.
2016-09-25 17:01:39 +02:00
Jakob Unterwurzacher
5f726aaa9d contentenc: add GCM-SIV support
Also add ReverseDummyNonce nonce generation.
2016-09-25 16:43:17 +02:00
Jakob Unterwurzacher
7f87ed78f2 cryptocore: add support for GCM-SIV 2016-09-25 16:43:17 +02:00
Jakob Unterwurzacher
fca1b82417 fusefrontend: relay Utimens to go-fuse
Commit af5441dcd9 has caused a
regression ( https://github.com/rfjakob/gocryptfs/issues/35 )
that is fixed by this commit.

The go-fuse library by now has all the syscall wrappers in
place to correctly handle Utimens, also for symlinks.

Instead of duplicating the effort here just call into go-fuse.

Closes #35
2016-09-25 16:30:29 +02:00
Jakob Unterwurzacher
af5441dcd9 fusefrontend: use NsecToTimespec() for Utimens
This fixes a build problem on 32-bit hosts:

  internal/fusefrontend/file.go:400: cannot use a.Unix() (type int64) as
  type int32 in assignment
  internal/fusefrontend/file.go:406: cannot use m.Unix() (type int64) as
  type int32 in assignment

It also enables full nanosecond timestamps for dates
after 1970.
2016-08-09 22:18:46 +02:00
Jakob Unterwurzacher
e8a234f658 Add godoc comments to all internal packages 2016-07-06 21:51:25 +02:00
Jakob Unterwurzacher
741bf0726e syscallcompat: OSX: add Mknodat wrapper
Protip: find naked *at syscalls using:

   git grep "syscall." | grep "at(" | grep -v syscallcompat
2016-07-03 20:22:22 +02:00
Jakob Unterwurzacher
d8524c7369 syscallcompat: OSX: add Unlinkat wrapper
Also, replace remaining naked syscall.Openat calls.
2016-07-03 20:17:40 +02:00
Jakob Unterwurzacher
1d7728959c fusefrontend: downgrade Renameat log message to debug 2016-07-03 20:08:09 +02:00
Jakob Unterwurzacher
79851bf6cc syscallcompat: OSX: add Renamat wrapper
Adds a poor man's renameat implementation for OSX.
2016-07-03 20:05:32 +02:00
Jakob Unterwurzacher
9b725c15cf syscallcompat: OSX: add Fallocate and Openat wrappers
...and convert all calls to syscall.{Fallocate,Openat}
to syscallcompat .

Both syscalls are not available on OSX. We emulate Openat and just
return EOPNOTSUPP for Fallocate.
2016-07-03 19:18:34 +02:00
Jakob Unterwurzacher
c9a472c12f syscallcompat: move syscall wrapper to their own package
We will get more of them as OSX also lacks support for openat.
2016-07-03 17:51:40 +02:00
Jakob Unterwurzacher
e574a6cc1f nametransform: hide detailed padding error behind the debug flag
unPad16 returns detailed errors including the position of the
incorrect bytes. Kill a possible padding oracle by lumping
everything into a generic error.

The detailed error is only logged if debug is active.
2016-07-03 15:35:58 +02:00
Jakob Unterwurzacher
54470baa23 fusefrontend: add fallocate support
Mode=0 (default) and mode=1 (keep size) are supported.
The patch includes test cases and the whole thing passed xfstests.

Fixes https://github.com/rfjakob/gocryptfs/issues/1 .
2016-07-02 19:52:09 +02:00
Jakob Unterwurzacher
04ad063515 fusefronted: move Truncate() and Allocate() to their own file
These are large complicated implementations that will share some
code.
2016-07-02 15:35:06 +02:00
Jakob Unterwurzacher
7b22b426b9 contentenc: rename PlaintextRange and CiphertextRange
The name could be misunderstood and actually caused a bug:
doWrite used to always preallocate 4128 instead of the actual
data length.
2016-07-02 00:12:36 +02:00
Jakob Unterwurzacher
f2b4d57068 fusefrontend: coalesce grows in Truncate()
We were growing the file block-by-block which was pretty
inefficient. We now coalesce all the grows into a single
Ftruncate. Also simplifies the code!

Simplistic benchmark: Before:

  $ time truncate -s 1000M foo
  real	0m0.568s

After:

  $ time truncate -s 1000M foo
  real	0m0.205s
2016-07-01 23:32:27 +02:00
Jakob Unterwurzacher
ae77d18527 fusefrontend: better comments for Truncate 2016-07-01 09:23:04 +02:00
Jakob Unterwurzacher
e994ffa27d fusefrontend: handle dir-overwrites-dir on XFS
XFS returns a different error code if you try to overwrite
a non-empty directory with a directory:

XFS:  mv: cannot move ‘foo’ to ‘bar/foo’: File exists
ext4: mv: cannot move 'foo' to 'bar/foo': Directory not empty

So have EEXIST trigger the Rmdir logic as well.

Fixes issue #20
Link: https://github.com/rfjakob/gocryptfs/issues/20
2016-06-30 23:17:54 +02:00
Jakob Unterwurzacher
3288d98703 fusefrontend: fix PlaintextNames versions of Mkdir, Rmdir
The "!fs.args.DirIV" special case was removed by b17f0465c7
but that, by accident, also removed the handling for
PlaintextNames.

Re-add it as an explicit PlaintextNames special case.

Also adds support for removing directories that miss their
gocryptfs.diriv file for some reason.
2016-06-27 00:27:36 +02:00
Jakob Unterwurzacher
0115588680 main, fusefrontend: enable suid functionality
FUSE filesystems are mounted with "nosuid" by default. If we run as root,
we can use device files by passing the opposite mount option, "suid".

Also we have to use syscall.Chmod instead of os.Chmod because the
portability translation layer "syscallMode" messes up the sgid
and suid bits.

Fixes 70% of the failures in xfstests generic/193. The remaining are
related to truncate, but we err on the safe side:

    $ diff -u tests/generic/193.out /home/jakob/src/fuse-xfstests/results//generic/193.out.bad
    [...]
     check that suid/sgid bits are cleared after successful truncate...
     with no exec perm
     before: -rwSr-Sr--
    -after:  -rw-r-Sr--
    +after:  -rw-r--r--
2016-06-26 20:13:21 +02:00
Jakob Unterwurzacher
23cc0657f4 fusefronted: preserve owner if running as root
If allow_other is set and we run as root, try to give newly created files to
the right user.
2016-06-26 19:18:13 +02:00
Jakob Unterwurzacher
38767ab527 fuserfrontend: support truncate(2) by wrapping ftruncate(2)
Support truncate(2) by opening the file and calling ftruncate(2)
While the glibc "truncate" wrapper seems to always use ftruncate, fsstress from
xfstests uses this a lot by calling "truncate64" directly.
2016-06-26 18:41:04 +02:00
Jakob Unterwurzacher
b558901e66 Drop deprecated "-gcmiv128" option
The GCMIV128 feature flag is already mandatory, dropping the command
line option is the final step.

Completes https://github.com/rfjakob/gocryptfs/issues/29 .
2016-06-23 22:10:19 +02:00
Jakob Unterwurzacher
3d59a72ba9 Drop deprecated "-emenames" option
The EMENames feature flag is already mandatory, dropping the command
line option is the final step.
2016-06-23 21:56:50 +02:00
Jakob Unterwurzacher
b17f0465c7 Drop deprecated "-diriv" option
The DirIV feature flag is already mandatory, dropping the command
line option is the final step.
2016-06-23 21:38:59 +02:00
Jakob Unterwurzacher
6c3f97399a Rename internal "toggledlog" package to "tlog"
tlog is used heavily everywhere and deserves a shorter name.

Renamed using sed magic, without any manual rework:

   find * -type f -exec sed -i 's/toggledlog/tlog/g' {} +
2016-06-15 23:30:44 +02:00
Jakob Unterwurzacher
393e531afd Fix warnings reported by Go 1.6 "go tool vet -shadow=true"
Warnings were:

  main.go:234: declaration of err shadows declaration at main.go:163:
  internal/fusefrontend/file.go:401: declaration of err shadows declaration at internal/fusefrontend/file.go:379:
  internal/fusefrontend/file.go:419: declaration of err shadows declaration at internal/fusefrontend/file.go:379:
  internal/fusefrontend/fs_dir.go:140: declaration of err shadows declaration at internal/fusefrontend/fs_dir.go:97:
2016-06-14 22:46:23 +02:00
Jakob Unterwurzacher
5b1eed35ee fusefrontend: Utimens: convert ENOENT to EBADF
If /proc/self/fd/X did not exist, the actual error is that the file
descriptor was invalid.

go-fuse's pathfs prefers using an open fd even for path-based operations
but does not take any locks to prevent the fd from being closed.
Instead, it retries the operation by path if it get EBADF. So this
change allows the retry logic to work correctly.

This fixes the error

    rsync: failed to set times on "/tmp/ping.Kgw.mnt/linux-3.0/[...]/.dvb_demux.c.N7YlEM":
    No such file or directory (2)

that was triggered by pingpong-rsync.bash.
2016-06-09 22:21:00 +02:00
Jakob Unterwurzacher
5da292828c fusefrontend: fix chown on dangling symlinks
We (actually, go-fuse) used to call Chown() instead of Lchown()
which meant that the operation would fail on dangling symlinks.

Fix this by calling os.Lchown() ourself. Also add a test case
for this.
2016-06-08 00:32:44 +02:00
Jakob Unterwurzacher
a602e798b1 fusefrontend: report an error if all files in a directory were invalid
Just presenting an empty directory means that the user does not know
that things went wrong unless he checks the syslog or tries to delete
the directory.

It would be nice to report the error even if only some files were
invalid. However, go-fuse does not allow returning the valid
directory entries AND an error.
2016-06-04 16:39:27 +02:00
Jakob Unterwurzacher
5dd9576a11 fusefrontend: replace unreliable "fd < 0" check
... with the "released" boolean.

For some reason, the "f.fd.Fd() < 0" check did not work reliably,
leading to nil pointer panics on the following wlock.lock().

The problem was discovered during fsstress testing and is unlikely
to happen in normal operations.

With this change, we passed 1700+ fsstress iterations.
2016-05-30 09:36:06 +02:00
Jakob Unterwurzacher
1648c54adb fusefrontend: use sync.Once for one-time warnings
Using a simple boolean was racy (which was harmless
in this case) and non-idomatic.
2016-05-29 22:50:03 +02:00
Jakob Unterwurzacher
fd53dfd2ad fusefronted: check Fstat return value on file create
The Fstat call should never fail, but still, if it does return an error
it should be handled properly.
2016-05-29 22:43:48 +02:00
Jakob Unterwurzacher
77813bdc13 fusefrontend: simplify wlockMap
mapMutex can be anonymous and using an RWMutex is overkill
because the lock protects very short sections.
2016-05-29 22:40:05 +02:00
Jakob Unterwurzacher
4c0cb37c50 fusefrontend: remove unused "forgotten" variable
The functionality has long been replaced by the fd < 0
check.
2016-05-29 13:46:47 +02:00
David Gnedt
a93bcabe9c Encrypt path in statfs() calls
Paths in statfs() calls were not encrypted resulting in
an Function not implemented error, when the unencrypted
path didn't exist in the underlying (encrypted)
filesystem.

$ df plain/existingdir
df: ‘plain/existingdir’: Function not implemented
2016-05-12 23:24:39 +02:00
Jakob Unterwurzacher
ba7c798418 fusefrontend: fix panic due to concurrently unregistered wlock
Commit 730291feab properly freed wlock when the file descriptor is
closed. However, concurrently running Write and Truncates may
still want to lock it. Check if the fd has been closed first.
2016-05-08 23:21:20 +02:00
Jakob Unterwurzacher
730291feab fusefrontend: fix wlock memory leak
The write lock was not freed on release, causing a slow memory leak.

This was noticed by running extractloop.bash for 10 hours.
2016-05-05 13:38:39 +02:00
Jakob Unterwurzacher
4d79fba285 prelloc: warn and continue if fallocate(2) is not supported
This makes gocryptfs work at all on ZFS.
See https://github.com/rfjakob/gocryptfs/issues/22 .
2016-04-20 22:47:31 +02:00
Jakob Unterwurzacher
bd5405189e Fix "go tool vet -shadow=true" warnings
Among those one real bug.
2016-04-10 21:31:15 +02:00
Jakob Unterwurzacher
db72fcea41 longnames: fix fsstress failure, use dirfd
Using dirfd-relative operations allows safe lockless handling
of the ".name" files.
2016-04-10 21:31:08 +02:00
Jakob Unterwurzacher
63d3e51734 longnames: use symbolic constants instead of naked ints 2016-04-10 12:36:43 +02:00
Jakob Unterwurzacher
653d4a619c longnames part II: Rename, Unlink, Rmdir, Mknod, Mkdir + tests 2016-02-07 14:02:09 +01:00
Jakob Unterwurzacher
6b5d977cce Move OpenDir to fs_dir.go 2016-02-07 11:29:54 +01:00
Jakob Unterwurzacher
e111e20649 longnames part I: Create and OpenDir work with long filenames > 176 bytes
Todo: Rename, Unlink, Rmdir, Mknod, Mkdir
2016-02-06 22:54:14 +01:00
Jakob Unterwurzacher
c74772bc8d Run go fmt 2016-02-06 20:23:36 +01:00
Jakob Unterwurzacher
b0ee5258b1 Fix tests - were broken by the refactoring 2016-02-06 20:22:45 +01:00
Jakob Unterwurzacher
9078a77850 Move pathfs_frontend to internal/fusefrontend
"git status" for reference:

renamed:    pathfs_frontend/args.go -> internal/fusefrontend/args.go
renamed:    pathfs_frontend/compat_darwin.go -> internal/fusefrontend/compat_darwin.go
renamed:    pathfs_frontend/compat_linux.go -> internal/fusefrontend/compat_linux.go
renamed:    pathfs_frontend/file.go -> internal/fusefrontend/file.go
renamed:    pathfs_frontend/file_holes.go -> internal/fusefrontend/file_holes.go
renamed:    pathfs_frontend/fs.go -> internal/fusefrontend/fs.go
renamed:    pathfs_frontend/fs_dir.go -> internal/fusefrontend/fs_dir.go
renamed:    pathfs_frontend/names.go -> internal/fusefrontend/names.go
renamed:    pathfs_frontend/write_lock.go -> internal/fusefrontend/write_lock.go
modified:   main.go
2016-02-06 19:27:59 +01:00