Commit Graph

85 Commits

Author SHA1 Message Date
Jakob Unterwurzacher
a08d55f42d fusefronted: optimize NFS streaming writes by saving one Stat()
Stat() calls are expensive on NFS as they need a full network
round-trip. We detect when a write immediately follows the
last one and skip the Stat in this case because the write
cannot create a file hole.

On my (slow) NAS, this takes the write speed from 24MB/s to
41MB/s.
2016-10-28 21:17:53 +02:00
Jakob Unterwurzacher
d64ccf7cf4 fusefrontend: move hole padding check out of Write()
The details of the hole handling don't have to be in
Write, so move it away.
2016-10-25 22:37:45 +02:00
Jakob Unterwurzacher
6538dc15af fusefrontend: rename "createsHole" to clearer "createsCiphertextHole"
...and add comments for what is happening.
2016-10-25 21:19:37 +02:00
Jakob Unterwurzacher
aeda9721d0 Fix misspellings
Close https://github.com/rfjakob/gocryptfs/issues/54
2016-10-24 19:18:13 +02:00
Jakob Unterwurzacher
589748548f tests: add 1980.tar.gz extract test
Test that we get the right timestamp when extracting a tarball.

Also simplify the workaround in doTestUtimesNano() and fix the
fact that it was running no test at all.
2016-10-19 22:25:54 +02:00
Jakob Unterwurzacher
600ceece35 lint fixes 2016-10-19 01:12:45 +02:00
Jakob Unterwurzacher
891a3b4c8a fusefrontend: Utimens: one more band-aid
Revert once https://github.com/hanwen/go-fuse/pull/131 is merged.
2016-10-16 20:20:00 +02:00
Jakob Unterwurzacher
5144470e3d fusefrontend: Utimens: ugly band-aid for nil pointer crash in go-fuse
Crash is described at https://github.com/rfjakob/gocryptfs/issues/48 .
Revert this once https://github.com/hanwen/go-fuse/pull/131 is merged.
2016-10-16 15:08:05 +02:00
Valient Gough
b764917cd5 lint fixes 2016-10-04 23:18:33 +02:00
Jakob Unterwurzacher
a2510efe12 reverse: use per-purpose nonce generation
Also pull all the deterministic nonce code into fusefrontend_reverse
to greatly simplify the normal code path.
2016-09-29 21:56:49 +02:00
Jakob Unterwurzacher
12808138ef contentenc: add "ExternalNonce" mode
This will be used for strong symlink encryption in reverse mode.
2016-09-25 17:44:19 +02:00
Jakob Unterwurzacher
5f726aaa9d contentenc: add GCM-SIV support
Also add ReverseDummyNonce nonce generation.
2016-09-25 16:43:17 +02:00
Jakob Unterwurzacher
fca1b82417 fusefrontend: relay Utimens to go-fuse
Commit af5441dcd9 has caused a
regression ( https://github.com/rfjakob/gocryptfs/issues/35 )
that is fixed by this commit.

The go-fuse library by now has all the syscall wrappers in
place to correctly handle Utimens, also for symlinks.

Instead of duplicating the effort here just call into go-fuse.

Closes #35
2016-09-25 16:30:29 +02:00
Jakob Unterwurzacher
af5441dcd9 fusefrontend: use NsecToTimespec() for Utimens
This fixes a build problem on 32-bit hosts:

  internal/fusefrontend/file.go:400: cannot use a.Unix() (type int64) as
  type int32 in assignment
  internal/fusefrontend/file.go:406: cannot use m.Unix() (type int64) as
  type int32 in assignment

It also enables full nanosecond timestamps for dates
after 1970.
2016-08-09 22:18:46 +02:00
Jakob Unterwurzacher
9b725c15cf syscallcompat: OSX: add Fallocate and Openat wrappers
...and convert all calls to syscall.{Fallocate,Openat}
to syscallcompat .

Both syscalls are not available on OSX. We emulate Openat and just
return EOPNOTSUPP for Fallocate.
2016-07-03 19:18:34 +02:00
Jakob Unterwurzacher
c9a472c12f syscallcompat: move syscall wrapper to their own package
We will get more of them as OSX also lacks support for openat.
2016-07-03 17:51:40 +02:00
Jakob Unterwurzacher
54470baa23 fusefrontend: add fallocate support
Mode=0 (default) and mode=1 (keep size) are supported.
The patch includes test cases and the whole thing passed xfstests.

Fixes https://github.com/rfjakob/gocryptfs/issues/1 .
2016-07-02 19:52:09 +02:00
Jakob Unterwurzacher
04ad063515 fusefronted: move Truncate() and Allocate() to their own file
These are large complicated implementations that will share some
code.
2016-07-02 15:35:06 +02:00
Jakob Unterwurzacher
7b22b426b9 contentenc: rename PlaintextRange and CiphertextRange
The name could be misunderstood and actually caused a bug:
doWrite used to always preallocate 4128 instead of the actual
data length.
2016-07-02 00:12:36 +02:00
Jakob Unterwurzacher
f2b4d57068 fusefrontend: coalesce grows in Truncate()
We were growing the file block-by-block which was pretty
inefficient. We now coalesce all the grows into a single
Ftruncate. Also simplifies the code!

Simplistic benchmark: Before:

  $ time truncate -s 1000M foo
  real	0m0.568s

After:

  $ time truncate -s 1000M foo
  real	0m0.205s
2016-07-01 23:32:27 +02:00
Jakob Unterwurzacher
ae77d18527 fusefrontend: better comments for Truncate 2016-07-01 09:23:04 +02:00
Jakob Unterwurzacher
0115588680 main, fusefrontend: enable suid functionality
FUSE filesystems are mounted with "nosuid" by default. If we run as root,
we can use device files by passing the opposite mount option, "suid".

Also we have to use syscall.Chmod instead of os.Chmod because the
portability translation layer "syscallMode" messes up the sgid
and suid bits.

Fixes 70% of the failures in xfstests generic/193. The remaining are
related to truncate, but we err on the safe side:

    $ diff -u tests/generic/193.out /home/jakob/src/fuse-xfstests/results//generic/193.out.bad
    [...]
     check that suid/sgid bits are cleared after successful truncate...
     with no exec perm
     before: -rwSr-Sr--
    -after:  -rw-r-Sr--
    +after:  -rw-r--r--
2016-06-26 20:13:21 +02:00
Jakob Unterwurzacher
38767ab527 fuserfrontend: support truncate(2) by wrapping ftruncate(2)
Support truncate(2) by opening the file and calling ftruncate(2)
While the glibc "truncate" wrapper seems to always use ftruncate, fsstress from
xfstests uses this a lot by calling "truncate64" directly.
2016-06-26 18:41:04 +02:00
Jakob Unterwurzacher
6c3f97399a Rename internal "toggledlog" package to "tlog"
tlog is used heavily everywhere and deserves a shorter name.

Renamed using sed magic, without any manual rework:

   find * -type f -exec sed -i 's/toggledlog/tlog/g' {} +
2016-06-15 23:30:44 +02:00
Jakob Unterwurzacher
393e531afd Fix warnings reported by Go 1.6 "go tool vet -shadow=true"
Warnings were:

  main.go:234: declaration of err shadows declaration at main.go:163:
  internal/fusefrontend/file.go:401: declaration of err shadows declaration at internal/fusefrontend/file.go:379:
  internal/fusefrontend/file.go:419: declaration of err shadows declaration at internal/fusefrontend/file.go:379:
  internal/fusefrontend/fs_dir.go:140: declaration of err shadows declaration at internal/fusefrontend/fs_dir.go:97:
2016-06-14 22:46:23 +02:00
Jakob Unterwurzacher
5b1eed35ee fusefrontend: Utimens: convert ENOENT to EBADF
If /proc/self/fd/X did not exist, the actual error is that the file
descriptor was invalid.

go-fuse's pathfs prefers using an open fd even for path-based operations
but does not take any locks to prevent the fd from being closed.
Instead, it retries the operation by path if it get EBADF. So this
change allows the retry logic to work correctly.

This fixes the error

    rsync: failed to set times on "/tmp/ping.Kgw.mnt/linux-3.0/[...]/.dvb_demux.c.N7YlEM":
    No such file or directory (2)

that was triggered by pingpong-rsync.bash.
2016-06-09 22:21:00 +02:00
Jakob Unterwurzacher
5dd9576a11 fusefrontend: replace unreliable "fd < 0" check
... with the "released" boolean.

For some reason, the "f.fd.Fd() < 0" check did not work reliably,
leading to nil pointer panics on the following wlock.lock().

The problem was discovered during fsstress testing and is unlikely
to happen in normal operations.

With this change, we passed 1700+ fsstress iterations.
2016-05-30 09:36:06 +02:00
Jakob Unterwurzacher
1648c54adb fusefrontend: use sync.Once for one-time warnings
Using a simple boolean was racy (which was harmless
in this case) and non-idomatic.
2016-05-29 22:50:03 +02:00
Jakob Unterwurzacher
fd53dfd2ad fusefronted: check Fstat return value on file create
The Fstat call should never fail, but still, if it does return an error
it should be handled properly.
2016-05-29 22:43:48 +02:00
Jakob Unterwurzacher
4c0cb37c50 fusefrontend: remove unused "forgotten" variable
The functionality has long been replaced by the fd < 0
check.
2016-05-29 13:46:47 +02:00
Jakob Unterwurzacher
ba7c798418 fusefrontend: fix panic due to concurrently unregistered wlock
Commit 730291feab properly freed wlock when the file descriptor is
closed. However, concurrently running Write and Truncates may
still want to lock it. Check if the fd has been closed first.
2016-05-08 23:21:20 +02:00
Jakob Unterwurzacher
730291feab fusefrontend: fix wlock memory leak
The write lock was not freed on release, causing a slow memory leak.

This was noticed by running extractloop.bash for 10 hours.
2016-05-05 13:38:39 +02:00
Jakob Unterwurzacher
bd5405189e Fix "go tool vet -shadow=true" warnings
Among those one real bug.
2016-04-10 21:31:15 +02:00
Jakob Unterwurzacher
c74772bc8d Run go fmt 2016-02-06 20:23:36 +01:00
Jakob Unterwurzacher
9078a77850 Move pathfs_frontend to internal/fusefrontend
"git status" for reference:

renamed:    pathfs_frontend/args.go -> internal/fusefrontend/args.go
renamed:    pathfs_frontend/compat_darwin.go -> internal/fusefrontend/compat_darwin.go
renamed:    pathfs_frontend/compat_linux.go -> internal/fusefrontend/compat_linux.go
renamed:    pathfs_frontend/file.go -> internal/fusefrontend/file.go
renamed:    pathfs_frontend/file_holes.go -> internal/fusefrontend/file_holes.go
renamed:    pathfs_frontend/fs.go -> internal/fusefrontend/fs.go
renamed:    pathfs_frontend/fs_dir.go -> internal/fusefrontend/fs_dir.go
renamed:    pathfs_frontend/names.go -> internal/fusefrontend/names.go
renamed:    pathfs_frontend/write_lock.go -> internal/fusefrontend/write_lock.go
modified:   main.go
2016-02-06 19:27:59 +01:00