Commit Graph

1701 Commits

Author SHA1 Message Date
Jakob Unterwurzacher b1d631d432 v2api: fix double-lock in truncate 2020-07-12 13:13:24 +02:00
Jakob Unterwurzacher 6196a5b5fe v2api: File2: implement Release, Read, Write, Fsync, Flush, Allocate
Fortunately, this just means fixing up the function
signatures.
2020-07-12 12:59:01 +02:00
Jakob Unterwurzacher d539a4c214 v2api: fix RootNode cast 2020-07-11 20:27:47 +02:00
Jakob Unterwurzacher b0342fae5d v2api: fix TestOpenBackingDir 2020-07-11 20:15:47 +02:00
Jakob Unterwurzacher 7de3330d70 v2api: implement Rename 2020-07-11 19:56:45 +02:00
Jakob Unterwurzacher 84344834c4 v2api: remove OpenatUserCtx, MknodatUserCtx helpers
Instead, use the new toFuseCtx() function introduced
in an earlier commit.
2020-07-11 19:44:45 +02:00
Jakob Unterwurzacher 250dbc6436 v2api: implement Symlink 2020-07-11 19:43:07 +02:00
Jakob Unterwurzacher c35b575d5f v2api: implement Link 2020-07-11 19:32:38 +02:00
Jakob Unterwurzacher b971c75e67 v2api: implement Mknod 2020-07-11 19:23:04 +02:00
Jakob Unterwurzacher 855b4a95b7 v2api: implement Statfs 2020-07-11 18:59:54 +02:00
Jakob Unterwurzacher a39fc8ab2e v2api: fix crash on umount
fs.Mount already called srv.Serve() for us.
2020-07-08 22:21:27 +02:00
Jakob Unterwurzacher 40a31e0c32 v2api: list interfaces to be implemented 2020-07-05 20:22:24 +02:00
Jakob Unterwurzacher 63f68a0fcd v2api: implement Setattr 2020-07-05 20:10:53 +02:00
Jakob Unterwurzacher c22e78ee41 v2api: implement Opendir 2020-07-05 19:34:30 +02:00
Jakob Unterwurzacher 1f4e554168 v2api: merge openBackingDir into root_node.go 2020-07-04 21:42:04 +02:00
Jakob Unterwurzacher d2139e18ef v2api: implement Open() 2020-07-04 21:37:44 +02:00
Jakob Unterwurzacher d73e4b3f7c v2api: add prepareAtSyscall helper 2020-07-04 21:16:20 +02:00
Jakob Unterwurzacher 23180794fe v2api: implement Readlink 2020-07-04 20:32:02 +02:00
Jakob Unterwurzacher 1618fbbac5 v2api: implement Unlink 2020-06-21 14:08:53 +02:00
Jakob Unterwurzacher cc0b94a3c5 v2api: implement Rmdir 2020-06-21 13:57:04 +02:00
Jakob Unterwurzacher 192a29075a v2api: implement Mkdir 2020-06-21 13:46:08 +02:00
Jakob Unterwurzacher f6ded09e36 v2api: implement Create 2020-06-21 13:25:12 +02:00
Jakob Unterwurzacher 74a4accf0c v2api: collect RootNode code in root_node.go 2020-06-21 12:44:32 +02:00
Jakob Unterwurzacher ebdf58b9eb v2api: implement GetAttr and Readdir 2020-06-21 12:42:18 +02:00
Jakob Unterwurzacher 77632b7554 getdents_c: read from two threads
This was an attempt to make the C code more
similar to Go (which also reads from multiple threads).

However, I still could not repro the ENOENT problems.

https://github.com/rfjakob/gocryptfs/issues/483
2020-06-21 12:01:34 +02:00
Jakob Unterwurzacher 6aa9f5636f v2api: implement Lookup()
Compiles, but untested otherwise. No caching.
2020-06-21 12:01:34 +02:00
Jakob Unterwurzacher 9b8ce55383 inomap: clarify TranslateStat function comment
Also, fix the namespaceMap comment.
2020-06-21 12:01:34 +02:00
Jakob Unterwurzacher 2aad58f9ec v2api (go-fuse v2 api): initial noop implementation
Compiles and mounts but does nothing useful.
2020-06-21 12:01:24 +02:00
Jakob Unterwurzacher 3b61244b72 tests: TestBadname: simplify test by using empty files
Simplify the tests by using empty files. Empty
files are valid, and we don't check the content
anyway.

Also adjust comment style a little and add
a missing break statement.
2020-06-06 12:53:45 +02:00
Jakob Unterwurzacher 261cf12829 badname: stop trying to decrypt at length 22
At least 16 bytes due to AES --> at least 22 characters in base64
2020-06-06 12:49:45 +02:00
DerDonut a8230d271f Added auto decryption of invalid file names
Changed invalid file decoding and decryption. Function
DecryptName now shortens the filename until the filename is
decodable and decryptable. Will work with valid **and**
invalid Base64URL delimiter (valid delimiter [0-9a-zA-z_\\-].
If the filename is not decryptable at all, it returns the
original cipher name with flag suffix Changed cli tests to
generate decryptable and undecryptable file names with correct
encrypted content. Replacing #474, extends #393
2020-06-06 12:20:31 +02:00
Jakob Unterwurzacher a56e7cc5ac sshfs-benchmark.bash: fix locale trouble and move to tests
Locale trouble was

   sshfs-benchmark.bash: line 31: printf: 4.71: invalid number

because printf expected "4,71" in the German locale.
Force the C locale.
2020-06-06 12:15:41 +02:00
Jeremy Hilliker 274e0d27b7 contrib/getdents-debug: fix function call missing argument from 22e3eec153 2020-06-01 10:18:18 +02:00
Jakob Unterwurzacher 22e3eec153 getdents-debug: loop and stop on first error
Also try to improve and unify output a little.

$ ./getdents /usr/share/man/man1
  1: unix.Getdents: n=9984; n=9984; n=9968; n=9976; n=9984; n=9968; n=10000; n=9976; n=9992; n=10000; n=9976; n=9992; n=2312; n=0; err=<nil>; total 122112 bytes
  2: unix.Getdents: n=9984; n=48; n=9976; n=9968; n=9976; n=9976; n=9992; n=9984; n=9992; n=10000; n=9976; n=9968; n=10000; n=2272; n=0; err=<nil>; total 122112 bytes
  3: unix.Getdents: n=9984; n=9984; n=9968; n=704; n=10000; n=10000; n=9968; n=9968; n=9992; n=10000; n=9960; n=9992; n=9992; n=1600; n=0; err=<nil>; total 122112 bytes
  4: unix.Getdents: n=9984; n=9984; n=9968; n=9976; n=9984; n=32; n=9992; n=9984; n=9992; n=10000; n=9976; n=9968; n=10000; n=2272; n=0; err=<nil>; total 122112 bytes

$ ./getdents_c /usr/share/man/man1
  1: getdents64: n=9984; n=9984; n=9968; n=9976; n=9984; n=9968; n=10000; n=9976; n=9992; n=10000; n=9976; n=9992; n=2312; n=0; errno=0 total 122112 bytes
  2: getdents64: n=9984; n=9984; n=9968; n=9976; n=9984; n=9968; n=10000; n=9976; n=9992; n=10000; n=9976; n=9992; n=2312; n=0; errno=0 total 122112 bytes
  3: getdents64: n=9984; n=9984; n=9968; n=9976; n=9984; n=9968; n=10000; n=9976; n=9992; n=10000; n=9976; n=9992; n=2312; n=0; errno=0 total 122112 bytes
  4: getdents64: n=9984; n=9984; n=9968; n=9976; n=9984; n=9968; n=10000; n=9976; n=9992; n=10000; n=9976; n=9992; n=2312; n=0; errno=0 total 122112 bytes
2020-05-28 23:23:53 +02:00
Jakob Unterwurzacher b275c53fa7 contrib/getdents-debug: implement getdents -loop
$ ./getdents -loop /mnt/synology/public/tmp/g1
unix.Getdents: n=4176; n=4176; n=4176; n=4176; n=4176; n=3192; n=0; err=<nil>; total 24072 bytes
unix.Getdents: n=4176; n=4176; n=4176; n=4176; n=4176; n=3192; n=0; err=<nil>; total 24072 bytes
unix.Getdents: n=4176; n=-1; err=no such file or directory; total 4176 bytes
2020-05-25 23:32:11 +02:00
Jakob Unterwurzacher 0d522e0d3b Add contrib/getdents-debug/readdirnames
Another way to repro the problem in
https://github.com/rfjakob/gocryptfs/issues/483
2020-05-24 23:51:38 +02:00
Jakob Unterwurzacher 6019598fdb contrib: collect getdents stuff in getdents-debug folder 2020-05-24 23:46:41 +02:00
Jakob Unterwurzacher 71c0481f0e Revert "fusefrontend: don't always clear the dircache in Rename"
As noticed by @slackner in
cb8872577d (commitcomment-39405233) ,
this is not safe.

This reverts commit cb8872577d.
2020-05-24 23:36:11 +02:00
Jakob Unterwurzacher 15ff79bf14 syscallcompat: warn when Getdents truncates data
On CIFS mounts, unix.Getdents can return sudden ENOENT
in the middle of data. This will not be reported as an error
by user space tools, so return EIO instead.

Also log it as a warning.

https://github.com/rfjakob/gocryptfs/issues/483
2020-05-24 23:30:25 +02:00
Jakob Unterwurzacher b3350f0ebb contrib: add getdents_c
Same thing like contrib/getdents, but written in C.
2020-05-24 23:29:59 +02:00
Jakob Unterwurzacher c7a9425e1b Add contrib/getdents
Small tool to try to debug unix.Getdents problems on CIFS mounts
https://github.com/rfjakob/gocryptfs/issues/483
2020-05-24 22:54:58 +02:00
Jakob Unterwurzacher 1a91a11e00 contrib/sshfs-benchmark.bash: add rmdir, mkdir, touch
And also, stop using the wrong directory for sshfs git init.

sshfs-benchmark.bash:    sshfs  gocryptfs-on-sshfs
git init                  4.35                7.82
rsync                     7.72               11.66
rm -R                     2.71               11.04
mkdir                     1.33                4.15
rmdir                     0.47                3.97
touch                     2.32                2.85
rm                        0.45                0.45
2020-05-24 22:03:40 +02:00
Jakob Unterwurzacher cb8872577d fusefrontend: don't always clear the dircache in Rename
When filename encryption is on, we do know when we
overwrite a directory, and can clear only in this case.

sshfs-benchmark.bash:    sshfs  gocryptfs-on-sshfs
git init                  1.74                7.80
rsync                     6.19               11.63
2020-05-24 20:19:27 +02:00
Jakob Unterwurzacher 2a9b99a0ef fusefrontend: don't clear dircache on Mkdir
Mkdir can not cause existing entries in the cache to go
stale. So don't clear it. Benchmark results:

sshfs-benchmark.bash:    sshfs  gocryptfs-on-sshfs
git init                  1.65                8.74
rsync                     6.09               17.54
2020-05-24 15:30:14 +02:00
Jakob Unterwurzacher 11dfcfd6c0 contrib: add sshfs-benchmark.bash
Let's get some reproducible numbers for
https://github.com/rfjakob/gocryptfs/issues/481
and
https://github.com/rfjakob/gocryptfs/issues/410

Example run:

$ ./sshfs-benchmark.bash nuetzlich.net
working directory: /tmp/sshfs-benchmark.bash.vu4
sshfs mounted: nuetzlich.net:/tmp -> sshfs.mnt
gocryptfs mounted: sshfs.mnt/sshfs-benchmark.bash.KM9/gocryptfs.crypt -> gocryptfs.mnt

                         sshfs  gocryptfs-on-sshfs
git init                  1.68               11.23
rsync                     6.07               20.35
2020-05-24 15:30:14 +02:00
Jakob Unterwurzacher 25f1727de9 syscallcompat: getdents: retry on EINTR
Fixes: https://github.com/rfjakob/gocryptfs/issues/483
Related: https://github.com/golang/go/issues/38836
2020-05-23 22:54:23 +02:00
Jakob Unterwurzacher f8ad2ac3e2 dircache: increase cache size & lifetime
Looking at the dircache debug output, we see
that a "git status" workload has a very bad
cache hit rate because the entries expire or
get evicted before they can be reused.

Increase both cache size and lifetime for
a 4x speedup:

Before: 75s
After:  17s

https://github.com/rfjakob/gocryptfs/issues/410
2020-05-17 21:37:36 +02:00
Jakob Unterwurzacher f6088e5008 dircache: improve debug messages
Before:

Lookup "errno.html/1/2/3/4/5": miss
Store: "errno.html/1/2/3/4/5" fd=26 iv=21be6e083d60dcabfe7368264d5082b7
Lookup "errno.html": hit 25 6d68a16d217978915036a3bd55428ae7
Lookup "errno.html/1": hit 25 932a464c299b3430c5e55c924f98ac4d
Lookup "errno.html/1/2": hit 25 7d53348b1692d537f017bf86b3cf5feb
Lookup "errno.html/1/2/3": hit 25 2aef1c9d1ab2b55b163215053fefe703
Lookup "errno.html/1/2/3/4": hit 25 cb802be53721c46a46247c5e4e0f4ce6
Lookup "errno.html/1/2/3/4": hit 25 cb802be53721c46a46247c5e4e0f4ce6
Lookup "errno.html": hit 25 6d68a16d217978915036a3bd55428ae7

After:

Lookup "earlyoom/.git/refs"                     hit fd=10 dup=17 iv=6ae2cecd269a25e8d946aff6afe9b8b8
Lookup "earlyoom/.git/refs/remotes"             hit fd=19 dup=17 iv=f04c2d2a5bcc33ebdeaca664859c980d
Lookup "earlyoom/.git/refs/remotes/origin"      miss
Store  "earlyoom/.git/refs/remotes/origin"      fd=17 iv=834a64a1697c9f5705455ba6dbed22b5
Lookup "earlyoom"                               hit fd=7 dup=25 iv=2303a892d6e2357c483574a8070b7679
Lookup "earlyoom/.git"                          hit fd=11 dup=25 iv=d43ca4aff23720c57789c9f62f0aee00
Lookup "earlyoom/.git"                          hit fd=11 dup=25 iv=d43ca4aff23720c57789c9f62f0aee00
Lookup "earlyoom/.git/refs"                     hit fd=10 dup=25 iv=6ae2cecd269a25e8d946aff6afe9b8b8
Lookup "earlyoom/.git/refs/heads"               hit fd=13 dup=25 iv=f9245e7c066b9adc768a1a666da9fbc8
2020-05-17 21:26:56 +02:00
Jakob Unterwurzacher bf66da6880 Update changelog 2020-05-17 19:40:30 +02:00
Jakob Unterwurzacher 416080203b main: accept multiple -passfile options
Each file will be read and then concatenated
for the effictive password. This can be used as a
kind of multi-factor authenticiton.

Fixes https://github.com/rfjakob/gocryptfs/issues/288
2020-05-17 19:31:04 +02:00