Commit Graph

1627 Commits

Author SHA1 Message Date
Jakob Unterwurzacher
518771e4e2 fusefrontend_reverse: use inomap for inode number translation
Gets rid of static inode number value limitations.

Fixes https://github.com/rfjakob/gocryptfs/issues/457
2020-05-03 15:22:10 +02:00
Jakob Unterwurzacher
db93a6c54c tests: reverse: add inode mapping test (TestVirtualFileIno)
Verify that virtual files get assigned inode numbers
we expect.
2020-05-03 14:49:32 +02:00
Jakob Unterwurzacher
483054efaa inomap: comment constants 2020-04-19 22:09:21 +02:00
Jakob Unterwurzacher
9f9d59ded9 inomap: rework logic to efficiently support flags
Adding flags allows to use inomap in reverse mode,
replacing the clunky inoBaseDirIV/inoBaseNameFile
logic that causes problems with high underlying
inode numbers ( https://github.com/rfjakob/gocryptfs/issues/457 )

Microbenchmarks (values below) show that the "SingleDev"
case is now much slower due to an extra map lookup,
but this has no visible effects in ./test.bash results,
so there was no time spent optimizing the case further.

$ go test -bench=.
goos: linux
goarch: amd64
pkg: github.com/rfjakob/gocryptfs/internal/inomap
BenchmarkTranslateSingleDev-4   	18757510	        61.5 ns/op
BenchmarkTranslateManyDevs-4    	18061515	        64.5 ns/op
PASS
ok  	github.com/rfjakob/gocryptfs/internal/inomap	2.467s
2020-04-19 22:00:56 +02:00
Jakob Unterwurzacher
fcdeb52390 inomap: add benchmark
$ go test -bench=.
goos: linux
goarch: amd64
pkg: github.com/rfjakob/gocryptfs/internal/inomap
BenchmarkTranslateSingleDev-4   	202479382	         5.88 ns/op
BenchmarkTranslateManyDevs-4    	16095795	        71.9 ns/op
PASS
ok  	github.com/rfjakob/gocryptfs/internal/inomap	3.039s
2020-04-19 21:35:06 +02:00
Jakob Unterwurzacher
1c169ac55e build.bash: handle missing git tags
The case of a git repo without any tags used to fail
with:

  fatal: No names found, cannot describe anything.

Now we continue, using "[no_tags_found]" as the
version string.
2020-04-18 17:44:51 +02:00
Jakob Unterwurzacher
5da5e9fdf2 build.bash: don't enable -buildmode=pie for static builds
Causes warnings:

  $ ./build-without-openssl.bash
  # github.com/rfjakob/gocryptfs
  loadinternal: cannot find runtime/cgo
  # github.com/rfjakob/gocryptfs/gocryptfs-xray
  loadinternal: cannot find runtime/cgo
  # github.com/rfjakob/gocryptfs/contrib/statfs
  loadinternal: cannot find runtime/cgo
  gocryptfs v1.7.1-48-gf6b1c68 without_openssl; go-fuse v1.0.1-0.20190319092520-161a16484456; 2020-04-18 go1.13.6 linux/amd64

https://github.com/golang/go/issues/30986
2020-04-18 17:09:25 +02:00
Jakob Unterwurzacher
f6b1c680b3 nametransform: update comment & simplify tests
The comment still mentioned CBC, which has been removed
a long time ago.

The test definition can be rewritten using slice literals,
saving sume stuttering.
2020-04-18 16:14:48 +02:00
Jakob Unterwurzacher
8f5c2a613d Prefer Go stdlib aes-gcm on arm64 with aes instructions
We used to prefer openssl in this situation, which
used to make sense, but now Go gained an optimized
assembly implementation for aes-gcm on arm64 with
aes instructions:

  root@q1:~/go/src/github.com/rfjakob/gocryptfs# ./gocryptfs -speed
  gocryptfs v1.7.1-46-g73436d9; go-fuse v1.0.1-0.20190319092520-161a16484456; 2020-04-13 go1.14.2 linux/arm64
  AES-GCM-256-OpenSSL      212.30 MB/s    (selected in auto mode)
  AES-GCM-256-Go           452.30 MB/s
  AES-SIV-512-Go           100.25 MB/s
  XChaCha20-Poly1305-Go    137.35 MB/s

https://github.com/rfjakob/gocryptfs/issues/452
2020-04-13 22:34:07 +02:00
Jakob Unterwurzacher
73436d9419 travis: fix Go Modules build problems 2020-04-13 18:06:50 +02:00
Jakob Unterwurzacher
a7d562d114 build.bash: use GOFLAGS -trimpath and enable PIE
GOFLAGS exists since Go 1.11: https://golang.org/doc/go1.11

https://github.com/rfjakob/gocryptfs/pull/460
2020-04-13 16:03:51 +02:00
Jakob Unterwurzacher
e509b27a5c Convert build and packaging scripts to Go Modules 2020-04-13 14:54:04 +02:00
Jakob Unterwurzacher
cad711993d dep: migrate to Go Modules
Following https://blog.golang.org/migrating-to-go-modules
2020-04-13 14:54:04 +02:00
Jakob Unterwurzacher
9a7ceef09e shellcheck: make top-level bash scripts warning-free
And run shellcheck in test.bash.
2020-04-13 14:54:04 +02:00
Jakob Unterwurzacher
a2ad14b9ac build.bash: append branch name != master 2020-04-13 14:54:04 +02:00
Jakob Unterwurzacher
210db84e20 speed: show gocryptfs version
Output now looks like this

  $ gocryptfs -speed
  gocryptfs v1.7.1-38-gbe3b9df-dirty; go-fuse v2.0.2-57-gd1cfa17; 2020-04-13 go1.13.6 linux/amd64
  AES-GCM-256-OpenSSL 	 607.90 MB/s
  AES-GCM-256-Go      	 920.75 MB/s	(selected in auto mode)
  AES-SIV-512-Go      	 169.85 MB/s
  XChaCha20-Poly1305-Go	 794.30 MB/s

and has go version and arch information, which is important
when comparing results.
2020-04-13 14:54:04 +02:00
Jakob Unterwurzacher
488111ce39 inomap: split into separate package
inomap will also be used by fusefrontend_reverse
in the future. Split if off openfiletable to make
it independent.
2020-04-13 14:54:04 +02:00
Jakob Unterwurzacher
194030f18a speed: add XChaCha20-Poly1305-Go
https://github.com/rfjakob/gocryptfs/issues/452
2020-04-13 14:54:04 +02:00
Oscar
75f16771ff Update manpage examples 2020-04-07 22:04:40 +02:00
Jakob Unterwurzacher
2568518992 crossbuild.bash: set GOARM=7
From https://github.com/golang/go/wiki/GoArm :

  In cross compilation situations, it is recommended
  that you always set an appropriate GOARM value
  along with GOARCH.

The value seems to default to GOARM=5 if not set
during cross-compilation.
2020-02-29 21:52:59 +01:00
Jakob Unterwurzacher
f82b9caa9c speed: add code comments 2020-02-29 21:26:28 +01:00
Jakob Unterwurzacher
fdfaa849f8 tests: test xattr acls
Fixes https://github.com/rfjakob/gocryptfs/issues/453
2020-02-29 20:38:48 +01:00
Jakob Unterwurzacher
ca9e912a28 fusefrontend: drop xattr user namespace restriction
We used to restrict setting xattrs to the "user."
namespace. I don't see a real reason for this
anymore, and it causes trouble for users who are using
acls.

Tests will be added in the next commit.

https://github.com/rfjakob/gocryptfs/issues/453
2020-02-29 20:12:43 +01:00
orcas
9ec042f2f6 Show undecryptable filenames if they match supplied glob
Resolves https://github.com/rfjakob/gocryptfs/issues/393
2020-02-28 22:17:59 +01:00
Jakob Unterwurzacher
eeefddad6a README: update changelog 2020-02-15 22:02:27 +01:00
Jakob Unterwurzacher
6eca07e36e tests: randomize data in testWriteN
Just writing zeros carries the risk of not detecting
wrongly created file holes. Write random data instead.
2020-02-15 21:56:08 +01:00
Jakob Unterwurzacher
97743858ce performance.txt: update result for governor = performance
Also add big signs where the cpu has changed, as the results
are not comparable when the cpu is different.

Also update encfs results.
2020-02-15 21:42:39 +01:00
Jakob Unterwurzacher
abc59fa968 contentenc: encryptBlocksParallel: explain why last part runs in new goroutine
The result is counter-intuitive, so explain it here.
2020-02-15 21:28:12 +01:00
Jakob Unterwurzacher
ff210a06fb main: respect GOMAXPROCS environment variable
If the user sets GOMAXPROCS explicitely, we should
respect it.
2020-02-15 17:44:40 +01:00
Jakob Unterwurzacher
d5ce340c02 merge prefer_openssl package into stupidgcm
Now that I have discovered golang.org/x/sys/cpu and that Go
versions below 1.6 are uncommon, there was not much useful
code left in prefer_openssl.

Merge the remains into stupidgcm.
2020-02-15 17:21:30 +01:00
Jakob Unterwurzacher
9aeb2a3df6 contentenc: move parallel encryption into encryptBlocksParallel
Make the logic self-contained in the new helper function.
2020-02-15 15:49:12 +01:00
Jakob Unterwurzacher
a92db18fe7 README: Raw64 change should be bold
This was a new feature flag.
2020-02-15 14:40:03 +01:00
Jakob Unterwurzacher
747e994682 Add contrib/cleanup-tmp-mounts.sh
Useful when you have lots of broken mounts after something in
the test suite went wrong.
2020-02-15 14:38:13 +01:00
Jakob Unterwurzacher
dc21cd3572 tests: cli: add TestSymlinkedCipherdir
https://github.com/rfjakob/gocryptfs/issues/450
2020-01-25 16:54:57 +01:00
Jakob Unterwurzacher
a48d7fac2c main: haveFusermount2: respect PATH environment variable
Use exec.LookPath() to find fusermount in the user's PATH
first. Fall back to /bin/fusermount for the case that PATH
is not set, like go-fuse does.

Fixes https://github.com/rfjakob/gocryptfs/issues/448
2020-01-14 23:12:56 +01:00
Jakob Unterwurzacher
367b7e8647 Travis CI: upgrade from Ubuntu 16.04 "Xenial" to Ubuntu 18.04 "Bionic"
Newer kernel and (acc. to Travis documentation) faster boot times.
2019-12-29 14:21:48 +01:00
Jakob Unterwurzacher
7116838abb Travis CI: clean up Go versions 2019-12-29 14:16:06 +01:00
Jakob Unterwurzacher
db43bfc4c0 main: detect fusermount3 and don't add nonempty
fusermount3 (i.e. fusermount from libfuse 3.x) has dropped
the `nonempty` option.

Detect fusermount3 and don't add `nonempty` in this case.

Fixes https://github.com/rfjakob/gocryptfs/pull/440
2019-12-28 23:51:45 +01:00
Pavol Rusnak
1364b44ae3 remove Trezor support 2019-12-28 19:50:49 +01:00
Jakob Unterwurzacher
7dda2363e1 Fix "go get" failure
Move the statusTxtContent to fix this confusing error
when running `go get github.com/rfjakob/gocryptfs/...`:

  $ go get github.com/rfjakob/gocryptfs/...
  # github.com/rfjakob/gocryptfs/tests/example_filesystems
  tests/example_filesystems/example_test_helpers.go:22:16: undefined: statusTxtContent
  tests/example_filesystems/example_test_helpers.go:75:16: undefined: statusTxtContent
2019-12-24 00:38:40 +01:00
Jakob Unterwurzacher
9178aa1534 MANPAGE: describe how to unmount, and that default options are fine
Feedback received during the recent Go user group. If you haven't
used FUSE before, you don't know how to unmount, and it was not
described in the man page!

As for the options, there are many, and new users are intimidated
by it. State clearly that defaults are fine.
2019-11-17 11:04:41 +01:00
Jakob Unterwurzacher
8ec804d825 fs: add uint64 cast to fix Darwin build
Error from Travis CI was:

 +GOOS=darwin
 +GOARCH=amd64
 +go build -tags without_openssl
 # github.com/rfjakob/gocryptfs/internal/fusefrontend
 internal/fusefrontend/fs.go:88:45: cannot use st.Dev (type int32) as type uint64 in argument to openfiletable.NewInumMap

Add uint64 to fix it.
2019-11-16 23:21:37 +01:00
Jakob Unterwurzacher
bb6155a51f fusefrontend: use inummap
translate inode numbers on different devices to fix
collisions.

Fixes https://github.com/rfjakob/gocryptfs/issues/435
2019-11-16 21:36:27 +01:00
Jakob Unterwurzacher
e5d5ab3973 openfiletable: add inummap
Generates unique inode numbers for files on different
devices.

https://github.com/rfjakob/gocryptfs/issues/435
2019-11-16 21:35:26 +01:00
Jakob Unterwurzacher
31ff557f85 tests: also check inode number in VerifySize
Check Stat() vs Fstat() result. Not very useful
at the moment as the kernel never seems to call Fstat().
2019-11-16 21:25:14 +01:00
Jakob Unterwurzacher
7e9e63b430 contrib/statvsfstat: add statvsfstat tool
Cli tool to compare the result of Stat() and Fstat().
2019-11-16 21:19:08 +01:00
Jakob Unterwurzacher
f7ddae24ed tests: fsstress: clean up leftover mounts on startup
Broken mounts may accumulate when the fs crashes.
2019-11-10 22:06:40 +01:00
Jakob Unterwurzacher
d404dcd3e3 tests: drop "-l" flag from fsstress-loopback.bash
No longer supported by go-fuse loopback.
2019-11-10 20:59:07 +01:00
Jakob Unterwurzacher
e4148028af fusefrontend: warn about missing diriv
The comment is outdated, at this point, we should
really not get any errors from ReadDirIVAt.

The change is best seen when running the fsck tests. Before:

  fsck: error opening dir "missing_diriv": 2=no such file or directory

After:

  OpenDir "K2m0E6qzIfoLkVZJanoUiQ": could not read gocryptfs.diriv: no such file or directory
  fsck: error opening dir "missing_diriv": 5=input/output error

See https://github.com/rfjakob/gocryptfs/issues/403 , where
the extra info would have been helpful.
2019-11-03 20:38:15 +01:00
Jakob Unterwurzacher
3b67bc5711 main: show "-fsck" in help text
Suggested at https://github.com/rfjakob/gocryptfs/issues/403
2019-11-03 20:19:22 +01:00