Commit Graph

2064 Commits (libgocryptfs)

Author SHA1 Message Date
Hardcore Sushi 39af24d4e6
libgocryptfs: update gocryptfs 2021-06-27 19:07:50 +02:00
Jakob Unterwurzacher db81614cd6 canonical-benchmarks.bash: handle relative paths
Passing a relative path was broken because we cd'ed
away first.
2021-06-27 11:48:41 +02:00
Jakob Unterwurzacher 2a9d70d48f fido2: drop `-v` option (PIN request)
We used to pass `-v` on `gocryptfs -init` but not for
mount, which seems strange by itself, but more importantly,
`-v` does not work on Yubikeys.

Drop `-v`.

2021-06-27 11:17:29 +02:00
Jakob Unterwurzacher d6c8d892ff fido2: pretty-print fidoCommand in debug output
2021-06-27 11:12:40 +02:00
Jakob Unterwurzacher fe616ddad5 doc: update performance.txt 2021-06-26 20:57:39 +02:00
Jakob Unterwurzacher 49507ea869 tests/fsck: delete obsolete script run_fsck.bash
Not called by anybody.
2021-06-26 19:27:58 +02:00
Jakob Unterwurzacher ad3eeaedc5 tests, maxlen.bash: speed up TestMaxlen using QUICK=1
From >6 to <1 second.
2021-06-26 19:13:24 +02:00
Jakob Unterwurzacher 446c3d7e93 tests: matrix: show content detail on mismatch 2021-06-26 18:58:29 +02:00
Jakob Unterwurzacher 4fd95b718b fusefrontend: delete openBackingDir 2021-06-26 18:49:54 +02:00
Jakob Unterwurzacher 5306fc345b fusefrontend: convert last callers from openBackingDir to prepareAtSyscall 2021-06-26 18:49:54 +02:00
Jakob Unterwurzacher 1f29542b39 tests: better error message on ctlsock query failure 2021-06-26 18:49:54 +02:00
Jakob Unterwurzacher 45648e567a fusefrontend: ctlsock: get rid of unneccessary wrapper function 2021-06-26 18:49:54 +02:00
Jakob Unterwurzacher f9f4bd214f fusefrontend: convert ctlsock from openBackingDir to prepareAtSyscall
openBackingDir will be removed.

Also, remove leftover debug printfs.
2021-06-26 18:49:54 +02:00
Jakob Unterwurzacher ee59b5269b fusefrontend: convert openBackingDir tests to prepareAtSyscall
openBackingDir will be removed.
2021-06-26 16:28:30 +02:00
Jakob Unterwurzacher cbd5e8ba01 tests/default: add maxlen.bash test 2021-06-26 16:09:04 +02:00
Jakob Unterwurzacher 389aba6a6b maxlen.bash: suppress progress output if not on a terminal 2021-06-26 16:08:29 +02:00
Jakob Unterwurzacher 84e702126a fusefrontend: implement recursive diriv caching
The new contrib/maxlen.bash showed that we have exponential
runtime with respect to directory depth.

The new recursive diriv caching is a lot smarter as it caches
intermediate lookups. maxlen.bash now completes in a few seconds.

xfstests results same as
2d158e4c82/screenlog.0 :

  Failures: generic/035 generic/062 generic/080 generic/093 generic/099 generic/215 generic/285 generic/319 generic/426 generic/444 generic/467 generic/477 generic/523
  Failed 13 of 580 tests

benchmark.bash results are identical:

  $ ./benchmark.bash
  Testing gocryptfs at /tmp/benchmark.bash.BdQ: gocryptfs v2.0.1-17-g6b09bc0; go-fuse v2.1.1-0.20210611132105-24a1dfe6b4f8; 2021-06-25 go1.16.5 linux/amd64
  /tmp/benchmark.bash.BdQ.mnt is a mountpoint
  WRITE: 262144000 bytes (262 MB, 250 MiB) copied, 0,4821 s, 544 MB/s
  READ:  262144000 bytes (262 MB, 250 MiB) copied, 0,266061 s, 985 MB/s
  UNTAR: 8,280
  MD5:   4,564
  LS:    1,745
  RM:    2,244
2021-06-25 13:56:53 +02:00
Jakob Unterwurzacher 05b813f202 nametransform: rename BadNameFlag to BadnameSuffix 2021-06-21 12:12:44 +02:00
Jakob Unterwurzacher 689b74835b nametransform: gather badname functions in badname.go 2021-06-21 12:10:04 +02:00
Jakob Unterwurzacher 2efef1e270 nametransform: delete NameTransformer interface
Useless layer of indirection.
2021-06-21 11:53:33 +02:00
Jakob Unterwurzacher e244b51491 tests: cli: add TestZerokey
TestZerokey verifies that `gocryptfs -zerokey` uses the same options as
`gocryptfs -init`.
2021-06-21 11:48:16 +02:00
Jakob Unterwurzacher 6b0e63c1a8 Improve startup debug output
The startup debug output was very verbose but still missing some
effective crypto settings.
2021-06-21 11:32:04 +02:00
Jakob Unterwurzacher c5d8fa83ae nametransform: pass badname patterns via New
This means we can unexport the field.
2021-06-20 19:09:46 +02:00
Jakob Unterwurzacher 203e65066f main: use JSONDump helper for debug output 2021-06-20 18:25:07 +02:00
Jakob Unterwurzacher 50630e9f3d fido2: hide "FIDO2" in gocryptfs.conf if not used
Result of:

$ gocryptfs -init foo
$ cat foo/gocryptfs.conf


	"Creator": "gocryptfs v2.0.1",
	"EncryptedKey": "FodEdNHD/cCwv1n5BuyAkbIOnJ/O5gfdCh3YssUCJ2DUr0A8DrQ5NH2SLhREeWRL3V8EMiPO2Ncr5IVwE4SSxQ==",
	"ScryptObject": {
		"Salt": "brGaw9Jg1kbPuSXFiwoxqK2oXFTgbniSgpiB+cu+67Y=",
		"N": 65536,
		"R": 8,
		"P": 1,
		"KeyLen": 32
	"Version": 2,
	"FeatureFlags": [
	"FIDO2": {
		"CredentialID": null,
		"HMACSalt": null


	"Creator": "gocryptfs v2.0.1-5-gf9718eb-dirty.DerDonut-badnamecontent",
	"EncryptedKey": "oFMj1lS1ZsM/vEfanNMeCTPw3PZr5VWeL7ap8Jd8YQm6evy2BAhtQ/pd6RzDx84wlCz9TpxqHRihuwSEMnOWWg==",
	"ScryptObject": {
		"Salt": "JZ/5mhy4a8EAQ/wDF1POIEe4/Ss38cfJgXgj26DuA4M=",
		"N": 65536,
		"R": 8,
		"P": 1,
		"KeyLen": 32
	"Version": 2,
	"FeatureFlags": [
2021-06-20 18:09:21 +02:00
DerDonut a611810ff4 Badname file content access
This proposal is the counterpart of the modifications from the `-badname`
parameter. It modifies the plain -> cipher mapping for filenames when using
`-badname` parameter. The new function `EncryptAndHashBadName` tries to find a
cipher filename for the given plain name with the following steps:

1. If `badname` is disabled or direct mapping is successful: Map directly
(default and current behaviour)

2. If a file with badname flag has a valid cipher file, this is returned
(=File just ends with the badname flag)

3. If a file with a badname flag exists where only the badname flag was added,
this is returned (=File cipher name could not be decrypted by function
`DecryptName` and just the badname flag was added)

4. Search for all files which cipher file name extists when cropping more and
more characters from the end. If only 1 file is found, return this

5. Return an error otherwise

This allows file access in the file browsers but most important it allows that
you rename files with undecryptable cipher names in the plain directories.
Renaming those files will then generate a proper cipher filename One
backdraft: When mounting the cipher dir with -badname parameter, you can never
create (or rename to) files whose file name ends with the badname file flag
(at the moment this is " GOCRYPTFS_BAD_NAME"). This will cause an error.

I modified the CLI test function to cover additional test cases. Test [Case
cannot be performed since the cli tests are executed in panic mode. The
testing is stopped on error. Since the function`DecryptName` produces internal
errors when hitting non-decryptable file names, this test was omitted.

This implementation is a proposal where I tried to change the minimum amount
of existing code. Another possibility would be instead of creating the new
function `EncryptAndHashBadName` to modify the signature of the existing
function `EncryptAndHashName(name string, iv []byte)` to
`EncryptAndHashName(name string, iv []byte, dirfd int)` and integrate the
functionality into this function directly. You may allow calling with dirfd=-1
or other invalid values an then performing the current functionality.
2021-06-20 18:09:21 +02:00
Jakob Unterwurzacher cdddd1d711 MANPAGE: describe -badname 2021-06-20 18:09:21 +02:00
Jakob Unterwurzacher 9ac77d42d1 contrib/maxlen.bash: also test dir and path length
Move the script from tests to contrib as it may now
be useful to somebody else.
2021-06-20 18:09:21 +02:00
Marcel Bochtler 2e738efe86 README: Add MacPorts install instructions
See [1] for the Portfile.

2021-06-19 20:22:20 +02:00
Marcel Bochtler 7889346947 README: Rename Mac OS X to its latest name
2021-06-19 20:22:20 +02:00
Hardcore Sushi 847d4fa781
Genesis patch 2021-06-11 15:52:46 +02:00
Jakob Unterwurzacher 98ab59c96e go.mod: update go-fuse
Memory compaction was merged
( 24a1dfe6b4 )

2021-06-11 15:24:39 +02:00
Jakob Unterwurzacher 9046d6d922 README: sync compile instructions with gocryptfs-website 2021-06-08 10:07:49 +02:00
Jakob Unterwurzacher 3dad9e0648 README: recommend build-without-openssl.bash
For build.bash, list missing dependencies: gcc, pkg-config

2021-06-08 09:55:04 +02:00
Jakob Unterwurzacher 14d422dc9c README: update for v2.0.1 release 2021-06-07 09:34:18 +02:00
Jakob Unterwurzacher a08ef90811 crossbuild.bash: skip Apple Silicon build on old Go versions 2021-06-06 21:10:24 +02:00
Jakob Unterwurzacher e44d4fcb96 crossbuild.bash: disable CGO
build-without-openssl.bash also disables CGO, so
this makes it more real-world-y.

But the real reason is that disabling CGO hopefully
fixes this travis ci build failure:

+go build -tags without_openssl
/home/travis/.gimme/versions/go1.13.15.linux.amd64/pkg/tool/linux_amd64/link: running gcc failed: exit status 1
/usr/bin/ld: unrecognized option '-pagezero_size'
/usr/bin/ld: use the --help option for usage information
collect2: error: ld returned 1 exit status
The command "./crossbuild.bash" exited with 2.
2021-06-06 19:28:02 +02:00
Jakob Unterwurzacher 17f859d3c4 fusefronted: report plaintext size on symlink creation
gocryptfs 2.0 introduced the regression that the size
reported at symlink creation was the ciphertext size,
which is wrong.

Report the plaintext size.

2021-06-06 19:22:16 +02:00
Jakob Unterwurzacher 6910f86705 crossbuild.bash: also build for Apple M1 2021-06-05 15:07:18 +02:00
Jakob Unterwurzacher e48f2377ec syscallcompat: drop obsolete wrappers
These are now available cross-platform in the unix
2021-06-05 15:06:30 +02:00
Jakob Unterwurzacher c0e7530216 Update README for v2.0 2021-06-05 14:47:57 +02:00
Jakob Unterwurzacher 5ed1a90c7e doc: add benchmark for v2.0, reformat table 2021-06-05 14:46:49 +02:00
Jakob Unterwurzacher 3a1f009c1a Add contrib/atomicrename
$ ./contrib/atomicrename/atomicrename -h
atomicrename creates 100 "src" files in the current directory, renames
them in random order over a single "dst" file while reading the "dst"
file concurrently in a loop.

Progress and errors are reported as they occour in addition to a summary
printed at the end. cifs and fuse filesystems are known to fail, local
filesystems and nfs seem ok.

See for background info.
2021-06-04 22:17:13 +02:00
Jakob Unterwurzacher 015cd066e1 fido2: quote argument strings in debug output
Tested using

  gocryptfs -init -debug -fido2 "hello world" cipherdir

Output before:

  callFidoCommand: executing "/usr/bin/fido2-cred" with args [fido2-cred -M -h -v hello world]


  callFidoCommand: executing "/usr/bin/fido2-cred" with args ["fido2-cred" "-M" "-h" "-v" "hello world"]

2021-06-03 22:03:21 +02:00
Jakob Unterwurzacher 8f2be5d93c fsck: mark temporary mount read-only
We don't write during fsck, but somebody else might try
2021-06-02 19:32:20 +02:00
Jakob Unterwurzacher 307dfd7f71 fsck: clean up temporary mountpoint
We used to leave directories like


behind. Let's clean up after ourselves.
2021-06-02 19:31:55 +02:00
Jakob Unterwurzacher a38e5988ba fusefrontend: run acl Setxattr in user context
The result of setting an acl depends on who runs the

Fixes fuse-xfstests generic/375
2021-06-02 19:10:36 +02:00
Jakob Unterwurzacher b23e21f61f fusefrontend: catch ReadAt integer overflow
Discovered by xfstests generic/564 .

Failure was:

generic/564	- output mismatch (see /opt/fuse-xfstests/results//generic/564.out.bad)
    --- tests/generic/564.out	2021-05-08 21:11:05.307395966 +0200
    +++ /opt/fuse-xfstests/results//generic/564.out.bad	2021-05-19 19:01:16.912888879 +0200
    @@ -31,7 +31,7 @@
     source range beyond 8TiB returns 0

     destination range beyond 8TiB returns EFBIG
    -copy_range: File too large
    +copy_range: Function not implemented
2021-06-02 18:20:05 +02:00
Jakob Unterwurzacher 04858ddd22 nametransform: check name validity on encryption
xfstests generic/523 discovered that we allowed to set
xattrs with "/" in the name, but did not allow to read
them later.

With this change we do not allow to set them in the first
2021-06-02 14:29:48 +02:00
Jakob Unterwurzacher 242cdf966f go.mod: update go-fuse to get acl fixes
Done using:

go get
go mod tidy
2021-05-30 09:34:03 +02:00