Commit Graph

1934 Commits (c50d67f1039b8db90de441a3907994adb21668e8)

Author SHA1 Message Date
Jakob Unterwurzacher b8d78d6a31 inomap: warn on first use of spillMap
We normally should not need it, warn if we do.
As the tests run with -wpanic, we would catch it.
2021-08-16 17:13:14 +02:00
Jakob Unterwurzacher dd24fed532 Makefile: add uninstall target 2021-08-13 10:41:48 +02:00
Jakob Unterwurzacher 9a8dfd98ef main: accept magic /dev/fd/ mountpoint
2021-08-12 19:19:50 +02:00
Jakob Unterwurzacher 831e225616 syscallcompat: use BTRFS_SUPER_MAGIC from unix lib 2021-08-11 20:28:20 +02:00
Jakob Unterwurzacher 4e7efb1dff Makefile: don't ever run "git clean -dxff"
"make ci" almost made me lose the new quirks files.
Let's not do that anymore.
2021-08-11 20:23:46 +02:00
Jakob Unterwurzacher 2d386fc92e syscallcompat: move quirks logic here & fix darwin
We need to look at f_fstypename acc. to :

> As filesystem type numbers are now assigned at runtime in
> recent versions of MacOS, you must use f_fstypename to
> determine the type.
2021-08-11 20:23:35 +02:00
Jakob Unterwurzacher 0c16616117 main: add testcases for convertToDoubleDash & parseCliOpts 2021-08-10 19:42:33 +02:00
Jakob Unterwurzacher 463f6e8962 main: take advantage of pflag slice types
Our multipleStrings type is now built in.
2021-08-10 19:09:58 +02:00
Jakob Unterwurzacher 88f5e8d76e main: show specific error on command line parse failure 2021-08-10 19:09:12 +02:00
Jakob Unterwurzacher 527e72bf80 main: autoformat import block
The autoformatter now always wants to move the ensurefds012
import into the import block. Accept it and fix the spelling
of "alphabetically".
2021-08-10 18:28:27 +02:00
Jakob Unterwurzacher f53f52b046 main: switch from flag to pflag
Need support for flags at any position for
2021-08-10 18:24:35 +02:00
Jakob Unterwurzacher 8c9a1c1121 main: push TestPrefixOArgs testcase struct into TestPrefixOArgs
No need to have it declared globally.
2021-08-10 09:33:19 +02:00
Jakob Unterwurzacher c3c9513e65 fusefrontend: add quirks for MacOS ExFAT
This also moves the quirks logic into fusefrontend.

2021-08-02 20:01:26 +02:00
Jakob Unterwurzacher 75cf36fe7b go mod: upgrade go-fuse
Let's not crash anymore when we see inode number 1
( 0aaef6dde4 )
2021-08-02 19:16:53 +02:00
Jakob Unterwurzacher c6b0c777a1 README: Update Changelog 2021-07-31 17:48:44 +02:00
Jakob Unterwurzacher 1bc1db620b fusefrontend: -sharedstorage: present stable inode numbers
Use the Gen field (inode generation) to distinguish hard links
while passing the real inode numbers to userspace.

2021-07-31 13:24:25 +02:00
Jakob Unterwurzacher eecbcbb090 tests: matrix: add TestPwd
2021-07-31 13:23:05 +02:00
Jakob Unterwurzacher 1dfd6b7b76 fusefrontend: prepareAtSyscall: handle error when opening ourselves
Error handling was missing here, so we would later get confusing
EBADF errors due to dirfd being -1.
2021-07-31 10:53:32 +02:00
Jakob Unterwurzacher 0ca302f12a fusefrontend: implement fsync on directories
2021-07-29 20:39:50 +02:00
Jakob Unterwurzacher e83b79b4c2 fido2: actually drop `-v` flag
Commit 2a9d70d48f only
dropped the flag on mount but not on `-init`.

Also drop it on `-init`.

Fixes (part II)
2021-07-29 12:47:40 +02:00
Jakob Unterwurzacher 51bddd826e go mod: set version to 1.16 & drop explicit "-mod=vendor" from ci
This makes "go build" automatically use the vendor
directory, if present.

See for details.

Up to now, we ignored the vendor dir completely!

2021-07-29 12:36:53 +02:00
Jakob Unterwurzacher 6f0ed4b8c4 github ci: Add Github Actions CI
Add Github Actions and delete defunct Travis CI.
2021-07-29 12:36:05 +02:00
Jakob Unterwurzacher db81614cd6 canonical-benchmarks.bash: handle relative paths
Passing a relative path was broken because we cd'ed
away first.
2021-06-27 11:48:41 +02:00
Jakob Unterwurzacher 2a9d70d48f fido2: drop `-v` option (PIN request)
We used to pass `-v` on `gocryptfs -init` but not for
mount, which seems strange by itself, but more importantly,
`-v` does not work on Yubikeys.

Drop `-v`.

2021-06-27 11:17:29 +02:00
Jakob Unterwurzacher d6c8d892ff fido2: pretty-print fidoCommand in debug output
2021-06-27 11:12:40 +02:00
Jakob Unterwurzacher fe616ddad5 doc: update performance.txt 2021-06-26 20:57:39 +02:00
Jakob Unterwurzacher 49507ea869 tests/fsck: delete obsolete script run_fsck.bash
Not called by anybody.
2021-06-26 19:27:58 +02:00
Jakob Unterwurzacher ad3eeaedc5 tests, maxlen.bash: speed up TestMaxlen using QUICK=1
From >6 to <1 second.
2021-06-26 19:13:24 +02:00
Jakob Unterwurzacher 446c3d7e93 tests: matrix: show content detail on mismatch 2021-06-26 18:58:29 +02:00
Jakob Unterwurzacher 4fd95b718b fusefrontend: delete openBackingDir 2021-06-26 18:49:54 +02:00
Jakob Unterwurzacher 5306fc345b fusefrontend: convert last callers from openBackingDir to prepareAtSyscall 2021-06-26 18:49:54 +02:00
Jakob Unterwurzacher 1f29542b39 tests: better error message on ctlsock query failure 2021-06-26 18:49:54 +02:00
Jakob Unterwurzacher 45648e567a fusefrontend: ctlsock: get rid of unneccessary wrapper function 2021-06-26 18:49:54 +02:00
Jakob Unterwurzacher f9f4bd214f fusefrontend: convert ctlsock from openBackingDir to prepareAtSyscall
openBackingDir will be removed.

Also, remove leftover debug printfs.
2021-06-26 18:49:54 +02:00
Jakob Unterwurzacher ee59b5269b fusefrontend: convert openBackingDir tests to prepareAtSyscall
openBackingDir will be removed.
2021-06-26 16:28:30 +02:00
Jakob Unterwurzacher cbd5e8ba01 tests/default: add maxlen.bash test 2021-06-26 16:09:04 +02:00
Jakob Unterwurzacher 389aba6a6b maxlen.bash: suppress progress output if not on a terminal 2021-06-26 16:08:29 +02:00
Jakob Unterwurzacher 84e702126a fusefrontend: implement recursive diriv caching
The new contrib/maxlen.bash showed that we have exponential
runtime with respect to directory depth.

The new recursive diriv caching is a lot smarter as it caches
intermediate lookups. maxlen.bash now completes in a few seconds.

xfstests results same as
2d158e4c82/screenlog.0 :

  Failures: generic/035 generic/062 generic/080 generic/093 generic/099 generic/215 generic/285 generic/319 generic/426 generic/444 generic/467 generic/477 generic/523
  Failed 13 of 580 tests

benchmark.bash results are identical:

  $ ./benchmark.bash
  Testing gocryptfs at /tmp/benchmark.bash.BdQ: gocryptfs v2.0.1-17-g6b09bc0; go-fuse v2.1.1-0.20210611132105-24a1dfe6b4f8; 2021-06-25 go1.16.5 linux/amd64
  /tmp/benchmark.bash.BdQ.mnt is a mountpoint
  WRITE: 262144000 bytes (262 MB, 250 MiB) copied, 0,4821 s, 544 MB/s
  READ:  262144000 bytes (262 MB, 250 MiB) copied, 0,266061 s, 985 MB/s
  UNTAR: 8,280
  MD5:   4,564
  LS:    1,745
  RM:    2,244
2021-06-25 13:56:53 +02:00
Jakob Unterwurzacher 05b813f202 nametransform: rename BadNameFlag to BadnameSuffix 2021-06-21 12:12:44 +02:00
Jakob Unterwurzacher 689b74835b nametransform: gather badname functions in badname.go 2021-06-21 12:10:04 +02:00
Jakob Unterwurzacher 2efef1e270 nametransform: delete NameTransformer interface
Useless layer of indirection.
2021-06-21 11:53:33 +02:00
Jakob Unterwurzacher e244b51491 tests: cli: add TestZerokey
TestZerokey verifies that `gocryptfs -zerokey` uses the same options as
`gocryptfs -init`.
2021-06-21 11:48:16 +02:00
Jakob Unterwurzacher 6b0e63c1a8 Improve startup debug output
The startup debug output was very verbose but still missing some
effective crypto settings.
2021-06-21 11:32:04 +02:00
Jakob Unterwurzacher c5d8fa83ae nametransform: pass badname patterns via New
This means we can unexport the field.
2021-06-20 19:09:46 +02:00
Jakob Unterwurzacher 203e65066f main: use JSONDump helper for debug output 2021-06-20 18:25:07 +02:00
Jakob Unterwurzacher 50630e9f3d fido2: hide "FIDO2" in gocryptfs.conf if not used
Result of:

$ gocryptfs -init foo
$ cat foo/gocryptfs.conf


	"Creator": "gocryptfs v2.0.1",
	"EncryptedKey": "FodEdNHD/cCwv1n5BuyAkbIOnJ/O5gfdCh3YssUCJ2DUr0A8DrQ5NH2SLhREeWRL3V8EMiPO2Ncr5IVwE4SSxQ==",
	"ScryptObject": {
		"Salt": "brGaw9Jg1kbPuSXFiwoxqK2oXFTgbniSgpiB+cu+67Y=",
		"N": 65536,
		"R": 8,
		"P": 1,
		"KeyLen": 32
	"Version": 2,
	"FeatureFlags": [
	"FIDO2": {
		"CredentialID": null,
		"HMACSalt": null


	"Creator": "gocryptfs v2.0.1-5-gf9718eb-dirty.DerDonut-badnamecontent",
	"EncryptedKey": "oFMj1lS1ZsM/vEfanNMeCTPw3PZr5VWeL7ap8Jd8YQm6evy2BAhtQ/pd6RzDx84wlCz9TpxqHRihuwSEMnOWWg==",
	"ScryptObject": {
		"Salt": "JZ/5mhy4a8EAQ/wDF1POIEe4/Ss38cfJgXgj26DuA4M=",
		"N": 65536,
		"R": 8,
		"P": 1,
		"KeyLen": 32
	"Version": 2,
	"FeatureFlags": [
2021-06-20 18:09:21 +02:00
DerDonut a611810ff4 Badname file content access
This proposal is the counterpart of the modifications from the `-badname`
parameter. It modifies the plain -> cipher mapping for filenames when using
`-badname` parameter. The new function `EncryptAndHashBadName` tries to find a
cipher filename for the given plain name with the following steps:

1. If `badname` is disabled or direct mapping is successful: Map directly
(default and current behaviour)

2. If a file with badname flag has a valid cipher file, this is returned
(=File just ends with the badname flag)

3. If a file with a badname flag exists where only the badname flag was added,
this is returned (=File cipher name could not be decrypted by function
`DecryptName` and just the badname flag was added)

4. Search for all files which cipher file name extists when cropping more and
more characters from the end. If only 1 file is found, return this

5. Return an error otherwise

This allows file access in the file browsers but most important it allows that
you rename files with undecryptable cipher names in the plain directories.
Renaming those files will then generate a proper cipher filename One
backdraft: When mounting the cipher dir with -badname parameter, you can never
create (or rename to) files whose file name ends with the badname file flag
(at the moment this is " GOCRYPTFS_BAD_NAME"). This will cause an error.

I modified the CLI test function to cover additional test cases. Test [Case
cannot be performed since the cli tests are executed in panic mode. The
testing is stopped on error. Since the function`DecryptName` produces internal
errors when hitting non-decryptable file names, this test was omitted.

This implementation is a proposal where I tried to change the minimum amount
of existing code. Another possibility would be instead of creating the new
function `EncryptAndHashBadName` to modify the signature of the existing
function `EncryptAndHashName(name string, iv []byte)` to
`EncryptAndHashName(name string, iv []byte, dirfd int)` and integrate the
functionality into this function directly. You may allow calling with dirfd=-1
or other invalid values an then performing the current functionality.
2021-06-20 18:09:21 +02:00
Jakob Unterwurzacher cdddd1d711 MANPAGE: describe -badname 2021-06-20 18:09:21 +02:00
Jakob Unterwurzacher 9ac77d42d1 contrib/maxlen.bash: also test dir and path length
Move the script from tests to contrib as it may now
be useful to somebody else.
2021-06-20 18:09:21 +02:00
Marcel Bochtler 2e738efe86 README: Add MacPorts install instructions
See [1] for the Portfile.

2021-06-19 20:22:20 +02:00