Commit Graph

394 Commits

Author SHA1 Message Date
Jakob Unterwurzacher cbf282861b tests/matrix: don't leak fds in TestConcurrentReadCreate
We leaked a file descriptor for each empty file we encountered.
2021-09-01 10:28:33 +02:00
a1346054 7c2255be90 *: trim trailing whitespace 2021-09-01 10:22:01 +02:00
a1346054 c63f7e9f64 shell scripts: fix shellcheck warnings 2021-09-01 10:22:01 +02:00
Jakob Unterwurzacher 34d8a498c4 Unbreak hyperlinks broken by go mod v2 conversion
Commit

  69d88505fd go mod: declare module version v2

translated all instances of "github.com/rfjakob/gocryptfs/" to
"github.com/rfjakob/gocryptfs/v2/".

Unfortunately, this included hyperlinks.

Unbreak the hyperlinks like this:

  find . -name \*.go | xargs sed -i s%https://github.com/rfjakob/gocryptfs/v2/%https://github.com/rfjakob/gocryptfs/v2/%
2021-08-30 11:31:01 +02:00
Jakob Unterwurzacher 7b25ff39c8 fsstress-gocryptfs: fuse-xfstests now lives in /opt
fuse-xfstests should be installed to /opt now to make
the terminal output independent of the user name
(as done in https://github.com/rfjakob/fuse-xfstests/wiki/results_2021-06-02 )
2021-08-26 07:46:28 +02:00
Jakob Unterwurzacher ab7308639b tests/example_filesystems: add v2.2-xchacha-deterministic-names
Combines both new flags.
2021-08-24 13:53:50 +02:00
Jakob Unterwurzacher d70875aaa9 tests/example_filesystems: add deterministic-names and xchacha 2021-08-24 13:50:07 +02:00
Jakob Unterwurzacher b12ad292d4 tests/cli: add -xchacha tests 2021-08-23 16:17:04 +02:00
Jakob Unterwurzacher 2fb1d52746 tests/matrix: add -xchacha test 2021-08-23 16:00:41 +02:00
Jakob Unterwurzacher b02812f8b3 test/cli: actually run TestZerokey
As the filename did not end in _test.go, TestZerokey
was not actually run. Fix that.
	renamed:    tests/cli/zerokey.go -> tests/cli/zerokey_test.go
2021-08-23 15:17:04 +02:00
Jakob Unterwurzacher 69d88505fd go mod: declare module version v2
Our git version is v2+ for some time now, but go.mod
still declared v1. Hopefully making both match makes
https://pkg.go.dev/github.com/rfjakob/gocryptfs/v2 work.

All the import paths have been fixed like this:

  find . -name \*.go | xargs sed -i s%github.com/rfjakob/gocryptfs/%github.com/rfjakob/gocryptfs/v2/%
2021-08-23 15:05:15 +02:00
Jakob Unterwurzacher fbccb16043 -deterministic-names: implement for reverse mode, too 2021-08-20 17:06:18 +02:00
Jakob Unterwurzacher 2a9dea2973 -deterministic-names: accept flag on -init
And store it in gocryptfs.conf (=remove DirIV feature flag).
2021-08-20 15:57:40 +02:00
Jakob Unterwurzacher 195d9d18a9 Implement -deterministic-names: extended -zerodiriv
-deterministc-names uses all-zero dirivs but does not write
them to disk anymore.
2021-08-20 10:58:42 +02:00
Jose M Perez 8f94083a21 Flag -zerodiriv to create all diriv as all zero byte files 2021-08-19 18:05:54 +02:00
Jakob Unterwurzacher be2bd4eec7 golangci-lint: fix issues found by "unused" and "deadcode"
Except xattrSupported, this is a false positive.

$ golangci-lint run --disable-all --enable unused --enable deadcode
gocryptfs-xray/xray_main.go:24:5: `GitVersionFuse` is unused (deadcode)
var GitVersionFuse = "[GitVersionFuse not set - please compile using ./build.bash]"
    ^
tests/symlink_race/main.go:47:6: `chmodLoop` is unused (deadcode)
func chmodLoop() {
     ^
internal/readpassword/extpass_test.go:11:5: `testPw` is unused (deadcode)
var testPw = []byte("test")
    ^
tests/reverse/xattr_test.go:13:6: func `xattrSupported` is unused (unused)
func xattrSupported(path string) bool {
     ^
internal/fusefrontend_reverse/rpath.go:20:22: func `(*RootNode).abs` is unused (unused)
func (rfs *RootNode) abs(relPath string, err error) (string, error) {
                     ^
tests/matrix/matrix_test.go:310:6: `sContains` is unused (deadcode)
func sContains(haystack []string, needle string) bool {
2021-08-19 08:34:49 +02:00
Jakob Unterwurzacher d8b8232c3c test_helpers: actually use global testParentDir variable
Typo inside doInit.
2021-08-19 08:34:49 +02:00
Jakob Unterwurzacher 8ee595dd48 Fix issues found by "go vet"
Issues were:

 # github.com/rfjakob/gocryptfs/contrib/findholes/holes
 contrib/findholes/holes/holes.go:136:2: unreachable code
 # github.com/rfjakob/gocryptfs/tests/root_test_test
 tests/root_test/root_test.go:139:2: unreachable code

Also make sure we actually run "go vet" against the whole
codebase.
2021-08-19 08:34:49 +02:00
Jakob Unterwurzacher c86981342b golangci-lint: fix issues found by gosimple
Everything except the

	if err2.Err == syscall.EOPNOTSUPP

case. Gets too confusing when collapsed into a single line.

Issues were:

$ golangci-lint run --disable-all --enable gosimple
mount.go:473:2: S1008: should use 'return strings.HasPrefix(v, "fusermount version")' instead of 'if strings.HasPrefix(v, "fusermount version") { return true }; return false' (gosimple)
	if strings.HasPrefix(v, "fusermount version") {
	^
cli_args.go:258:5: S1002: should omit comparison to bool constant, can be simplified to `args.forcedecode` (gosimple)
	if args.forcedecode == true {
	   ^
cli_args.go:263:6: S1002: should omit comparison to bool constant, can be simplified to `args.aessiv` (gosimple)
		if args.aessiv == true {
		   ^
cli_args.go:267:6: S1002: should omit comparison to bool constant, can be simplified to `args.reverse` (gosimple)
		if args.reverse == true {
		   ^
internal/stupidgcm/stupidgcm.go:227:6: S1002: should omit comparison to bool constant, can be simplified to `g.forceDecode` (gosimple)
		if g.forceDecode == true {
		   ^
gocryptfs-xray/xray_tests/xray_test.go:23:5: S1004: should use !bytes.Equal(out, expected) instead (gosimple)
	if bytes.Compare(out, expected) != 0 {
	   ^
gocryptfs-xray/xray_tests/xray_test.go:40:5: S1004: should use !bytes.Equal(out, expected) instead (gosimple)
	if bytes.Compare(out, expected) != 0 {
	   ^
gocryptfs-xray/paths_ctlsock.go:34:20: S1002: should omit comparison to bool constant, can be simplified to `!eof` (gosimple)
	for eof := false; eof == false; line++ {
	                  ^
tests/reverse/xattr_test.go:19:2: S1008: should use 'return err2.Err != syscall.EOPNOTSUPP' instead of 'if err2.Err == syscall.EOPNOTSUPP { return false }; return true' (gosimple)
	if err2.Err == syscall.EOPNOTSUPP {
	^
internal/fusefrontend/node.go:459:45: S1002: should omit comparison to bool constant, can be simplified to `!nameFileAlreadyThere` (gosimple)
		if nametransform.IsLongContent(cName2) && nameFileAlreadyThere == false {
		                                          ^
tests/xattr/xattr_integration_test.go:221:2: S1008: should use 'return err2.Err != syscall.EOPNOTSUPP' instead of 'if err2.Err == syscall.EOPNOTSUPP { return false }; return true' (gosimple)
	if err2.Err == syscall.EOPNOTSUPP {
	^
tests/test_helpers/helpers.go:338:19: S1002: should omit comparison to bool constant, can be simplified to `open` (gosimple)
	if err != nil && open == true {
	                 ^
tests/matrix/concurrency_test.go:121:7: S1004: should use !bytes.Equal(buf, content) instead (gosimple)
			if bytes.Compare(buf, content) != 0 {
			   ^
2021-08-19 08:34:44 +02:00
Jakob Unterwurzacher 64793fedf4 Fix issues found by ineffassign
gocryptfs$ ineffassign ./...

/home/jakob/go/src/github.com/rfjakob/gocryptfs/internal/configfile/config_file.go:243:2: ineffectual assignment to scryptHash
/home/jakob/go/src/github.com/rfjakob/gocryptfs/internal/configfile/config_file.go:272:2: ineffectual assignment to scryptHash
/home/jakob/go/src/github.com/rfjakob/gocryptfs/internal/fusefrontend/file.go:285:3: ineffectual assignment to fileID
/home/jakob/go/src/github.com/rfjakob/gocryptfs/internal/fusefrontend/node.go:367:3: ineffectual assignment to err
/home/jakob/go/src/github.com/rfjakob/gocryptfs/internal/fusefrontend/node_open_create.go:68:2: ineffectual assignment to fd
/home/jakob/go/src/github.com/rfjakob/gocryptfs/mount.go:308:2: ineffectual assignment to masterkey
/home/jakob/go/src/github.com/rfjakob/gocryptfs/gocryptfs-xray/xray_main.go:156:13: ineffectual assignment to err
/home/jakob/go/src/github.com/rfjakob/gocryptfs/internal/fusefrontend/prepare_syscall_test.go:65:16: ineffectual assignment to errno
/home/jakob/go/src/github.com/rfjakob/gocryptfs/internal/syscallcompat/open_nofollow_test.go:34:2: ineffectual assignment to fd
/home/jakob/go/src/github.com/rfjakob/gocryptfs/tests/defaults/acl_test.go:111:6: ineffectual assignment to err
/home/jakob/go/src/github.com/rfjakob/gocryptfs/tests/defaults/acl_test.go:181:2: ineffectual assignment to sz
/home/jakob/go/src/github.com/rfjakob/gocryptfs/tests/defaults/acl_test.go:198:2: ineffectual assignment to sz
/home/jakob/go/src/github.com/rfjakob/gocryptfs/tests/defaults/main_test.go:365:8: ineffectual assignment to err
/home/jakob/go/src/github.com/rfjakob/gocryptfs/tests/xattr/xattr_fd_test.go:30:6: ineffectual assignment to err
/home/jakob/go/src/github.com/rfjakob/gocryptfs/tests/xattr/xattr_fd_test.go:66:6: ineffectual assignment to err
2021-08-18 15:48:01 +02:00
Jakob Unterwurzacher 0bc9784508 reverse: fix "exclude all but" case
With test.

Fixes https://github.com/rfjakob/gocryptfs/issues/588
2021-08-18 11:38:56 +02:00
Jakob Unterwurzacher 3df1c62430 tests/reverse/TestExcludeTestFs: test trailing slash 2021-08-18 11:04:56 +02:00
Jakob Unterwurzacher 884398eec3 tests/reverse/TestExcludeTestFs: improve comments & code style
No functional changes.
2021-08-18 11:04:40 +02:00
Jakob Unterwurzacher b8ddc49ede tests/cli/TestBadname: make sure case 5 is never decodable
Sometimes, by chance, case 5 resulted in valid decrypted data:

--- FAIL: TestBadname (0.08s)
    cli_test.go:885: Case 5 failed: "J7Rbo1BvfXojpBEr0Qrt_invalid_file GOCRYPTFS_BAD_NAME" in ["file GOCRYPTFS_BAD_NAME,\x9e$O\xc3j\x8c\xd0\x06\x01#\f%k\x02\xcanvalid_file GOCRYPTFS_BAD_NAME,mzaZRF9_0IU-_5vv2wPC_invalid_file GOCRYPTFS_BAD_NAME,file,file_invalid_file GOCRYPTFS_BAD_NAME,mzaZRF9_0IU-_5vv2wP_invalid_file GOCRYPTFS_BAD_NAME"]

Add percent signs so base64 decoding always fails.

Fixes https://github.com/rfjakob/gocryptfs/runs/3347883728
2021-08-17 15:17:36 +02:00
Jakob Unterwurzacher eeb267950a tests/reverse: replace os.ReadDir to support older Go versions 2021-08-16 22:05:33 +02:00
Jakob Unterwurzacher e2d3834b1c tests/root_test: show failing command detail
https://github.com/kdave/btrfs-progs/issues/392
2021-08-16 19:24:04 +02:00
Jakob Unterwurzacher b2724070d9 reverse mode: implement -one-file-system
Fixes https://github.com/rfjakob/gocryptfs/issues/475
2021-08-16 19:23:58 +02:00
Jakob Unterwurzacher ad4b99170b tests/cli: escape filenames in TestBadname logs
I just got this message (not reproducible) with unescaped
binary garbage.

  UnmountErr: "/var/tmp/gocryptfs-test-parent-1026/114471933/TestMountBackground.899727687.mnt" was not found in MountInfo, cannot check for FD leaks
  UnmountErr: "/var/tmp/gocryptfs-test-parent-1026/114471933/TestConfigPipe.212912444.mnt" was not found in MountInfo, cannot check for FD leaks
  DecryptName "mzaZRF9_0IU-_5vv2wPC_i": unPad16 error: Padding too long, padLen=49 > 16
  OpenDir ".": invalid entry "KqQ346cuOAFHv_qSta5PhAwrongPattern": bad message
  DecryptName "mzaZRF9_0IU-_5vv2wP_in": unPad16 error: Padding byte at i=10 is invalid
  --- FAIL: TestBadname (0.11s)
      cli_test.go:885: Case 5 failed: 'KqQ346cuOAFHv_qSta5P_invalid_file GOCRYPTFS_BAD_NAME' in [file_invalid_file GOCRYPTFS_BAD_NAME,file,mzaZRF9_0IU-_5vv2wPC_invalid_file GOCRYPTFS_BAD_NAME,file GOCRYPTFS_BAD_NAME,�*A���y���Gfnvalid_file GOCRYPTFS_BAD_NAME,mzaZRF9_0IU-_5vv2wP_invalid_file GOCRYPTFS_BAD_NAME]
  Invalid cipherdir: directory /var/tmp/gocryptfs-test-parent-1026/114471933/TestInitNotEmpty not empty
  FAIL
  FAIL	github.com/rfjakob/gocryptfs/tests/cli	4.817s
2021-08-16 17:15:07 +02:00
Jakob Unterwurzacher 2d386fc92e syscallcompat: move quirks logic here & fix darwin
We need to look at f_fstypename acc. to
https://stackoverflow.com/a/52299141/1380267 :

> As filesystem type numbers are now assigned at runtime in
> recent versions of MacOS, you must use f_fstypename to
> determine the type.

https://github.com/rfjakob/gocryptfs/issues/585
2021-08-11 20:23:35 +02:00
Jakob Unterwurzacher 1bc1db620b fusefrontend: -sharedstorage: present stable inode numbers
Use the Gen field (inode generation) to distinguish hard links
while passing the real inode numbers to userspace.

Fixes https://github.com/rfjakob/gocryptfs/issues/584
2021-07-31 13:24:25 +02:00
Jakob Unterwurzacher eecbcbb090 tests: matrix: add TestPwd
https://github.com/rfjakob/gocryptfs/issues/584
2021-07-31 13:23:05 +02:00
Jakob Unterwurzacher 0ca302f12a fusefrontend: implement fsync on directories
Fixes https://github.com/rfjakob/gocryptfs/issues/587
2021-07-29 20:39:50 +02:00
Jakob Unterwurzacher db81614cd6 canonical-benchmarks.bash: handle relative paths
Passing a relative path was broken because we cd'ed
away first.
2021-06-27 11:48:41 +02:00
Jakob Unterwurzacher 49507ea869 tests/fsck: delete obsolete script run_fsck.bash
Not called by anybody.
2021-06-26 19:27:58 +02:00
Jakob Unterwurzacher ad3eeaedc5 tests, maxlen.bash: speed up TestMaxlen using QUICK=1
From >6 to <1 second.
2021-06-26 19:13:24 +02:00
Jakob Unterwurzacher 446c3d7e93 tests: matrix: show content detail on mismatch 2021-06-26 18:58:29 +02:00
Jakob Unterwurzacher 1f29542b39 tests: better error message on ctlsock query failure 2021-06-26 18:49:54 +02:00
Jakob Unterwurzacher cbd5e8ba01 tests/default: add maxlen.bash test 2021-06-26 16:09:04 +02:00
Jakob Unterwurzacher 05b813f202 nametransform: rename BadNameFlag to BadnameSuffix 2021-06-21 12:12:44 +02:00
Jakob Unterwurzacher 2efef1e270 nametransform: delete NameTransformer interface
Useless layer of indirection.
2021-06-21 11:53:33 +02:00
Jakob Unterwurzacher e244b51491 tests: cli: add TestZerokey
TestZerokey verifies that `gocryptfs -zerokey` uses the same options as
`gocryptfs -init`.
2021-06-21 11:48:16 +02:00
DerDonut a611810ff4 Badname file content access
This proposal is the counterpart of the modifications from the `-badname`
parameter. It modifies the plain -> cipher mapping for filenames when using
`-badname` parameter. The new function `EncryptAndHashBadName` tries to find a
cipher filename for the given plain name with the following steps:

1. If `badname` is disabled or direct mapping is successful: Map directly
(default and current behaviour)

2. If a file with badname flag has a valid cipher file, this is returned
(=File just ends with the badname flag)

3. If a file with a badname flag exists where only the badname flag was added,
this is returned (=File cipher name could not be decrypted by function
`DecryptName` and just the badname flag was added)

4. Search for all files which cipher file name extists when cropping more and
more characters from the end. If only 1 file is found, return this

5. Return an error otherwise

This allows file access in the file browsers but most important it allows that
you rename files with undecryptable cipher names in the plain directories.
Renaming those files will then generate a proper cipher filename One
backdraft: When mounting the cipher dir with -badname parameter, you can never
create (or rename to) files whose file name ends with the badname file flag
(at the moment this is " GOCRYPTFS_BAD_NAME"). This will cause an error.

I modified the CLI test function to cover additional test cases. Test [Case
7](https://github.com/DerDonut/gocryptfs/blob/badnamecontent/tests/cli/cli_test.go#L712)
cannot be performed since the cli tests are executed in panic mode. The
testing is stopped on error. Since the function`DecryptName` produces internal
errors when hitting non-decryptable file names, this test was omitted.

This implementation is a proposal where I tried to change the minimum amount
of existing code. Another possibility would be instead of creating the new
function `EncryptAndHashBadName` to modify the signature of the existing
function `EncryptAndHashName(name string, iv []byte)` to
`EncryptAndHashName(name string, iv []byte, dirfd int)` and integrate the
functionality into this function directly. You may allow calling with dirfd=-1
or other invalid values an then performing the current functionality.
2021-06-20 18:09:21 +02:00
Jakob Unterwurzacher 9ac77d42d1 contrib/maxlen.bash: also test dir and path length
Move the script from tests to contrib as it may now
be useful to somebody else.

https://github.com/rfjakob/gocryptfs/issues/552
2021-06-20 18:09:21 +02:00
Jakob Unterwurzacher 17f859d3c4 fusefronted: report plaintext size on symlink creation
gocryptfs 2.0 introduced the regression that the size
reported at symlink creation was the ciphertext size,
which is wrong.

Report the plaintext size.

Fixes https://github.com/rfjakob/gocryptfs/issues/574
2021-06-06 19:22:16 +02:00
Jakob Unterwurzacher 04858ddd22 nametransform: check name validity on encryption
xfstests generic/523 discovered that we allowed to set
xattrs with "/" in the name, but did not allow to read
them later.

With this change we do not allow to set them in the first
place.
2021-06-02 14:29:48 +02:00
Jakob Unterwurzacher 0e37fbe042 tests: TestFileHoleCopy: accept +-4kB
Failure looked like this:

--- FAIL: TestFileHoleCopy (3.73s)
    --- FAIL: TestFileHoleCopy/k81 (0.04s)
        file_holes_test.go:93: size changed: st0.Blocks=88 st2.Blocks=96
    file_holes_test.go:147: aborting further subtests

$ findholes TestFileHoleCopy.k81.1
         0 data
     36864 hole
     45056 data
     50434 hole
     50434 eof

$ findholes TestFileHoleCopy.k81.2
         0 data
     36864 hole
     45056 data
     50434 hole
     50434 eof

$ filefrag -v TestFileHoleCopy.k81.1
Filesystem type is: ef53
File size of TestFileHoleCopy.k81.1 is 50434 (13 blocks of 4096 bytes)
 ext:     logical_offset:        physical_offset: length:   expected: flags:
   0:        0..       2:   23702311..  23702313:      3:
   1:        3..       8:   20389855..  20389860:      6:   23702314:
   2:       11..      12:   23702314..  23702315:      2:   20389863: last,eof
TestFileHoleCopy.k81.1: 3 extents found

$ filefrag -v TestFileHoleCopy.k81.2
Filesystem type is: ef53
File size of TestFileHoleCopy.k81.2 is 50434 (13 blocks of 4096 bytes)
 ext:     logical_offset:        physical_offset: length:   expected: flags:
   0:        0..       2:   20389861..  20389863:      3:
   1:        3..       4:   23702316..  23702317:      2:   20389864:
   2:        5..       6:   20389864..  20389865:      2:   23702318:
   3:        7..       8:   23702318..  23702319:      2:   20389866:
   4:       11..      12:   23702320..  23702321:      2:             last,eof
TestFileHoleCopy.k81.2: 4 extents found
2021-05-29 16:56:20 +02:00
Jakob Unterwurzacher 738a9e006a fusefrontend: rewrite Lseek SEEK_DATA / SEEK_HOLE
In response to the discussion of the xfstests mailing list [1],
I looked at the Lseek implementation, which was naive and
did not handle all cases correctly.

The new implementation aligns the returned values to 4096 bytes
as most callers expect.

A lot of tests are added to verify that we handle all
cases correctly now.

[1]: https://www.spinics.net/lists/fstests/msg16554.html
2021-05-29 16:05:36 +02:00
Jakob Unterwurzacher 9695f0e524 tests: add TestFileHoleCopy
Currently fails.
2021-05-26 13:17:56 +02:00
Jakob Unterwurzacher 629d0a5ad7 tests: re-enable TestInoReuseEvil
Problem in go-fuse has long been fixed.
2021-05-26 13:17:56 +02:00
Jakob Unterwurzacher 7d72baca69 tests: add TestHaveDotdot
As discovered by xfstests generic/401 [1], during the move to
the v2 api we seem to have lost the "." and ".." directory
entries.

[1]: 4ef5b032bc/screenlog.0 (L520)
2021-05-26 13:17:56 +02:00