This proposal is the counterpart of the modifications from the `-badname`
parameter. It modifies the plain -> cipher mapping for filenames when using
`-badname` parameter. The new function `EncryptAndHashBadName` tries to find a
cipher filename for the given plain name with the following steps:
1. If `badname` is disabled or direct mapping is successful: Map directly
(default and current behaviour)
2. If a file with badname flag has a valid cipher file, this is returned
(=File just ends with the badname flag)
3. If a file with a badname flag exists where only the badname flag was added,
this is returned (=File cipher name could not be decrypted by function
`DecryptName` and just the badname flag was added)
4. Search for all files which cipher file name extists when cropping more and
more characters from the end. If only 1 file is found, return this
5. Return an error otherwise
This allows file access in the file browsers but most important it allows that
you rename files with undecryptable cipher names in the plain directories.
Renaming those files will then generate a proper cipher filename One
backdraft: When mounting the cipher dir with -badname parameter, you can never
create (or rename to) files whose file name ends with the badname file flag
(at the moment this is " GOCRYPTFS_BAD_NAME"). This will cause an error.
I modified the CLI test function to cover additional test cases. Test [Case
7](https://github.com/DerDonut/gocryptfs/blob/badnamecontent/tests/cli/cli_test.go#L712)
cannot be performed since the cli tests are executed in panic mode. The
testing is stopped on error. Since the function`DecryptName` produces internal
errors when hitting non-decryptable file names, this test was omitted.
This implementation is a proposal where I tried to change the minimum amount
of existing code. Another possibility would be instead of creating the new
function `EncryptAndHashBadName` to modify the signature of the existing
function `EncryptAndHashName(name string, iv []byte)` to
`EncryptAndHashName(name string, iv []byte, dirfd int)` and integrate the
functionality into this function directly. You may allow calling with dirfd=-1
or other invalid values an then performing the current functionality.
gocryptfs 2.0 introduced the regression that the size
reported at symlink creation was the ciphertext size,
which is wrong.
Report the plaintext size.
Fixes https://github.com/rfjakob/gocryptfs/issues/574
xfstests generic/523 discovered that we allowed to set
xattrs with "/" in the name, but did not allow to read
them later.
With this change we do not allow to set them in the first
place.
In response to the discussion of the xfstests mailing list [1],
I looked at the Lseek implementation, which was naive and
did not handle all cases correctly.
The new implementation aligns the returned values to 4096 bytes
as most callers expect.
A lot of tests are added to verify that we handle all
cases correctly now.
[1]: https://www.spinics.net/lists/fstests/msg16554.html
As discovered by xfstests generic/401 [1], during the move to
the v2 api we seem to have lost the "." and ".." directory
entries.
[1]: 4ef5b032bc/screenlog.0 (L520)
Also fix incomplete uid restoration in TestSupplementaryGroups
and replace syscall.Setregid and friends with unix.Setregid and
friends.
This test is added to check if have problems handling a full disk.
The ticket https://github.com/rfjakob/gocryptfs/issues/550 states
that the full disk was not where the backing gocryptfs filesystem
was, but this has no effect on gocryptfs, so we test the harder
case.
Make `gocryptfs.diriv` and `gocryptfs.xxx.name` files world-readable to make encrypted backups easier
when mounting via fstab.
Having the files follow chmod/chown of their parent does not seem
to be worth the hassle. The content of the diriv files is not
secret, and both diriv and name files are protected by the
perms of the parent dir.
Fixes https://github.com/rfjakob/gocryptfs/issues/539
We would hang like this
./fsstress-loopback.bash
Recompile go-fuse loopback: v2.0.3-7-g0b6cbc5
Waiting for mount: xxxxxxxx^C
if TMPDIR has a trailing /. The reason is that the
paths in /proc/self/mounts are normalized, while
TMPDIR may not be.
Directly accessing the Nodes does not work properly,
as there is no way to attach a newly LOOKUPped Node
to the tree. This means Path() does not work.
Use an actual mount instead and walk the tree.
Chasing a bug that seems to have nothing to do
with magic names, as it already triggers during
warmup:
--- FAIL: TestMagicNames (0.00s)
matrix_test.go:773: Testing n="warmup1"
matrix_test.go:773: Testing n="warmup2"
matrix_test.go:820: no such file or directory
Simplify the tests by using empty files. Empty
files are valid, and we don't check the content
anyway.
Also adjust comment style a little and add
a missing break statement.
Changed invalid file decoding and decryption. Function
DecryptName now shortens the filename until the filename is
decodable and decryptable. Will work with valid **and**
invalid Base64URL delimiter (valid delimiter [0-9a-zA-z_\\-].
If the filename is not decryptable at all, it returns the
original cipher name with flag suffix Changed cli tests to
generate decryptable and undecryptable file names with correct
encrypted content. Replacing #474, extends #393
Locale trouble was
sshfs-benchmark.bash: line 31: printf: 4.71: invalid number
because printf expected "4,71" in the German locale.
Force the C locale.
Looking at the dircache debug output, we see
that a "git status" workload has a very bad
cache hit rate because the entries expire or
get evicted before they can be reused.
Increase both cache size and lifetime for
a 4x speedup:
Before: 75s
After: 17s
https://github.com/rfjakob/gocryptfs/issues/410
Move the statusTxtContent to fix this confusing error
when running `go get github.com/rfjakob/gocryptfs/...`:
$ go get github.com/rfjakob/gocryptfs/...
# github.com/rfjakob/gocryptfs/tests/example_filesystems
tests/example_filesystems/example_test_helpers.go:22:16: undefined: statusTxtContent
tests/example_filesystems/example_test_helpers.go:75:16: undefined: statusTxtContent
On Fedora, /tmp is tmpfs, which behaves differently than ext4
(inode numbers are never reused, for example).
Use /var/tmp, which is ext4 on Fedora, to get a more realistic
test environment.
This also allows us to drop the xattr workaround.
When running
$ go test ./tests/matrix/
in isolation, it failed like this:
fd leak? before, after:
[0r=/dev/null 3w=/dev/null 5r=/proc/8078/fd (hidden:4)]
[0r=/dev/null 3w=/dev/null 5w=/tmp/go-build366655199/b001/testlog.txt 7r=/proc/8078/fd (hidden:4)]
Filter by prefix to get rid of this spurious test failure.
When we run tests as root, they will leave root-owned files
in testParentDir, which causes trouble when we run tests as
a normal user later on. Split by UID.
CheckTrailingGarbage was called even when "-passfile" was
used, which is stupid, and causes false positives:
https://github.com/rfjakob/gocryptfs/issues/391
(false error "Received trailing garbage after the password"
when using -passfile in .bash_profile)
Instead of trying to improve the logic to handle that case
and make everything even more complicated, delete the function.
It is unclear if actually helps in some cases, and it definitely
harms as shown by the above bug report.
This adds support for gitignore-like wildcards and exclude patters in
reverse mode. It (somewhat) fixes#273: no regexp support, but the
syntax should be powerful enough to satisfy most needs.
Also, since adding a lot of --exclude options can be tedious, it adds
the --exclude-from option to read patterns from a file (or files).
Ironically, the test for DirIV races had a data race itself
by writing to a bool without taking a lock.
Found by running "./test.bash -race":
WARNING: DATA RACE
Write at 0x00c00001dea5 by goroutine 22:
github.com/rfjakob/gocryptfs/tests/defaults.TestDirIVRace.func1()
/home/jakob/go/src/github.com/rfjakob/gocryptfs/tests/defaults/diriv_test.go:39 +0x38
github.com/rfjakob/gocryptfs/tests/defaults.TestDirIVRace()
/home/jakob/go/src/github.com/rfjakob/gocryptfs/tests/defaults/diriv_test.go:73 +0x65c
testing.tRunner()
/usr/local/go/src/testing/testing.go:865 +0x163
Previous read at 0x00c00001dea5 by goroutine 23:
github.com/rfjakob/gocryptfs/tests/defaults.TestDirIVRace.func2()
/home/jakob/go/src/github.com/rfjakob/gocryptfs/tests/defaults/diriv_test.go:51 +0x8b
Goroutine 22 (running) created at:
testing.(*T).Run()
/usr/local/go/src/testing/testing.go:916 +0x699
testing.runTests.func1()
/usr/local/go/src/testing/testing.go:1157 +0xa8
testing.tRunner()
/usr/local/go/src/testing/testing.go:865 +0x163
testing.runTests()
/usr/local/go/src/testing/testing.go:1155 +0x523
testing.(*M).Run()
/usr/local/go/src/testing/testing.go:1072 +0x2eb
github.com/rfjakob/gocryptfs/tests/defaults.TestMain()
/home/jakob/go/src/github.com/rfjakob/gocryptfs/tests/defaults/main_test.go:21 +0xe1
main.main()
_testmain.go:76 +0x222
Goroutine 23 (running) created at:
github.com/rfjakob/gocryptfs/tests/defaults.TestDirIVRace()
/home/jakob/go/src/github.com/rfjakob/gocryptfs/tests/defaults/diriv_test.go:43 +0x48d
testing.tRunner()
/usr/local/go/src/testing/testing.go:865 +0x163
==================
--- FAIL: TestDirIVRace (0.00s)
testing.go:809: race detected during execution of test
FAIL
Jakob Unterwurzacher