Commit Graph

101 Commits

Author SHA1 Message Date
Jakob Unterwurzacher
87a6bb370a nametransform: fix math.MaxInt build failure on older Go
Failure is:

  # github.com/rfjakob/gocryptfs/v2/internal/nametransform
  internal/nametransform/names.go:47:33: undefined: math.MaxInt

math.MaxInt was only introduced in Go 1.17. Use MaxInt32 instead,
which is good enough, even on amd64. It only has to be larger than
any name we might encounter.
2021-10-21 16:44:05 +02:00
Jakob Unterwurzacher
dc32710045 nametransform: add longNameMax parameter
Determines when to start hashing long names instead
of hardcoded 255. Will be used to alleviate "name too long"
issues some users see on cloud storage.

https://github.com/rfjakob/gocryptfs/issues/499
2021-10-21 14:55:30 +02:00
Jakob Unterwurzacher
34d8a498c4 Unbreak hyperlinks broken by go mod v2 conversion
Commit

  69d88505fd go mod: declare module version v2

translated all instances of "github.com/rfjakob/gocryptfs/" to
"github.com/rfjakob/gocryptfs/v2/".

Unfortunately, this included hyperlinks.

Unbreak the hyperlinks like this:

  find . -name \*.go | xargs sed -i s%https://github.com/rfjakob/gocryptfs/v2/%https://github.com/rfjakob/gocryptfs/v2/%
2021-08-30 11:31:01 +02:00
Jakob Unterwurzacher
69d88505fd go mod: declare module version v2
Our git version is v2+ for some time now, but go.mod
still declared v1. Hopefully making both match makes
https://pkg.go.dev/github.com/rfjakob/gocryptfs/v2 work.

All the import paths have been fixed like this:

  find . -name \*.go | xargs sed -i s%github.com/rfjakob/gocryptfs/%github.com/rfjakob/gocryptfs/v2/%
2021-08-23 15:05:15 +02:00
Jakob Unterwurzacher
195d9d18a9 Implement -deterministic-names: extended -zerodiriv
-deterministc-names uses all-zero dirivs but does not write
them to disk anymore.
2021-08-20 10:58:42 +02:00
Jose M Perez
8f94083a21 Flag -zerodiriv to create all diriv as all zero byte files 2021-08-19 18:05:54 +02:00
Jakob Unterwurzacher
05b813f202 nametransform: rename BadNameFlag to BadnameSuffix 2021-06-21 12:12:44 +02:00
Jakob Unterwurzacher
689b74835b nametransform: gather badname functions in badname.go 2021-06-21 12:10:04 +02:00
Jakob Unterwurzacher
2efef1e270 nametransform: delete NameTransformer interface
Useless layer of indirection.
2021-06-21 11:53:33 +02:00
Jakob Unterwurzacher
c5d8fa83ae nametransform: pass badname patterns via New
This means we can unexport the field.
2021-06-20 19:09:46 +02:00
DerDonut
a611810ff4 Badname file content access
This proposal is the counterpart of the modifications from the `-badname`
parameter. It modifies the plain -> cipher mapping for filenames when using
`-badname` parameter. The new function `EncryptAndHashBadName` tries to find a
cipher filename for the given plain name with the following steps:

1. If `badname` is disabled or direct mapping is successful: Map directly
(default and current behaviour)

2. If a file with badname flag has a valid cipher file, this is returned
(=File just ends with the badname flag)

3. If a file with a badname flag exists where only the badname flag was added,
this is returned (=File cipher name could not be decrypted by function
`DecryptName` and just the badname flag was added)

4. Search for all files which cipher file name extists when cropping more and
more characters from the end. If only 1 file is found, return this

5. Return an error otherwise

This allows file access in the file browsers but most important it allows that
you rename files with undecryptable cipher names in the plain directories.
Renaming those files will then generate a proper cipher filename One
backdraft: When mounting the cipher dir with -badname parameter, you can never
create (or rename to) files whose file name ends with the badname file flag
(at the moment this is " GOCRYPTFS_BAD_NAME"). This will cause an error.

I modified the CLI test function to cover additional test cases. Test [Case
7](https://github.com/DerDonut/gocryptfs/blob/badnamecontent/tests/cli/cli_test.go#L712)
cannot be performed since the cli tests are executed in panic mode. The
testing is stopped on error. Since the function`DecryptName` produces internal
errors when hitting non-decryptable file names, this test was omitted.

This implementation is a proposal where I tried to change the minimum amount
of existing code. Another possibility would be instead of creating the new
function `EncryptAndHashBadName` to modify the signature of the existing
function `EncryptAndHashName(name string, iv []byte)` to
`EncryptAndHashName(name string, iv []byte, dirfd int)` and integrate the
functionality into this function directly. You may allow calling with dirfd=-1
or other invalid values an then performing the current functionality.
2021-06-20 18:09:21 +02:00
Jakob Unterwurzacher
04858ddd22 nametransform: check name validity on encryption
xfstests generic/523 discovered that we allowed to set
xattrs with "/" in the name, but did not allow to read
them later.

With this change we do not allow to set them in the first
place.
2021-06-02 14:29:48 +02:00
Jakob Unterwurzacher
bed60101f4 nametransform: make gocryptfs.diriv and gocryptfs.xxx.name files world-readable
Make `gocryptfs.diriv` and `gocryptfs.xxx.name` files world-readable to make encrypted backups easier
when mounting via fstab.

Having the files follow chmod/chown of their parent does not seem
to be worth the hassle. The content of the diriv files is not
secret, and both diriv and name files are protected by the
perms of the parent dir.

Fixes https://github.com/rfjakob/gocryptfs/issues/539
2021-01-10 08:07:10 +01:00
Jakob Unterwurzacher
f3394ae286 nametransform: move permission constants to perms.go
Prep for https://github.com/rfjakob/gocryptfs/issues/539
2021-01-10 07:27:04 +01:00
Jakob Unterwurzacher
af4c1fb7a3 syscallcompat: retry ops on EINTR
Retry operations that have been shown to throw EINTR
errors on CIFS.

Todo: Solution for this pain in the back:

	warning: unix.Getdents returned errno 2 in the middle of data
	rm: cannot remove 'linux-3.0.old3/Documentation/ABI/removed': Input/output error

Progress towards fixing https://github.com/rfjakob/gocryptfs/issues/483 .
2020-10-14 00:35:16 +02:00
Jakob Unterwurzacher
84ed139cd2 v2api/reverse: implement Lookup for longname 2020-08-02 19:33:12 +02:00
Jakob Unterwurzacher
261cf12829 badname: stop trying to decrypt at length 22
At least 16 bytes due to AES --> at least 22 characters in base64
2020-06-06 12:49:45 +02:00
DerDonut
a8230d271f Added auto decryption of invalid file names
Changed invalid file decoding and decryption. Function
DecryptName now shortens the filename until the filename is
decodable and decryptable. Will work with valid **and**
invalid Base64URL delimiter (valid delimiter [0-9a-zA-z_\\-].
If the filename is not decryptable at all, it returns the
original cipher name with flag suffix Changed cli tests to
generate decryptable and undecryptable file names with correct
encrypted content. Replacing #474, extends #393
2020-06-06 12:20:31 +02:00
Jakob Unterwurzacher
ead7008a08 Fix spelling mistakes found by misspell
https://github.com/client9/misspell
2020-05-10 00:25:49 +02:00
Jakob Unterwurzacher
f6b1c680b3 nametransform: update comment & simplify tests
The comment still mentioned CBC, which has been removed
a long time ago.

The test definition can be rewritten using slice literals,
saving sume stuttering.
2020-04-18 16:14:48 +02:00
orcas
9ec042f2f6 Show undecryptable filenames if they match supplied glob
Resolves https://github.com/rfjakob/gocryptfs/issues/393
2020-02-28 22:17:59 +01:00
Jakob Unterwurzacher
ec17445b99 forward mode: create gocryptfs.diriv files with 0440 permissions
Makes it easier to share an encrypted folder via a network drive.

https://github.com/rfjakob/gocryptfs/issues/387
2019-03-30 20:06:40 +01:00
Eduardo M KALINOWSKI
3bc100aeb3 reverse mode: support wildcard exclude (--exclude-wildcard)
This adds support for gitignore-like wildcards and exclude patters in
reverse mode. It (somewhat) fixes #273: no regexp support, but the
syntax should be powerful enough to satisfy most needs.

Also, since adding a lot of --exclude options can be tedious, it adds
the --exclude-from option to read patterns from a file (or files).
2019-03-26 20:56:37 +01:00
Jakob Unterwurzacher
19cb6d046a nametransform: reject names longer than 255 chars
Looks like we allowed creating longer names by accident.
Fix that, and add a test that verifies it.
2019-02-17 17:05:05 +01:00
Sebastian Lackner
aae45b4d77 nametransform: Create *.name files with 0400 permission.
Similar to gocryptfs.iv files they are never modified.
2019-01-09 20:42:18 +01:00
Jakob Unterwurzacher
8074f12beb nametransform: ReadDirIVAt: return raw syscall error
Otherwise this can happen, as triggered by xfstests generic/011:

  go-fuse: can't convert error type: openat failed: too many open files

The app then gets a misleading "Function not implemented" error.
2019-01-04 23:21:27 +01:00
Sebastian Lackner
ab169443fd A few more spelling fixes.
Found with the 'codespell' utility.
2019-01-04 01:23:44 +01:00
Sebastian Lackner
61241b0588 nametransform: Add implicit syscall.O_RDONLY flag. 2019-01-03 18:24:05 +01:00
Jakob Unterwurzacher
4f66d66755 fusefrontend: add dirCache 2019-01-03 15:31:13 +01:00
Jakob Unterwurzacher
f6dad8d0fa nametransform: simplify WriteDirIV to WriteDirIVAt
Un-spaghettify the function and let the callers open
the directory.
2019-01-03 15:31:13 +01:00
Jakob Unterwurzacher
2b12bba274 fusefronted: make EncryptPath symlink-safe
Finally allows us to delete EncryptPathDirIV.
2019-01-02 21:45:40 +01:00
Jakob Unterwurzacher
d99a0480f7 nametransform: fix possible incomplete read in ReadLongNameAt
Pread() needs retry logic, so instead of implementing it ourselves,
use os.File.

Reported by @slackner at
c09bf1f228 (r31813394)
2019-01-02 00:09:17 +01:00
Jakob Unterwurzacher
2de3851abd nametransform: rename WriteLongName() -> WriteLongNameAt()
And also rename DeleteLongName() -> DeleteLongNameAt(). The
naming follow the names open the openat() etc syscalls.
2019-01-01 16:24:25 +01:00
Jakob Unterwurzacher
21f1f858b9 fusefrontend: make OpenDir() symlink-safe
Interestingly, little or no performance impact:

$ ./benchmark.bash
Testing gocryptfs at /tmp/benchmark.bash.39W: gocryptfs v1.6-42-g30c2349-dirty; go-fuse v20170619-66-g6df8ddc; 2018-11-04 go1.11
Downloading linux-3.0.tar.gz
/tmp/linux-3.0.tar.gz                  100%[=========================================================================>]  92.20M  2.93MB/s    in 31s
2018-11-04 21:44:44 URL:https://cdn.kernel.org/pub/linux/kernel/v3.0/linux-3.0.tar.gz [96675825/96675825] -> "/tmp/linux-3.0.tar.gz" [1]
WRITE: 262144000 bytes (262 MB, 250 MiB) copied, 1.1808 s, 222 MB/s
READ:  262144000 bytes (262 MB, 250 MiB) copied, 0.866438 s, 303 MB/s
UNTAR: 24.745
MD5:   12.050
LS:    3.525
RM:    9.544

Note: kernel has been updated:

$ uname -a
Linux brikett 4.18.16-200.fc28.x86_64 #1 SMP Sat Oct 20 23:53:47 UTC 2018 x86_64 x86_64 x86_64 GNU/Linux
2019-01-01 16:24:25 +01:00
Jakob Unterwurzacher
de3a2c1895 fusefrontend: mark a few more functions as symlink-safe / unsafe 2019-01-01 16:24:25 +01:00
Jakob Unterwurzacher
932efbd459 fusefrontend: make openBackingDir() symlink-safe
openBackingDir() used encryptPath(), which is not symlink-safe
itself. Drop encryptPath() and implement our own directory walk.

Adds three seconds to untar and two seconds to rm:

$ ./benchmark.bash
Testing gocryptfs at /tmp/benchmark.bash.MzG: gocryptfs v1.6-36-g8fb3c2f-dirty; go-fuse v20170619-66-g6df8ddc; 2018-10-14 go1.11
WRITE: 262144000 bytes (262 MB, 250 MiB) copied, 1.25078 s, 210 MB/s
READ:  262144000 bytes (262 MB, 250 MiB) copied, 1.0318 s, 254 MB/s
UNTAR: 20.941
MD5:   11.568
LS:    1.638
RM:    5.337
2019-01-01 16:24:09 +01:00
Jakob Unterwurzacher
c09bf1f228 fusefrontend: make DecryptPath() symlink-safe
DecryptPath is now symlink-safe through the use of *at()
functions.
2019-01-01 16:24:09 +01:00
Jakob Unterwurzacher
545a03da24 nametransform: comments: directly link to ioutil.WriteFile fix
So the reader does not have to read through the whole ticket.
The commit message has a nice summary of the problem.
2019-01-01 16:23:28 +01:00
Sebastian Lackner
87ced5f95d nametransform: Delete incomplete longname files on error. 2019-01-01 16:09:57 +01:00
Sebastian Lackner
874eaf9734 Assorted spelling fixes.
Mostly detected with the 'codespell' utility, but also includes some
manual grammar fixes.
2018-12-27 15:19:55 +01:00
Jakob Unterwurzacher
c270b21efc fusefrontend: get rid of os.File* wrapping
Directly use int file descriptors for the dirfd
and get rid of one level of indirection.
2018-09-23 12:17:26 +02:00
Jakob Unterwurzacher
c70df522d2 fusefrontend: doWrite: delete file header if first write fails
xfstests generic/083 fills the filesystem almost completely while
running fsstress in parallel. In fsck, these would show up:

  readFileID 2580: incomplete file, got 18 instead of 19 bytes

This could happen when writing the file header works, but writing
the actual data fails.

Now we kill the header again by truncating the file to zero.
2018-07-15 15:12:55 +02:00
Jakob Unterwurzacher
bbf5b72fff WriteDirIV: delete incomplete gocryptfs.diriv file if write fails
If the underlying filesystem is full, writing to gocryptfs.diriv may
fail, and later fsck show this:

	OpenDir "xyz": could not read gocryptfs.diriv: wanted 16 bytes, got 0

Uncovered by xfstests generic/083.

Also fixes a fd leak in the error path.
2018-07-15 12:02:39 +02:00
Jakob Unterwurzacher
b6c8960b01 fsck: clean up log output
Make sure we get only 1 warning output per
problem.

Also, add new corruption types to broken_fs_v1.4.
2018-04-02 18:32:30 +02:00
Jakob Unterwurzacher
9f8d0d8e57 gccgo: replace syscall.NAME_MAX with unix.NAME_MAX
For some reason the syscall.NAME_MAX constant does not exist
on gccgo, and it does not hurt us to use unix.NAME_MAX instead.

https://github.com/rfjakob/gocryptfs/issues/201
2018-02-01 23:50:11 +01:00
Sebastian Lackner
9bcde0c09e fusefrontend: Improve documentation of mkdirWithIv and WriteDirIV
As requested in https://github.com/rfjakob/gocryptfs/pull/179
2017-12-01 09:41:52 +01:00
Jakob Unterwurzacher
e97c23e083 syscallcompat: check that we get NOFOLLOW wherever possible
...and fix the instances where the AT_SYMLINK_NOFOLLOW /
O_NOFOLLOW / O_EXCL flag was missing.
2017-11-30 19:40:53 +01:00
Sebastian Lackner
614745ee57 fusefrontend: allow_other: close race between mkdir and chown
Fixes the same problem as described in 72b975867a,
except for directories instead of device nodes.
2017-11-29 13:28:04 +01:00
Sebastian Lackner
0f44c617d0 syscallcompat: Introduce unlinkat syscall with flags argument 2017-11-29 12:41:23 +01:00
Sebastian Lackner
9f56b33e0c fusefrontend: Fix longname handling for renames with existing target
Fixes https://github.com/rfjakob/gocryptfs/issues/170

Steps to reproduce the problem:

* Create a regular forward mount point
* Create a file with a shortname and one with a long filename
* Try to run 'mv <shortname> <longname>'

This should actually work and replace the existing file, but instead it
fails with:

    mv: cannot move '<shortname>' to '<longname>': File exists

The problem is the creation of the .name file. If the target already exists
we can safely ignore the EEXIST error and just keep the existing .name file.
2017-11-25 16:19:09 +01:00