hardcoresushi/libgocryptfs
1
0
Fork 0
Code Issues Pull Requests Releases Activity
1,463 Commits 2 Branches 0 Tags 6 MiB
d361f6e35b
Commit Graph

1463 Commits

This Branch
This Branch
All Branches
Author SHA1 Message Date
Jakob Unterwurzacher
a1fb456618 fusefrontend: make Rename() symlink-safe 
Use Openat() and the openBackingDir() helper so we
never follow symlinks.
 2018-09-23 12:17:59 +02:00
Jakob Unterwurzacher
897bb8924f fusefrontend: make Create() symlink-safe 
Use Openat() and the openBackingDir() helper so we
never follow symlinks.
 2018-09-23 12:17:59 +02:00
Jakob Unterwurzacher
63762b33af fusefrontend: Open(): fix dirfd leak 
Close was missing.
 2018-09-23 12:17:59 +02:00
Jakob Unterwurzacher
bead82c9fb fusefrontend: add named parameters to openBackingDir 
Named parameters make using the function easier.
 2018-09-23 12:17:59 +02:00
Jakob Unterwurzacher
c270b21efc fusefrontend: get rid of os.File* wrapping 
Directly use int file descriptors for the dirfd
and get rid of one level of indirection.
 2018-09-23 12:17:26 +02:00
Jakob Unterwurzacher
22fba4ac3e fusefrontent: make Open() symlink-safe 2018-09-23 12:17:26 +02:00
Jakob Unterwurzacher
05c8d4a1c4 tests: add symlink_race tool 
Help uncover symlink races.
 2018-09-23 12:17:26 +02:00
Jakob Unterwurzacher
2d01d5f2d4 tlog: always trim trailing newlines 
The messages we print through tlog sometimes do, sometimes do
not contain a trailing newline. The stdlib logger usually drops
trailing newlines automatically, but tlog postfixes ColorReset to
the caller's message, so the logger logic does not work when we
print colored output.

Drop the newlines on our own, and add a test.

Fixes the blank lines in fsck output:

~/go/src/github.com/rfjakob/gocryptfs/tests/fsck$ ./run_fsck.bash
Reading password from extpass program
Decrypting master key
OpenDir "": invalid entry "invalid_file_name.3": illegal base64 data at input byte 17
OpenDir "": invalid entry "invalid_file_name_2": bad message
fsck: corrupt entry in dir "": "invalid_file_name.3"
fsck: corrupt entry in dir "": "invalid_file_name_2"
OpenDir "": invalid entry "invalid_file_name____1": bad message
fsck: corrupt entry in dir "": "invalid_file_name____1"
doRead 4327225: corrupt block #0: stupidgcm: message authentication failed
fsck: error reading file "corrupt_file" (inum 4327225): 5=input/output error
cipherSize 40 < overhead 50: corrupt file

doRead 4327074: corrupt header: ParseHeader: invalid version, want=2 have=22616
cipherSize 40 < overhead 50: corrupt file

fsck: error reading file "corrupt_file_2" (inum 4327074): 5=input/output error
Readlink "s-P7PcQDUcVkoeMDnC3EYA": decrypting target failed: stupidgcm: message authentication failed
fsck: error reading symlink "corrupt_symlink": 5=input/output error
Readlink "iI0MtUdzELPeOAZYwYZFee169hpGgd3l2PXQBcc9sl4": decrypting target failed: illegal base64 data at input byte 0
fsck: error reading symlink "corrupt_symlink_2": 5=input/output error
OpenDir "yrwcjj2qoC4IYvhw9sbfRg": could not read gocryptfs.diriv: wanted 16 bytes, got 17
fsck: error opening dir "diriv_too_long": 5=input/output error
OpenDir "trqecbMNXdzLqzpk7fSfKw": could not read gocryptfs.diriv: wanted 16 bytes, got 3
fsck: error opening dir "diriv_too_short": 5=input/output error
cipherSize 8 < header size 18: corrupt file

readFileID 4327049: incomplete file, got 8 instead of 19 bytes
fsck: corrupt file "incomplete_file_1" (inode 4327049)
readFileID 4327038: incomplete file, got 18 instead of 19 bytes
fsck: corrupt file "incomplete_file_2" (inode 4327038)
cipherSize 1 < header size 18: corrupt file

readFileID 4327063: incomplete file, got 1 instead of 19 bytes
fsck: corrupt file "incomplete_file_3" (inode 4327063)
fsck: error opening dir "missing_diriv": 2=no such file or directory
ListXAttr: invalid xattr name "user.gocryptfs.0a5e7yWl0SGUGeWB0Sy2K0": bad message
fsck: corrupt xattr name on file "xattr_corrupt_name": "user.gocryptfs.0a5e7yWl0SGUGeWB0Sy2K0"
GetXAttr: stupidgcm: message authentication failed
fsck: error reading xattr "user.foo" from "xattr_corrupt_value": 5=input/output error
fsck summary: 15 corrupt files
 2018-09-23 11:28:49 +02:00
Jakob Unterwurzacher
737a2f2012 syscallcompat: untangle Openat flag check 
Check for O_NWFOLLOW and O_EXCL separately to
make the logic clearer.
 2018-09-22 19:38:47 +02:00
Jakob Unterwurzacher
9e6ee47bc9 tests: detect and report chmod failures earlier 
Instead of reporting the consequence:

    matrix_test.go:906: modeHave 0664 != modeWant 0777

Report it if chmod itself fails, and also report the old file mode:

    matrix_test.go:901: chmod 000 -> 777 failed: bad file descriptor
 2018-09-22 13:39:17 +02:00
Jakob Unterwurzacher
5ca6243eeb main: sanitize commas from fsname 
The cipherdir path is used as the fsname, as displayed
in "df -T". Now, having a comma in fsname triggers a sanity check
in go-fuse, aborting the mount with:

  /bin/fusermount: mount failed: Invalid argument
  fuse.NewServer failed: fusermount exited with code 256

Sanitize fsname by replacing any commas with underscores.

https://github.com/rfjakob/gocryptfs/issues/262
 2018-09-18 19:39:52 +02:00
Jakob Unterwurzacher
e8d8ae54d3 fusefrontend: use OpenDirNofollow in openBackingDir 
Rename openBackingPath to openBackingDir and use OpenDirNofollow
to be safe against symlink races. Note that openBackingDir is
not used in several important code paths like Create().

But it is used in Unlink, and the performance impact in the RM benchmark
to be acceptable:

Before

	$ ./benchmark.bash
	Testing gocryptfs at /tmp/benchmark.bash.bYO: gocryptfs v1.6-12-g930c37e-dirty; go-fuse v20170619-49-gb11e293; 2018-09-08 go1.10.3
	WRITE: 262144000 bytes (262 MB, 250 MiB) copied, 1.07979 s, 243 MB/s
	READ:  262144000 bytes (262 MB, 250 MiB) copied, 0.882413 s, 297 MB/s
	UNTAR: 16.703
	MD5:   7.606
	LS:    1.349
	RM:    3.237

After

	$ ./benchmark.bash
	Testing gocryptfs at /tmp/benchmark.bash.jK3: gocryptfs v1.6-13-g84d6faf-dirty; go-fuse v20170619-49-gb11e293; 2018-09-08 go1.10.3
	WRITE: 262144000 bytes (262 MB, 250 MiB) copied, 1.06261 s, 247 MB/s
	READ:  262144000 bytes (262 MB, 250 MiB) copied, 0.947228 s, 277 MB/s
	UNTAR: 17.197
	MD5:   7.540
	LS:    1.364
	RM:    3.410
 2018-09-08 19:27:33 +02:00
Jakob Unterwurzacher
84d6fafeca tests: restore Go 1.7 compatibility 
Go 1.7 does not have t.Name() yet.
 2018-09-08 18:19:53 +02:00
Jakob Unterwurzacher
930c37e03d syscallcompat: use O_PATH in OpenDirNofollow 
This fixes the "0100 directory" problem in reverse mode,
and should be slightly faster.
 2018-09-08 18:06:33 +02:00
Jakob Unterwurzacher
9ec9d0c49c syscallcompat: untangle OpenNofollow and rename to OpenDirNofollow 
The function used to do two things:

1) Walk the directory tree in a manner safe from symlink attacks
2) Open the final component in the mode requested by the caller

This change drops (2), which was only used once, and lets the caller
handle it. This simplifies the function and makes it fit for reuse in
forward mode in openBackingPath(), and for using O_PATH on Linux.
 2018-09-08 17:41:17 +02:00
Jakob Unterwurzacher
bc14f8dcb6 tests: add chmod test 
Makes sure we don't add regressions when fixing
https://github.com/rfjakob/gocryptfs/issues/259
 2018-09-08 17:00:23 +02:00
Jakob Unterwurzacher
21b5fae0e6 tests: restore TestUtimesNanoSymlink 
The function seems to have been renamed by 426b9536 by mistake.
Rename it back so the test is run again.
 2018-09-08 16:36:01 +02:00
Jakob Unterwurzacher
ca24c20694 main: don't read the config file twice (fix pipe bug) 
Instead, first Load() the file, then DecryptMasterKey().

Fixes https://github.com/rfjakob/gocryptfs/issues/258
 2018-09-08 13:04:33 +02:00
Jakob Unterwurzacher
2bdf7d5172 configfile: add LoadAndDecrypt wrapper 
Callers that do not want to decrypt the masterkey should
call plain Load().

https://github.com/rfjakob/gocryptfs/issues/258
 2018-09-08 12:40:29 +02:00
Jakob Unterwurzacher
09d28c293e configfile: split off masterkey decryption 
Preparation for fixing https://github.com/rfjakob/gocryptfs/issues/258
 2018-09-08 12:19:19 +02:00
Jakob Unterwurzacher
21eaa8f164 configfile: return specific error on empty input 
Report the actual problem instead of a generic
"unexpected end of JSON input".

https://github.com/rfjakob/gocryptfs/issues/258
 2018-09-08 12:18:26 +02:00
Jakob Unterwurzacher
22906cc5fa travis: add Go 1.11 
Released two days ago
 2018-08-26 13:04:18 +02:00
Jakob Unterwurzacher
658cc4aebb syscallcompat: drop Fchmodat flags 
These were silently ignored until now (!) but
are rejected by Go 1.11 stdlib.

Drop the flags so the tests work again, until
we figure out a better solution.

https://github.com/golang/go/issues/20130
 2018-08-26 13:04:01 +02:00
Jakob Unterwurzacher
91dc44c8b4 travis: show fs type for /tmp 2018-08-26 12:39:51 +02:00
Jakob Unterwurzacher
39f39cd63c README: update benchmark output 2018-08-18 18:38:14 +02:00
Jakob Unterwurzacher
21f32b1ba7 Update reverse benchmarks 2018-08-18 12:25:03 +02:00
Jakob Unterwurzacher
30ba8140f2 Update README for v1.6 release 2018-08-18 12:19:47 +02:00
Jakob Unterwurzacher
f2a0c6566e README: list trezor support in changelog 2018-08-15 23:44:46 +02:00
Jakob Unterwurzacher
7e111ea94c trezor: document -trezor in man page 
Also fix "--", which was shown as just "-" in the man pager.
 2018-08-15 23:34:23 +02:00
Jakob Unterwurzacher
bd054e70ef trezor: show support in version string 
Show enable_trezor in the version string if we were compiled
with `-tags enable_trezor`. And hide the `-trezor` flag from
the help output if we were not.
 2018-08-15 23:31:37 +02:00
Jakob Unterwurzacher
7771a33f65 dep: update dependencies 2018-08-15 23:24:03 +02:00
Jakob Unterwurzacher
182f4833ee build.bash: handle args containing spaces 
Fixes ./build.bash -tags "enable_trezor without_openssl"
 2018-08-15 23:17:44 +02:00
Jakob Unterwurzacher
dbd400d930 fusefrontend: truncateGrowFile: pass zeroPad error to caller 
Errors from zeroPad were ignored until now, as discovered
using xfstests generic/083.
 2018-08-15 17:25:22 +02:00
Jakob Unterwurzacher
0e1cbb75fe fsck: skip already-seen hard-linked files 
We may hit files several times due to hard links.
Only check the contents once.
 2018-08-15 14:02:05 +02:00
Jakob Unterwurzacher
db4accf3a3 fsck: don't print corse corruption offset 
The exact ciphertext block number (4KiB granularity) is
already printed in the doRead message. Don't cause
confusion by printing the 128KiB-granularity offset as
well.

  doRead 767: corrupt block #4: stupidgcm: message authentication failed
  fsck: error reading file "pa/d7/d14/f10c" (inum 767): 5=input/output error
 2018-08-15 13:20:40 +02:00
Jakob Unterwurzacher
5acfbc1b2f main: add -e as an alias for -exclude 2018-08-15 13:12:13 +02:00
Jakob Unterwurzacher
7a02f71fc2 fusefrontend_reverse: reject excludes for the root directory "" 
This is most likely a mistake by the user. Reject it.
 2018-08-15 12:28:29 +02:00
Jakob Unterwurzacher
8989905333 Update changelog 2018-08-11 23:39:49 +02:00
Jakob Unterwurzacher
ec2fdc19cf reverse mode: add --exclude option 
https://github.com/rfjakob/gocryptfs/issues/235
 2018-08-11 23:26:49 +02:00
Jakob Unterwurzacher
eaa5aecd42 cli: add multipleStrings type 
Will be used for --exclude.
 2018-08-11 23:25:17 +02:00
Jakob Unterwurzacher
06f1ea951b tests: VerifyExistence: panic on inconsistent results 
In the past we did not check whether all methods of checking
the file return the same result. We should.
 2018-08-11 22:37:22 +02:00
Jakob Unterwurzacher
f48b731e76 travis ci: drop go 1.5 and go 1.6 
Trying to build with these versions now throws this error:

  # golang.org/x/sys/unix
  ../../../golang.org/x/sys/unix/ioctl.go:18: undefined: runtime.KeepAlive

It looks like x/sys/unix has dropped support for older Go
versions.
 2018-08-02 20:06:34 +02:00
Jakob Unterwurzacher
d620054caf fsck: print inode number on file read error 
This makes it possible to find the file without mounting the fs.
 2018-07-23 22:34:41 +02:00
Jakob Unterwurzacher
f4a972ddf1 configfile: drop superflous Printf 
Before:

  $ gocryptfs -fsck .
  LoadConfFile: ReadFile: &os.PathError{Op:"open", Path:"/var/tmp/check-gocryptfs/scratchdev/gocryptfs.conf", Err:0xd}
  Cannot open config file: open /var/tmp/check-gocryptfs/scratchdev/gocryptfs.conf: permission denied

After:

  $ gocryptfs -fsck .
  Cannot open config file: open /var/tmp/check-gocryptfs/scratchdev/gocryptfs.conf: permission denied
 2018-07-23 22:25:40 +02:00
Jakob Unterwurzacher
f316f1b2df fusefronted: disallow writes running concurrently with reads 
As uncovered by xfstests generic/465, concurrent reads and writes
could lead to this,

  doRead 3015532: corrupt block #1039: stupidgcm: message authentication failed,

as the read could pick up a block that has not yet been completely written -
write() is not atomic!

Now writes take ContentLock exclusively, while reads take it shared,
meaning that multiple reads can run in parallel with each other, but
not with a write.

This also simplifies the file header locking.
 2018-07-22 22:29:22 +02:00
Jakob Unterwurzacher
c70df522d2 fusefrontend: doWrite: delete file header if first write fails 
xfstests generic/083 fills the filesystem almost completely while
running fsstress in parallel. In fsck, these would show up:

  readFileID 2580: incomplete file, got 18 instead of 19 bytes

This could happen when writing the file header works, but writing
the actual data fails.

Now we kill the header again by truncating the file to zero.
 2018-07-15 15:12:55 +02:00
Jakob Unterwurzacher
55bb22bad6 fusefrontend: doWrite: no need to take HeaderLock.RLock() 
Other writers are blocked by ContentLock already.
 2018-07-15 12:40:23 +02:00
Jakob Unterwurzacher
bbf5b72fff WriteDirIV: delete incomplete gocryptfs.diriv file if write fails 
If the underlying filesystem is full, writing to gocryptfs.diriv may
fail, and later fsck show this:

	OpenDir "xyz": could not read gocryptfs.diriv: wanted 16 bytes, got 0

Uncovered by xfstests generic/083.

Also fixes a fd leak in the error path.
 2018-07-15 12:02:39 +02:00
Jakob Unterwurzacher
bcca323cb7 contentenc: reserve one extra block in pool plaintext buffers 
File holes and -fsck can cause unaligned read accesses, which means
we have to decrypt one extra plaintext block.

xfstests generic/083 manage to crash -fsck like this:

generic/083	2018/07/14 15:25:21 wrong len=266240, want=131072
panic: wrong len=266240, want=131072

goroutine 1 [running]:
log.Panicf(0x67fc00, 0x15, 0xc4204fec90, 0x2, 0x2)
	/usr/local/go/src/log/log.go:333 +0xda
github.com/rfjakob/gocryptfs/internal/contentenc.(*bPool).Put(0xc4200d4800, 0xc4202f2000, 0x21000, 0x41000)
	/home/jakob/go/src/github.com/rfjakob/gocryptfs/internal/contentenc/bpool.go:27 +0x15d
github.com/rfjakob/gocryptfs/internal/fusefrontend.(*File).doRead(0xc4200b4500, 0xc42019e000, 0x0, 0x20000, 0x28400, 0x20000, 0xc42019e000, 0xc4204ff008, 0x435164, 0xc420000180)
	/home/jakob/go/src/github.com/rfjakob/gocryptfs/internal/fusefrontend/file.go:227 +0xba9
github.com/rfjakob/gocryptfs/internal/fusefrontend.(*File).Read(0xc4200b4500, 0xc42019e000, 0x20000, 0x20000, 0x28400, 0x0, 0x0, 0x0)
	/home/jakob/go/src/github.com/rfjakob/gocryptfs/internal/fusefrontend/file.go:246 +0x23e
main.(*fsckObj).file(0xc420069320, 0xc42001a630, 0x21)
	/home/jakob/go/src/github.com/rfjakob/gocryptfs/fsck.go:126 +0x21f
main.(*fsckObj).dir(0xc420069320, 0xc420014dc0, 0x1d)
	/home/jakob/go/src/github.com/rfjakob/gocryptfs/fsck.go:76 +0x387
main.(*fsckObj).dir(0xc420069320, 0xc42021dae0, 0x19)
	/home/jakob/go/src/github.com/rfjakob/gocryptfs/fsck.go:74 +0x347
 2018-07-15 11:39:19 +02:00
Jakob Unterwurzacher
38f79a1abc fsck: add debug output (enabled via -debug) 
Turn the commented-out fmt.Printf into debug output
via the tlog infrastructure.
 2018-07-15 11:34:30 +02:00