Commit Graph

870 Commits

Author SHA1 Message Date
Jakob Unterwurzacher 7b22b426b9 contentenc: rename PlaintextRange and CiphertextRange
The name could be misunderstood and actually caused a bug:
doWrite used to always preallocate 4128 instead of the actual
data length.
2016-07-02 00:12:36 +02:00
Jakob Unterwurzacher f2b4d57068 fusefrontend: coalesce grows in Truncate()
We were growing the file block-by-block which was pretty
inefficient. We now coalesce all the grows into a single
Ftruncate. Also simplifies the code!

Simplistic benchmark: Before:

  $ time truncate -s 1000M foo
  real	0m0.568s

After:

  $ time truncate -s 1000M foo
  real	0m0.205s
2016-07-01 23:32:27 +02:00
Jakob Unterwurzacher ae77d18527 fusefrontend: better comments for Truncate 2016-07-01 09:23:04 +02:00
Jakob Unterwurzacher e994ffa27d fusefrontend: handle dir-overwrites-dir on XFS
XFS returns a different error code if you try to overwrite
a non-empty directory with a directory:

XFS:  mv: cannot move ‘foo’ to ‘bar/foo’: File exists
ext4: mv: cannot move 'foo' to 'bar/foo': Directory not empty

So have EEXIST trigger the Rmdir logic as well.

Fixes issue #20
Link: https://github.com/rfjakob/gocryptfs/issues/20
2016-06-30 23:17:54 +02:00
Jakob Unterwurzacher c400aca5cf tlog: clean up messages from the go-fuse library
Drop the date and add the "go-fuse: " prefix so you can see
where the message is coming from.

Before:
  Jun 27 09:03:15 brikett gocryptfs[4150]: 2016/06/27 09:03:15 Unimplemented opcode INTERRUPT
After:
  Jun 27 09:10:58 brikett gocryptfs[4961]: go-fuse: Unimplemented opcode INTERRUPT
2016-06-27 09:12:03 +02:00
Jakob Unterwurzacher 3288d98703 fusefrontend: fix PlaintextNames versions of Mkdir, Rmdir
The "!fs.args.DirIV" special case was removed by b17f0465c7
but that, by accident, also removed the handling for
PlaintextNames.

Re-add it as an explicit PlaintextNames special case.

Also adds support for removing directories that miss their
gocryptfs.diriv file for some reason.
2016-06-27 00:27:36 +02:00
Jakob Unterwurzacher 547ddf4264 tlog: switch default logger to syslog
...unless "-nosyslog" is passed.

All gocryptfs messages already go to syslog, but the messages
that the go-fuse lib emits were still printed to stdout.

Fixes issue #13 ( https://github.com/rfjakob/gocryptfs/issues/13 )
2016-06-26 23:30:22 +02:00
Jakob Unterwurzacher 0115588680 main, fusefrontend: enable suid functionality
FUSE filesystems are mounted with "nosuid" by default. If we run as root,
we can use device files by passing the opposite mount option, "suid".

Also we have to use syscall.Chmod instead of os.Chmod because the
portability translation layer "syscallMode" messes up the sgid
and suid bits.

Fixes 70% of the failures in xfstests generic/193. The remaining are
related to truncate, but we err on the safe side:

    $ diff -u tests/generic/193.out /home/jakob/src/fuse-xfstests/results//generic/193.out.bad
    [...]
     check that suid/sgid bits are cleared after successful truncate...
     with no exec perm
     before: -rwSr-Sr--
    -after:  -rw-r-Sr--
    +after:  -rw-r--r--
2016-06-26 20:13:21 +02:00
Jakob Unterwurzacher 23cc0657f4 fusefronted: preserve owner if running as root
If allow_other is set and we run as root, try to give newly created files to
the right user.
2016-06-26 19:18:13 +02:00
Jakob Unterwurzacher 38767ab527 fuserfrontend: support truncate(2) by wrapping ftruncate(2)
Support truncate(2) by opening the file and calling ftruncate(2)
While the glibc "truncate" wrapper seems to always use ftruncate, fsstress from
xfstests uses this a lot by calling "truncate64" directly.
2016-06-26 18:41:04 +02:00
Jakob Unterwurzacher b558901e66 Drop deprecated "-gcmiv128" option
The GCMIV128 feature flag is already mandatory, dropping the command
line option is the final step.

Completes https://github.com/rfjakob/gocryptfs/issues/29 .
2016-06-23 22:10:19 +02:00
Jakob Unterwurzacher 3d59a72ba9 Drop deprecated "-emenames" option
The EMENames feature flag is already mandatory, dropping the command
line option is the final step.
2016-06-23 21:56:50 +02:00
Jakob Unterwurzacher e970b1fdb5 nametransform: drop unused noiv functions
As DirIV is now mandatory there is no user for the noiv functions.
2016-06-23 21:39:04 +02:00
Jakob Unterwurzacher b17f0465c7 Drop deprecated "-diriv" option
The DirIV feature flag is already mandatory, dropping the command
line option is the final step.
2016-06-23 21:38:59 +02:00
Jakob Unterwurzacher f0b4d2354d Refuse mounting of v0.6 and older filesystems 2016-06-19 20:01:04 +02:00
Jakob Unterwurzacher cc2a75b050 Mount v0.6 and older filesystems as read-only
This is part of the phase-out of very old filesystems.

See https://github.com/rfjakob/gocryptfs/wiki/Compatibility for
more info.
2016-06-16 23:23:09 +02:00
Jakob Unterwurzacher 3234b9b5ce tests: update config_test example files
Recreate the files so they carry all feature flags.

Also, create them with "-scryptn 10" to speed up the tests.
2016-06-16 23:19:05 +02:00
Jakob Unterwurzacher 96750a7d3c tests: exit with correct error code from TestMain
extpass_test and example_filesystems_test did it wrong,
always returning 0.
2016-06-16 21:56:23 +02:00
Jakob Unterwurzacher c76c952c19 readpassword: clean up leftover debug commit 2016-06-16 20:59:51 +02:00
Jakob Unterwurzacher 7e92ebe16a Rename nametransform, contentenc source files
Let's have shorter names, and merge *_api.go into the "main"
file.

No code changes.
2016-06-16 19:02:47 +02:00
Jakob Unterwurzacher 6c3f97399a Rename internal "toggledlog" package to "tlog"
tlog is used heavily everywhere and deserves a shorter name.

Renamed using sed magic, without any manual rework:

   find * -type f -exec sed -i 's/toggledlog/tlog/g' {} +
2016-06-15 23:30:44 +02:00
Jakob Unterwurzacher 09e88f31d1 toggledlog: assume command of color handling
We want to use colored error messages also outside of main,
so let's handle it in the logging package.

The fatal logger now automatically prints red.
2016-06-15 23:17:51 +02:00
Jakob Unterwurzacher c89455063c readpassword: create internal package for password reading
* Supports stdin
* Add tests for extpass and stdin

As per user request at https://github.com/rfjakob/gocryptfs/issues/30
2016-06-15 22:44:24 +02:00
Jakob Unterwurzacher 393e531afd Fix warnings reported by Go 1.6 "go tool vet -shadow=true"
Warnings were:

  main.go:234: declaration of err shadows declaration at main.go:163:
  internal/fusefrontend/file.go:401: declaration of err shadows declaration at internal/fusefrontend/file.go:379:
  internal/fusefrontend/file.go:419: declaration of err shadows declaration at internal/fusefrontend/file.go:379:
  internal/fusefrontend/fs_dir.go:140: declaration of err shadows declaration at internal/fusefrontend/fs_dir.go:97:
2016-06-14 22:46:23 +02:00
Jakob Unterwurzacher 5b1eed35ee fusefrontend: Utimens: convert ENOENT to EBADF
If /proc/self/fd/X did not exist, the actual error is that the file
descriptor was invalid.

go-fuse's pathfs prefers using an open fd even for path-based operations
but does not take any locks to prevent the fd from being closed.
Instead, it retries the operation by path if it get EBADF. So this
change allows the retry logic to work correctly.

This fixes the error

    rsync: failed to set times on "/tmp/ping.Kgw.mnt/linux-3.0/[...]/.dvb_demux.c.N7YlEM":
    No such file or directory (2)

that was triggered by pingpong-rsync.bash.
2016-06-09 22:21:00 +02:00
Jakob Unterwurzacher 5da292828c fusefrontend: fix chown on dangling symlinks
We (actually, go-fuse) used to call Chown() instead of Lchown()
which meant that the operation would fail on dangling symlinks.

Fix this by calling os.Lchown() ourself. Also add a test case
for this.
2016-06-08 00:32:44 +02:00
Jakob Unterwurzacher 0f4d350136 configfile: warn about missing feature flags
The plan is to drop support for the oldest filesystem versions
in gocryptfs v1.0. For now, we only warn the user.
2016-06-06 23:13:10 +02:00
Jakob Unterwurzacher 0c80cca674 toggledlog: convert remaing naked fmt.Print*
Several fatal errors were just printed to stdout, which
meant they were invisible when running the test suite.

Fix this by introducing toggledlog.Fatal and convert as
follows:

Fatal errors     -> toggledlog.Fatal
Warnings         -> toggledlog.Warn
Password prompts -> fmt.Fprintf
2016-06-05 14:32:07 +02:00
Jakob Unterwurzacher ca54b665e3 main: print actual error from LoadConfFile()
It may not have been a "Wrong password" after all.

Also, push down disabling the warning so LoadConfFile() can
warn about things that matter.
2016-06-05 13:44:22 +02:00
Jakob Unterwurzacher f2d208c464 configfile: use map[flagIota] for feature flags
This should make things saner and more extensible. It prepares
the infrastructure for "required feature flags" that will be used
to deprecate old gocryptfs version.
2016-06-05 12:54:45 +02:00
Jakob Unterwurzacher b97268c948 configfile: bake the "Creator" gocryptfs version into the file
This field is added for the convenience of users and
may help them to identify which gocryptfs version
they need to mount a filesystem.

The same information is essentially contained in FeatureFlags,
but this is more difficult to decode for humans.

It is completely ignored programmatically (also by older gocryptfs
versions).
2016-06-05 11:40:13 +02:00
Jakob Unterwurzacher a602e798b1 fusefrontend: report an error if all files in a directory were invalid
Just presenting an empty directory means that the user does not know
that things went wrong unless he checks the syslog or tries to delete
the directory.

It would be nice to report the error even if only some files were
invalid. However, go-fuse does not allow returning the valid
directory entries AND an error.
2016-06-04 16:39:27 +02:00
Jakob Unterwurzacher 80b027f830 nametransform, main: better error messages 2016-06-01 20:07:43 +02:00
Jakob Unterwurzacher 5dd9576a11 fusefrontend: replace unreliable "fd < 0" check
... with the "released" boolean.

For some reason, the "f.fd.Fd() < 0" check did not work reliably,
leading to nil pointer panics on the following wlock.lock().

The problem was discovered during fsstress testing and is unlikely
to happen in normal operations.

With this change, we passed 1700+ fsstress iterations.
2016-05-30 09:36:06 +02:00
Jakob Unterwurzacher 5e9953ec27 toggledlog: wpanic: use Logger.Panic instead of naked panic
This makes sure the panic message also ends up in syslog
(if enabled).
2016-05-30 09:26:59 +02:00
Jakob Unterwurzacher 1648c54adb fusefrontend: use sync.Once for one-time warnings
Using a simple boolean was racy (which was harmless
in this case) and non-idomatic.
2016-05-29 22:50:03 +02:00
Jakob Unterwurzacher fd53dfd2ad fusefronted: check Fstat return value on file create
The Fstat call should never fail, but still, if it does return an error
it should be handled properly.
2016-05-29 22:43:48 +02:00
Jakob Unterwurzacher 77813bdc13 fusefrontend: simplify wlockMap
mapMutex can be anonymous and using an RWMutex is overkill
because the lock protects very short sections.
2016-05-29 22:40:05 +02:00
Jakob Unterwurzacher 4c0cb37c50 fusefrontend: remove unused "forgotten" variable
The functionality has long been replaced by the fd < 0
check.
2016-05-29 13:46:47 +02:00
Jakob Unterwurzacher 888e147cd8 stupidgcm: add benchmark.bash wrapper
Add a simple bash wrapper to make it easier to run the GCM
benchmarks.
2016-05-22 15:49:09 +02:00
David Gnedt a93bcabe9c Encrypt path in statfs() calls
Paths in statfs() calls were not encrypted resulting in
an Function not implemented error, when the unencrypted
path didn't exist in the underlying (encrypted)
filesystem.

$ df plain/existingdir
df: ‘plain/existingdir’: Function not implemented
2016-05-12 23:24:39 +02:00
Jakob Unterwurzacher 4ad9d4e444 prefer_openssl: add amd64 constraint
Optimized assembly versions for Go GCM are only available
on amd64.
2016-05-12 09:50:36 +02:00
Jakob Unterwurzacher 49b597f07c prefer_openssl: autodetect whether to use OpenSSL or Go GCM
Go GCM is faster than OpenSSL if the CPU has AES instructions
and you are running Go 1.6+.

The "-openssl" option now defaults to "auto".

"gocryptfs -debug -version" displays the result of the autodetection.

See https://github.com/rfjakob/gocryptfs/issues/23 for details and
benchmarks.
2016-05-12 00:42:42 +02:00
Jakob Unterwurzacher ba7c798418 fusefrontend: fix panic due to concurrently unregistered wlock
Commit 730291feab properly freed wlock when the file descriptor is
closed. However, concurrently running Write and Truncates may
still want to lock it. Check if the fd has been closed first.
2016-05-08 23:21:20 +02:00
Jakob Unterwurzacher 730291feab fusefrontend: fix wlock memory leak
The write lock was not freed on release, causing a slow memory leak.

This was noticed by running extractloop.bash for 10 hours.
2016-05-05 13:38:39 +02:00
Jakob Unterwurzacher cf29ce3762 stupidgcm: set dummy locking callback.
In general, OpenSSL is only threadsafe if you provide a locking function
through CRYPTO_set_locking_callback. However, the GCM operations that
stupidgcm uses never call that function.

To guard against that ever changing, set a dummy locking callback
that crashes the app.
2016-05-05 00:09:08 +02:00
Jakob Unterwurzacher 906172938a stupidgcm: skip tests on Go 1.4 and older
Quoting from the patch:

	We compare against Go's built-in GCM implementation. Since stupidgcm only
	supports 128-bit IVs and Go only supports that from 1.5 onward, we cannot
	run these tests on older Go versions.
2016-05-05 00:09:08 +02:00
Jakob Unterwurzacher b4d45554f2 Revert "stupidgcm: print openssl error stack before panicing"
This did not help in debugging the openssl <= 1.0.1c issue at all
and makes the code more complex. Keep it simple.
2016-05-05 00:09:08 +02:00
Jakob Unterwurzacher 66156181ee cryptocore: support Go 1.4 in tests 2016-05-05 00:09:08 +02:00
Jakob Unterwurzacher 508a949d9d stupidgcm: reorder calls to support openssl <= 1.0.1c
This fixes the test failures on Travis CI.

Quoting from 07a4ff79d2

	/* Set expected tag value. A restriction in OpenSSL 1.0.1c and earlier
	 * required the tag before any AAD or ciphertext */
2016-05-05 00:08:25 +02:00
Jakob Unterwurzacher d0945b73d2 stupidgcm: print openssl error stack before panicing 2016-05-04 20:50:13 +02:00
Jakob Unterwurzacher 6c010c3080 stupidgcm: fix copy-paste error in panic message
Also, print the openssl version in Travis CI
2016-05-04 20:15:11 +02:00
Jakob Unterwurzacher 39f3a24484 stupidgcm: completely replace spacemonkeygo/openssl 2016-05-04 19:56:07 +02:00
Jakob Unterwurzacher c92190bf07 stupidgcm: add our own thin wrapper around openssl gcm
...complete with tests and benchmark.

This will allow us to get rid of the dependency to spacemonkeygo/openssl
that causes problems on Arch Linux
( https://github.com/rfjakob/gocryptfs/issues/21 )
2016-05-04 19:56:07 +02:00
Jakob Unterwurzacher 1bb907b38e cryptocore: add API tests 2016-05-04 19:56:07 +02:00
Jakob Unterwurzacher 4d79fba285 prelloc: warn and continue if fallocate(2) is not supported
This makes gocryptfs work at all on ZFS.
See https://github.com/rfjakob/gocryptfs/issues/22 .
2016-04-20 22:47:31 +02:00
Jakob Unterwurzacher bd5405189e Fix "go tool vet -shadow=true" warnings
Among those one real bug.
2016-04-10 21:31:15 +02:00
Jakob Unterwurzacher db72fcea41 longnames: fix fsstress failure, use dirfd
Using dirfd-relative operations allows safe lockless handling
of the ".name" files.
2016-04-10 21:31:08 +02:00
Jakob Unterwurzacher 63d3e51734 longnames: use symbolic constants instead of naked ints 2016-04-10 12:36:43 +02:00
Jakob Unterwurzacher 37a9b4c3ee Enable openssl in tests to support old Go versions
Go 1.4 and older do not support 128-bit IVs which caused
the tests to panic.
2016-03-03 00:59:58 +01:00
Jakob Unterwurzacher b5221c9651 Fix Go 1.4 build failure caused by a refactoring oversight 2016-03-03 00:48:32 +01:00
Jakob Unterwurzacher 653d4a619c longnames part II: Rename, Unlink, Rmdir, Mknod, Mkdir + tests 2016-02-07 14:02:09 +01:00
Jakob Unterwurzacher 6b5d977cce Move OpenDir to fs_dir.go 2016-02-07 11:29:54 +01:00
Jakob Unterwurzacher 3a4922b5d7 Add tests for long name creation and renaming (currently failing, obviously) 2016-02-07 10:55:13 +01:00
Jakob Unterwurzacher e111e20649 longnames part I: Create and OpenDir work with long filenames > 176 bytes
Todo: Rename, Unlink, Rmdir, Mknod, Mkdir
2016-02-06 22:54:14 +01:00
Jakob Unterwurzacher 6b27b6a1e7 Move dirIVCache to its own file 2016-02-06 20:27:28 +01:00
Jakob Unterwurzacher c74772bc8d Run go fmt 2016-02-06 20:23:36 +01:00
Jakob Unterwurzacher b0ee5258b1 Fix tests - were broken by the refactoring 2016-02-06 20:22:45 +01:00
Jakob Unterwurzacher 9078a77850 Move pathfs_frontend to internal/fusefrontend
"git status" for reference:

renamed:    pathfs_frontend/args.go -> internal/fusefrontend/args.go
renamed:    pathfs_frontend/compat_darwin.go -> internal/fusefrontend/compat_darwin.go
renamed:    pathfs_frontend/compat_linux.go -> internal/fusefrontend/compat_linux.go
renamed:    pathfs_frontend/file.go -> internal/fusefrontend/file.go
renamed:    pathfs_frontend/file_holes.go -> internal/fusefrontend/file_holes.go
renamed:    pathfs_frontend/fs.go -> internal/fusefrontend/fs.go
renamed:    pathfs_frontend/fs_dir.go -> internal/fusefrontend/fs_dir.go
renamed:    pathfs_frontend/names.go -> internal/fusefrontend/names.go
renamed:    pathfs_frontend/write_lock.go -> internal/fusefrontend/write_lock.go
modified:   main.go
2016-02-06 19:27:59 +01:00
Jakob Unterwurzacher 2b8cbd9441 Major refactoring: Split up "cryptfs" into several internal packages
"git status" for reference:

deleted:    cryptfs/cryptfs.go
deleted:    cryptfs/names_core.go
modified:   integration_tests/cli_test.go
modified:   integration_tests/helpers.go
renamed:    cryptfs/config_file.go -> internal/configfile/config_file.go
renamed:    cryptfs/config_test.go -> internal/configfile/config_test.go
renamed:    cryptfs/config_test/.gitignore -> internal/configfile/config_test/.gitignore
renamed:    cryptfs/config_test/PlaintextNames.conf -> internal/configfile/config_test/PlaintextNames.conf
renamed:    cryptfs/config_test/StrangeFeature.conf -> internal/configfile/config_test/StrangeFeature.conf
renamed:    cryptfs/config_test/v1.conf -> internal/configfile/config_test/v1.conf
renamed:    cryptfs/config_test/v2.conf -> internal/configfile/config_test/v2.conf
renamed:    cryptfs/kdf.go -> internal/configfile/kdf.go
renamed:    cryptfs/kdf_test.go -> internal/configfile/kdf_test.go
renamed:    cryptfs/cryptfs_content.go -> internal/contentenc/content.go
new file:   internal/contentenc/content_api.go
renamed:    cryptfs/content_test.go -> internal/contentenc/content_test.go
renamed:    cryptfs/file_header.go -> internal/contentenc/file_header.go
renamed:    cryptfs/intrablock.go -> internal/contentenc/intrablock.go
renamed:    cryptfs/address_translation.go -> internal/contentenc/offsets.go
new file:   internal/cryptocore/crypto_api.go
renamed:    cryptfs/gcm_go1.4.go -> internal/cryptocore/gcm_go1.4.go
renamed:    cryptfs/gcm_go1.5.go -> internal/cryptocore/gcm_go1.5.go
renamed:    cryptfs/nonce.go -> internal/cryptocore/nonce.go
renamed:    cryptfs/openssl_aead.go -> internal/cryptocore/openssl_aead.go
renamed:    cryptfs/openssl_benchmark.bash -> internal/cryptocore/openssl_benchmark.bash
renamed:    cryptfs/openssl_test.go -> internal/cryptocore/openssl_test.go
new file:   internal/nametransform/name_api.go
new file:   internal/nametransform/names_core.go
renamed:    cryptfs/names_diriv.go -> internal/nametransform/names_diriv.go
renamed:    cryptfs/names_noiv.go -> internal/nametransform/names_noiv.go
renamed:    cryptfs/names_test.go -> internal/nametransform/names_test.go
new file:   internal/nametransform/pad16.go
renamed:    cryptfs/log.go -> internal/toggledlog/log.go
renamed:    cryptfs/log_go1.4.go -> internal/toggledlog/log_go1.4.go
renamed:    cryptfs/log_go1.5.go -> internal/toggledlog/log_go1.5.go
modified:   main.go
modified:   masterkey.go
modified:   pathfs_frontend/file.go
modified:   pathfs_frontend/file_holes.go
modified:   pathfs_frontend/fs.go
modified:   pathfs_frontend/fs_dir.go
modified:   pathfs_frontend/names.go
modified:   test.bash
2016-02-06 19:22:35 +01:00