Compare commits

...

64 Commits

Author SHA1 Message Date
Jakob Unterwurzacher 75cace0568 cryptocore: simplify declarations
Reported by codacity:

internal/cryptocore/cryptocore.go
Minor icon MINOR
Code Style
should omit type AEADTypeEnum from declaration of var BackendAESSIV; it will be inferred from the right-hand side
var BackendAESSIV AEADTypeEnum = AEADTypeEnum{"AES-SIV-512", "Go", siv_aead.NonceSize}
Minor icon MINOR
Code Style
should omit type AEADTypeEnum from declaration of var BackendXChaCha20Poly1305; it will be inferred from the right-hand side
var BackendXChaCha20Poly1305 AEADTypeEnum = AEADTypeEnum{"XChaCha20-Poly1305", "Go", chacha20poly1305.NonceSizeX}
Minor icon MINOR
Code Style
should omit type AEADTypeEnum from declaration of var BackendXChaCha20Poly1305OpenSSL; it will be inferred from the right-hand side
var BackendXChaCha20Poly1305OpenSSL AEADTypeEnum = AEADTypeEnum{"XChaCha20-Poly1305", "OpenSSL", chacha20poly1305.NonceSizeX}
Found 2 possible new issues
internal/cryptocore/cryptocore.go
Minor icon MINOR
Code Style
should omit type AEADTypeEnum from declaration of var BackendOpenSSL; it will be inferred from the right-hand side
var BackendOpenSSL AEADTypeEnum = AEADTypeEnum{"AES-GCM-256", "OpenSSL", 16}
Minor icon MINOR
Code Style
should omit type AEADTypeEnum from declaration of var BackendGoGCM; it will be inferred from the right-hand side
var BackendGoGCM AEADTypeEnum = AEADTypeEnum{"AES-GCM-256", "Go", 16}
2021-09-28 18:35:37 +02:00
Jakob Unterwurzacher 5406284b9b build.bash: also try BSD date syntax for converting SOURCE_DATE_EPOCH
GNU date syntax does not work on macos.

Fixes https://github.com/rfjakob/gocryptfs/issues/570
2021-09-28 18:17:58 +02:00
Jakob Unterwurzacher e8e3598284 -init: suggest xchacha if we don't have AES accel
Example on Raspberry Pi 4:

$ ./gocryptfs/gocryptfs -init $(mktemp -d)
Notice: Your CPU does not have AES acceleration. Consider using -xchacha for better performance.
Choose a password for protecting your files.
Password:

https://github.com/rfjakob/gocryptfs/issues/607
2021-09-28 18:09:31 +02:00
Jakob Unterwurzacher c8996d2664 -info: add contentEncryption
Example:

$ ./gocryptfs -info ./tests/example_filesystems/v2.2-xchacha/
Creator:           gocryptfs v2.1-27-gabaa129-dirty.xchacha
FeatureFlags:      HKDF XChaCha20Poly1305 DirIV EMENames LongNames Raw64
EncryptedKey:      64B
ScryptObject:      Salt=32B N=1024 R=8 P=1 KeyLen=32
contentEncryption: XChaCha20-Poly1305
2021-09-28 18:09:31 +02:00
Jakob Unterwurzacher db1824a23a cryptocore: disentangle algorithm / library implementation name
Used in gocryptfs-xray, and will also be used in -info.
2021-09-28 18:09:31 +02:00
Jakob Unterwurzacher 5e67e183c0 README: set v2.2.0 release date 2021-09-25 16:45:36 +02:00
Jakob Unterwurzacher eceeaaad1f README: make changelog entries subheadings
This allows to anchor-link in to each release.
2021-09-25 16:44:06 +02:00
Jakob Unterwurzacher 53d51acd2b README: release will be called v2.2.0 instead of v2.2
pkg.go.dev really wants that we want to comply with
https://golang.org/doc/modules/version-numbers .

Trying v2.2-beta1 as in

    https://pkg.go.dev/github.com/rfjakob/gocryptfs/v2@v2.2-beta1

said "v2.2-beta1 is not a valid semantic version.".
2021-09-15 16:09:26 +02:00
Jakob Unterwurzacher 2d0ba24eca -speed: print cpu model
When somebody posts "gocryptfs -speed" results, they are
most helpful together with the CPU model. Add the cpu model
to the output.

Example:

$ ./gocryptfs -speed
gocryptfs v2.2.0-beta1-5-g52b0444-dirty; go-fuse v2.1.1-0.20210825171523-3ab5d95a30ae; 2021-09-14 go1.17.1 linux/amd64
cpu: Intel(R) Core(TM) i5-3470 CPU @ 3.20GHz; with AES acceleration
AES-GCM-256-OpenSSL       	 862.79 MB/s
AES-GCM-256-Go            	 997.71 MB/s	(selected in auto mode)
AES-SIV-512-Go            	 159.58 MB/s
XChaCha20-Poly1305-OpenSSL	 729.65 MB/s
XChaCha20-Poly1305-Go     	 843.97 MB/s	(selected in auto mode)
2021-09-14 18:58:22 +02:00
Jakob Unterwurzacher 61e37b2439 stupidgcm: add CpuHasAES()
Makes the code clearer, and will be used in the next commit.
2021-09-14 18:58:04 +02:00
Jakob Unterwurzacher 52b0444985 README: update example -speed output 2021-09-14 10:18:24 +02:00
Jakob Unterwurzacher cdbc48fe29 -speed: drop useless tab at end of line 2021-09-14 10:15:18 +02:00
Jakob Unterwurzacher d0cba59f6b README: highlight changes in v2.2, simplify pkg.go.dev link 2021-09-12 18:12:10 +02:00
Jakob Unterwurzacher 2a4380ac25 README: update changelog 2021-09-10 17:19:51 +02:00
Jakob Unterwurzacher c9b825c58a inomap: deterministically set root device
We used to have "first Translate() wins". This is not deterministic,
as the LOOKUP for the root directory does not seem to reach us, so
the first user LOOKUP would win, which may be on a mountpoint.
2021-09-10 17:17:16 +02:00
Jakob Unterwurzacher ee56103570 README: update changelog for v2.2-beta1 2021-09-10 12:22:02 +02:00
Jakob Unterwurzacher a85e39f682 Update README & MANPAGE 2021-09-10 12:17:22 +02:00
Jakob Unterwurzacher d023cd6c95 cli: drop -forcedecode flag
The rewritten openssl backend does not support this flag anymore,
and it was inherently dangerour. Drop it (ignored for compatibility)
2021-09-10 12:14:19 +02:00
Jakob Unterwurzacher c974116322 test.bash: call out if build-without-openssl.bash failed
This can print out compile errors that are hard to understand
if you are not aware that it builds without_openssl.
2021-09-10 12:09:30 +02:00
Jakob Unterwurzacher c50d67f103 profiling: accept parameters & show actual command lines 2021-09-10 11:51:41 +02:00
Jakob Unterwurzacher ad21647f25 -speed: show which xchacha implementation is preferred 2021-09-08 20:46:52 +02:00
Jakob Unterwurzacher 2620cad0dc tests/matrix: test xchacha with and without openssl 2021-09-08 20:34:01 +02:00
Jakob Unterwurzacher 94e8004b6c Make -openssl also apply to xchacha
Now that stupidgcm supports xchacha, make it available
on mount.
2021-09-08 20:32:16 +02:00
Jakob Unterwurzacher 1a58667293 stupidgcm: add PreferOpenSSL{AES256GCM,Xchacha20poly1305}
Add PreferOpenSSLXchacha20poly1305,
rename PreferOpenSSL -> PreferOpenSSLAES256GCM.
2021-09-08 19:48:13 +02:00
Jakob Unterwurzacher 85c2beccaf stupidgcm: normalize constructor naming
New() -> NewAES256GCM()

Also add missing NewChacha20poly1305
constructor in without_openssl.go.
2021-09-07 18:15:04 +02:00
Jakob Unterwurzacher f47e287c20 stupidgcm: revamp package documentation
Maybe interesting for people following
https://github.com/rfjakob/gocryptfs/issues/452
2021-09-07 18:15:04 +02:00
Jakob Unterwurzacher d598536709 stupidgcm: unexport stupidGCM struct
No need to have it exported.
2021-09-07 18:15:04 +02:00
Jakob Unterwurzacher 3a80db953d stupidgcm: allow zero-length input data
We used to panic in this case because it is useless.
But Go stdlib supports it, so we should as well.
2021-09-07 18:15:04 +02:00
Jakob Unterwurzacher 738d5a2b3a stupidgcm: fix build with CGO_ENABLED=1 without_openssl
We missed some "// +build" lines
2021-09-07 18:15:04 +02:00
Jakob Unterwurzacher d9510d0c0b stupidgcm: NewChacha20poly1305: avoid slice append
I noticed that growslice() shows up in the cpuprofile.
Avoiding slice append for the private jey copy gives a 0.6% speedup:

gocryptfs/internal/speed$ benchstat old new
name             old time/op   new time/op   delta
StupidXchacha-4   5.68µs ± 0%   5.65µs ± 0%  -0.63%  (p=0.008 n=5+5)

name             old speed     new speed     delta
StupidXchacha-4  721MB/s ± 0%  725MB/s ± 0%  +0.63%  (p=0.008 n=5+5)
2021-09-07 18:14:58 +02:00
Jakob Unterwurzacher 39b1070506 stupidgcm: add testConcurrency
Verifies that we don't corrupt data when called concurrently.
2021-09-07 18:14:05 +02:00
Jakob Unterwurzacher f89b14ee3d stupidgcm: cache C.EVP_chacha20_poly1305()
2% performance improvement, almost for free.

gocryptfs/internal/speed$ benchstat old new
name             old time/op   new time/op   delta
StupidXchacha-4   5.82µs ± 0%   5.68µs ± 0%  -2.37%  (p=0.008 n=5+5)

name             old speed     new speed     delta
StupidXchacha-4  704MB/s ± 0%  721MB/s ± 0%  +2.43%  (p=0.008 n=5+5)
2021-09-07 18:14:05 +02:00
Jakob Unterwurzacher 6a0206897c stupidgcm: add BenchmarkCCall
gocryptfs/internal/stupidgcm$ go test -bench .
goos: linux
goarch: amd64
pkg: github.com/rfjakob/gocryptfs/v2/internal/stupidgcm
cpu: Intel(R) Core(TM) i5-3470 CPU @ 3.20GHz
BenchmarkCCall-4   	15864030	        78.60 ns/op
PASS
ok  	github.com/rfjakob/gocryptfs/v2/internal/stupidgcm	1.898s
2021-09-07 18:14:05 +02:00
Jakob Unterwurzacher a2eaa5e3d1 speed: add BenchmarkStupidChacha
gocryptfs/internal/speed$ go test -bench .
goos: linux
goarch: amd64
pkg: github.com/rfjakob/gocryptfs/v2/internal/speed
cpu: Intel(R) Core(TM) i5-3470 CPU @ 3.20GHz
BenchmarkStupidGCM-4              	  249396	      4722 ns/op	 867.50 MB/s
BenchmarkStupidGCMDecrypt-4       	  257872	      4616 ns/op	 887.35 MB/s
BenchmarkGoGCM-4                  	  290952	      4097 ns/op	 999.83 MB/s
BenchmarkGoGCMDecrypt-4           	  294106	      4060 ns/op	1008.84 MB/s
BenchmarkAESSIV-4                 	   46520	     25532 ns/op	 160.42 MB/s
BenchmarkAESSIVDecrypt-4          	   46974	     25478 ns/op	 160.76 MB/s
BenchmarkXchacha-4                	  244108	      4881 ns/op	 839.14 MB/s
BenchmarkXchachaDecrypt-4         	  249658	      4786 ns/op	 855.86 MB/s
BenchmarkStupidXchacha-4          	  205339	      5768 ns/op	 710.11 MB/s
BenchmarkStupidXchachaDecrypt-4   	  204577	      5836 ns/op	 701.84 MB/s
BenchmarkStupidChacha-4           	  227510	      5224 ns/op	 784.06 MB/s
BenchmarkStupidChachaDecrypt-4    	  222787	      5359 ns/op	 764.34 MB/s
PASS
ok  	github.com/rfjakob/gocryptfs/v2/internal/speed	15.328s
2021-09-07 18:14:05 +02:00
Jakob Unterwurzacher b8c56ccffc stupidgcm: replace naked panics 2021-09-07 18:14:05 +02:00
Jakob Unterwurzacher 8f820c429d stupidgcm: fix without_openssl build
$ ./build-without-openssl.bash
internal/speed/speed.go:152:14: undefined: stupidgcm.NewXchacha20poly1305
2021-09-07 18:14:05 +02:00
Jakob Unterwurzacher c9728247ed test.bash: only check go files for naked panic
This found a lot of panics in the new file openssl_aead.c.
2021-09-07 18:14:05 +02:00
Jakob Unterwurzacher e2ec048a09 stupidgcm: introduce stupidAEADCommon and use for both chacha & gcm
Nice deduplication and brings the GCM decrypt speed up to par.

internal/speed$ benchstat old new
name                old time/op   new time/op   delta
StupidGCM-4          4.71µs ± 0%   4.66µs ± 0%   -0.99%  (p=0.008 n=5+5)
StupidGCMDecrypt-4   5.77µs ± 1%   4.51µs ± 0%  -21.80%  (p=0.008 n=5+5)

name                old speed     new speed     delta
StupidGCM-4         870MB/s ± 0%  879MB/s ± 0%   +1.01%  (p=0.008 n=5+5)
StupidGCMDecrypt-4  710MB/s ± 1%  908MB/s ± 0%  +27.87%  (p=0.008 n=5+5)
2021-09-07 18:14:05 +02:00
Jakob Unterwurzacher bf572aef88 stupidgcm: stupidChacha20poly1305.Open: batch C calls in aead_open
Gets the decryption speed to the same level as the
encryption speed.

internal/speed$ benchstat old.txt new.txt
name                    old time/op    new time/op    delta
StupidXchacha-4          732MB/s ± 0%   740MB/s ± 0%   ~     (p=1.000 n=1+1)
StupidXchachaDecrypt-4   602MB/s ± 0%   741MB/s ± 0%   ~     (p=1.000 n=1+1)
2021-09-07 18:14:05 +02:00
Jakob Unterwurzacher 3e27acb989 speed: add decryption benchmarks
gocryptfs/internal/speed$ go test -bench .
goos: linux
goarch: amd64
pkg: github.com/rfjakob/gocryptfs/v2/internal/speed
cpu: Intel(R) Core(TM) i5-3470 CPU @ 3.20GHz
BenchmarkStupidGCM-4              	  263742	      4523 ns/op	 905.61 MB/s
BenchmarkStupidGCMDecrypt-4       	  204858	      5779 ns/op	 708.76 MB/s
BenchmarkGoGCM-4                  	  291259	      4095 ns/op	1000.25 MB/s
BenchmarkGoGCMDecrypt-4           	  293886	      4061 ns/op	1008.53 MB/s
BenchmarkAESSIV-4                 	   46537	     25538 ns/op	 160.39 MB/s
BenchmarkAESSIVDecrypt-4          	   46770	     25627 ns/op	 159.83 MB/s
BenchmarkXchacha-4                	  243619	      4893 ns/op	 837.03 MB/s
BenchmarkXchachaDecrypt-4         	  248857	      4793 ns/op	 854.51 MB/s
BenchmarkStupidXchacha-4          	  213717	      5558 ns/op	 736.99 MB/s
BenchmarkStupidXchachaDecrypt-4   	  176635	      6782 ns/op	 603.96 MB/s
PASS
ok  	github.com/rfjakob/gocryptfs/v2/internal/speed	12.871s
2021-09-07 18:14:05 +02:00
Jakob Unterwurzacher 5046962634 speed: add bEncrypt helper, reuse dst buffer
The bEncrypt helper massively deduplicates the code,
and reusing the dst buffer gives higher performance,
and that's what gocryptfs does in normal operation via
sync.Pool.

$ benchstat old.txt new.txt
name             old time/op   new time/op    delta
StupidGCM-4       6.24µs ± 1%    4.65µs ± 0%  -25.47%  (p=0.008 n=5+5)
GoGCM-4           4.90µs ± 0%    4.10µs ± 0%  -16.44%  (p=0.008 n=5+5)
AESSIV-4          26.4µs ± 0%    25.6µs ± 0%   -2.90%  (p=0.008 n=5+5)
Xchacha-4         5.76µs ± 0%    4.91µs ± 0%  -14.79%  (p=0.008 n=5+5)
StupidXchacha-4   7.24µs ± 1%    5.48µs ± 0%  -24.33%  (p=0.008 n=5+5)

name             old speed     new speed      delta
StupidGCM-4      656MB/s ± 1%   880MB/s ± 0%  +34.15%  (p=0.008 n=5+5)
GoGCM-4          835MB/s ± 0%  1000MB/s ± 0%  +19.68%  (p=0.008 n=5+5)
AESSIV-4         155MB/s ± 0%   160MB/s ± 0%   +2.99%  (p=0.008 n=5+5)
Xchacha-4        711MB/s ± 0%   834MB/s ± 0%  +17.35%  (p=0.008 n=5+5)
StupidXchacha-4  565MB/s ± 1%   747MB/s ± 0%  +32.15%  (p=0.008 n=5+5)
2021-09-07 18:14:05 +02:00
Jakob Unterwurzacher d9e89cd021 stupidgcm: use aead_seal for gcm as well
$ benchstat old.txt new.txt
name         old time/op   new time/op   delta
StupidGCM-4   7.87µs ± 1%   6.64µs ± 2%  -15.65%  (p=0.000 n=10+10)

name         old speed     new speed     delta
StupidGCM-4  520MB/s ± 1%  617MB/s ± 2%  +18.56%  (p=0.000 n=10+10)
2021-09-07 18:14:05 +02:00
Jakob Unterwurzacher 69d626b26f stupidgcm: replace chacha20poly1305_seal with generic aead_seal 2021-09-07 18:14:05 +02:00
Jakob Unterwurzacher a3f5a8492a stupidgcm: batch C calls in chacha20poly1305_seal
Go has a high overhead for each C call, so batch
all openssl operations in the new C function chacha20poly1305_seal.

Benchmark results:

internal/speed$ go test -bench BenchmarkStupidXchacha -count 10 > old.txt
internal/speed$ go test -bench BenchmarkStupidXchacha -count 10 > new.txt

internal/speed$ benchstat old.txt new.txt
name             old time/op   new time/op   delta
StupidXchacha-4   8.79µs ± 1%   7.25µs ± 1%  -17.54%  (p=0.000 n=10+10)

name             old speed     new speed     delta
StupidXchacha-4  466MB/s ± 1%  565MB/s ± 1%  +21.27%  (p=0.000 n=10+10)
2021-09-07 18:14:05 +02:00
Jakob Unterwurzacher 9e1dd73e55 -speed: add XChaCha20-Poly1305-OpenSSL
$ ./gocryptfs -speed
gocryptfs v2.1-56-gdb1466f-dirty.stupidchacha; go-fuse v2.1.1-0.20210825171523-3ab5d95a30ae; 2021-09-02 go1.17 linux/amd64
AES-GCM-256-OpenSSL       	 529.53 MB/s
AES-GCM-256-Go            	 833.85 MB/s	(selected in auto mode)
AES-SIV-512-Go            	 155.27 MB/s
XChaCha20-Poly1305-Go     	 715.33 MB/s	(use via -xchacha flag)
XChaCha20-Poly1305-OpenSSL	 468.94 MB/s

https://github.com/rfjakob/gocryptfs/issues/452
2021-09-07 18:14:05 +02:00
Jakob Unterwurzacher 4017e4b22c stupidgcm: add stupidXchacha20poly1305
Implementation copied from
32db794688/chacha20poly1305/xchacha20poly1305.go
2021-09-07 18:14:05 +02:00
Jakob Unterwurzacher 591a56e7ae stupidgcm: stupidChacha20poly1305: normalize panic messages 2021-09-07 18:14:05 +02:00
Jakob Unterwurzacher 5df7ee815d stupidgcm: stupidChacha20poly1305: use byte array for key
Follow what golang.org/x/crypto/chacha20poly1305 does
for easier integration in the next commit.
2021-09-07 18:13:54 +02:00
Jakob Unterwurzacher 3ba74ac4fc stupidgcm: add testWipe test
After looking at the cover profile, this was the only untested
code except panic cases.
2021-09-02 10:17:01 +02:00
Jakob Unterwurzacher 961b8ca438 stupidgcm: deduplicate tests 2/2
Deduplicate the cipher setup that was identical
for all tests for each cipher.
2021-09-02 10:04:38 +02:00
Jakob Unterwurzacher 676a4ceb87 stupidgcm: deduplicate tests 1/2
Pull the code shared between chacha and gcm into
generic functions.
2021-09-02 09:57:20 +02:00
Jakob Unterwurzacher c9b090770a stupidgcm: add chacha20poly1305 via openssl
"stupidChacha20poly1305".

XChaCha will build upon this.
2021-09-02 09:30:28 +02:00
Jakob Unterwurzacher cbf282861b tests/matrix: don't leak fds in TestConcurrentReadCreate
We leaked a file descriptor for each empty file we encountered.
2021-09-01 10:28:33 +02:00
a1346054 7c2255be90 *: trim trailing whitespace 2021-09-01 10:22:01 +02:00
a1346054 6cb03b54fe *: fix spelling 2021-09-01 10:22:01 +02:00
a1346054 c63f7e9f64 shell scripts: fix shellcheck warnings 2021-09-01 10:22:01 +02:00
Jakob Unterwurzacher c505e73a13 README: explain where -xchacha makes sense 2021-08-30 20:00:00 +02:00
Jakob Unterwurzacher 4e3b7702af fusefrontend: remove leftover Printf
Commit b83ca9c921
inadveredly added a leftover debug Printf.

Delete it.
2021-08-30 11:39:44 +02:00
Jakob Unterwurzacher 34d8a498c4 Unbreak hyperlinks broken by go mod v2 conversion
Commit

  69d88505fd go mod: declare module version v2

translated all instances of "github.com/rfjakob/gocryptfs/" to
"github.com/rfjakob/gocryptfs/v2/".

Unfortunately, this included hyperlinks.

Unbreak the hyperlinks like this:

  find . -name \*.go | xargs sed -i s%https://github.com/rfjakob/gocryptfs/v2/%https://github.com/rfjakob/gocryptfs/v2/%
2021-08-30 11:31:01 +02:00
Jakob Unterwurzacher 17fe50ef74 README: compress Installation section
More content, less whitespace.
2021-08-30 10:18:33 +02:00
Jakob Unterwurzacher fab4ca07de README: update changelog 2021-08-30 10:18:23 +02:00
Jakob Unterwurzacher a99051b324 Reimplement -serialize_reads flag using new SyncRead mount flag
Let the kernel do the work for us.

See 15a8bb029a
for more info.
2021-08-30 09:53:58 +02:00
Jakob Unterwurzacher b83ca9c921 Remove serialize_reads package
Will be replaced by go-fuse's new SyncRead flag.

More info: https://github.com/hanwen/go-fuse/issues/395
SyncRead commit: 15a8bb029a
2021-08-30 09:41:38 +02:00
Jakob Unterwurzacher e69a85769f go mod: upgrade go-fuse to fix darwin build failure
Upgraded using

  go get -u github.com/hanwen/go-fuse/v2@master

to get 61df810860

Fixes https://github.com/rfjakob/gocryptfs/issues/597
2021-08-29 19:43:26 +02:00
96 changed files with 1744 additions and 1154 deletions

View File

@ -1,16 +1,16 @@
#!/bin/bash #!/bin/bash
set -eu set -eu
cd $(dirname "$0") cd "$(dirname "$0")"
# Render Markdown to a proper man(1) manpage # Render Markdown to a proper man(1) manpage
function render { render() {
IN=$1 IN=$1
OUT=$2 OUT=$2
echo "Rendering $IN to $OUT" echo "Rendering $IN to $OUT"
echo ".\\\" This man page was generated from $IN. View it using 'man ./$OUT'" > $OUT echo ".\\\" This man page was generated from $IN. View it using 'man ./$OUT'" > "$OUT"
echo ".\\\"" >> $OUT echo ".\\\"" >> "$OUT"
pandoc "$IN" -s -t man >> $OUT pandoc "$IN" -s -t man >> "$OUT"
} }
render MANPAGE.md gocryptfs.1 render MANPAGE.md gocryptfs.1

View File

@ -256,21 +256,10 @@ of a case where this may be useful is a situation where content is stored on a
filesystem that doesn't properly support UNIX ownership and permissions. filesystem that doesn't properly support UNIX ownership and permissions.
#### -forcedecode #### -forcedecode
Force decode of encrypted files even if the integrity check fails, instead of Obsolete and ignored on gocryptfs v2.2 and later.
failing with an IO error. Warning messages are still printed to syslog if corrupted
files are encountered.
It can be useful to recover files from disks with bad sectors or other corrupted
media. It shall not be used if the origin of corruption is unknown, specially
if you want to run executable files.
For corrupted media, note that you probably want to use dd_rescue(1) See https://github.com/rfjakob/gocryptfs/commit/d023cd6c95fcbc6b5056ba1f425d2ac3df4abc5a
instead, which will recover all but the corrupted 4kB block. for what it was and why it was dropped.
This option makes no sense in reverse mode. It requires gocryptfs to be compiled with openssl
support and implies -openssl true. Because of this, it is not compatible with -aessiv,
that uses built-in Go crypto.
Setting this option forces the filesystem to read-only and noexec.
#### -fsname string #### -fsname string
Override the filesystem name (first column in df -T). Can also be Override the filesystem name (first column in df -T). Can also be
@ -368,7 +357,7 @@ Mount the filesystem read-write (`-rw`, default) or read-only (`-ro`).
If both are specified, `-ro` takes precedence. If both are specified, `-ro` takes precedence.
#### -reverse #### -reverse
See the `-reverse` section in INIT FLAGS. You need to specifiy the See the `-reverse` section in INIT FLAGS. You need to specify the
`-reverse` option both at `-init` and at mount. `-reverse` option both at `-init` and at mount.
#### -serialize_reads #### -serialize_reads

145
README.md
View File

@ -55,30 +55,16 @@ A standalone Python tool that can decrypt files & file names is here:
Installation Installation
------------ ------------
Precompiled binaries that work on all x86_64 Linux systems are available for download from the github releases page. Precompiled binaries that work on all x86_64 Linux systems are available
for download from the github releases page. The `fuse` package from your
distribution must be installed for mounting to work.
On Debian, gocryptfs is available as a deb package: gocryptfs is also available as a package in most distributions. Examples:
```bash
apt install gocryptfs
```
On macOS, gocryptfs is available as a Homebrew formula: * Debian, Ubuntu: `apt install gocryptfs`
```bash * Fedora: `dnf install gocryptfs`
brew install gocryptfs * Arch: `pacman -S gocryptfs`
``` * MacPorts: `port install gocryptfs`
Alternatively, gocryptfs is also available via [MacPorts](https://www.macports.org/) on macOS:
```bash
sudo port install gocryptfs
```
On Fedora, gocryptfs is available as an rpm package:
```bash
sudo dnf install gocryptfs
```
If you use the standalone binary, make sure you install the `fuse` package
from your distributions package repository before running `gocryptfs`.
See the [Quickstart](https://nuetzlich.net/gocryptfs/quickstart/) page for more info. See the [Quickstart](https://nuetzlich.net/gocryptfs/quickstart/) page for more info.
@ -182,11 +168,13 @@ Example for a CPU with AES-NI:
``` ```
$ ./gocryptfs -speed $ ./gocryptfs -speed
gocryptfs v2.0; go-fuse v2.1.1-0.20210508151621-62c5aa1919a7; 2021-06-06 go1.16.5 linux/amd64 gocryptfs v2.2.0-beta1-5-g52b0444-dirty; go-fuse v2.1.1-0.20210825171523-3ab5d95a30ae; 2021-09-14 go1.17.1 linux/amd64
AES-GCM-256-OpenSSL 536.63 MB/s cpu: Intel(R) Core(TM) i5-3470 CPU @ 3.20GHz; with AES acceleration
AES-GCM-256-Go 831.84 MB/s (selected in auto mode) AES-GCM-256-OpenSSL 862.79 MB/s
AES-SIV-512-Go 155.85 MB/s AES-GCM-256-Go 997.71 MB/s (selected in auto mode)
XChaCha20-Poly1305-Go 700.02 MB/s (benchmark only, not selectable yet) AES-SIV-512-Go 159.58 MB/s
XChaCha20-Poly1305-OpenSSL 729.65 MB/s
XChaCha20-Poly1305-Go 843.97 MB/s (selected in auto mode)
``` ```
You can run `./benchmark.bash` to run gocryptfs' canonical set of You can run `./benchmark.bash` to run gocryptfs' canonical set of
@ -208,15 +196,30 @@ RM: 2,367
Changelog Changelog
--------- ---------
v2.2, IN PROGRESS #### v2.2.0, 2021-09-25
* `-deterministic-names`: new option for `-init`, both for reverse and forward mode. * **`-deterministic-names`: new option for `-init`**, both for reverse and forward mode.
Disables file name randomisation & `gocryptfs.diriv` files Disables file name randomisation & `gocryptfs.diriv` files
([#151](https://github.com/rfjakob/gocryptfs/issues/151), [#402](https://github.com/rfjakob/gocryptfs/issues/402), [#592](https://github.com/rfjakob/gocryptfs/pull/592)) ([#151](https://github.com/rfjakob/gocryptfs/issues/151), [#402](https://github.com/rfjakob/gocryptfs/issues/402), [#592](https://github.com/rfjakob/gocryptfs/pull/592))
* `-xchacha`: new option for `-init` (forward mode only). Selects XChaCha20-Poly1305 for content encryption. * New feature flag! You need gocryptfs v2.2 or higher to mount a filesystem that uses this flag.
Gives [better performance on embedded CPUs](https://gist.github.com/rfjakob/b28383f4c84263ac7c5388ccc262e38b) * **`-xchacha`: new option for `-init`** (forward mode only). Selects XChaCha20-Poly1305 for content encryption.
([#452](https://github.com/rfjakob/gocryptfs/issues/452)) Gives *much* better performance on CPUs without AES acceleration
([#452](https://github.com/rfjakob/gocryptfs/issues/452)).
* New feature flag! You need gocryptfs v2.2 or higher to mount a filesystem that uses this flag.
* Test with `gocryptfs -speed` what is fastest for your CPU, or read [here](https://github.com/rfjakob/gocryptfs/issues/452#issuecomment-908559414)
* Rewrite [OpenSSL backend](https://pkg.go.dev/github.com/rfjakob/gocryptfs/v2/internal/stupidgcm)
for better performance on AES-GCM-256-OpenSSL and XChaCha20-Poly1305-OpenSSL
* `-serialize_reads`: get rid of delay logic by taking advantage of the kernel flag
`FUSE_CAP_ASYNC_READ`
([go-fuse commit](https://github.com/hanwen/go-fuse/commit/15a8bb029a4e1a51e10043c370970596b1fbb737),
[gocryptfs commit](https://github.com/rfjakob/gocryptfs/commit/a99051b32452c9a781efe248c0014b65d4abddf7))
* Make obsolete `-devrandom` flag a no-op ([commit](https://github.com/rfjakob/gocryptfs/commit/61ef6b00a675456ee05d40f1ce44d693bc4be350))
* Make `-forcedecode` flag a no-op ([commit](https://github.com/rfjakob/gocryptfs/commit/d023cd6c95fcbc6b5056ba1f425d2ac3df4abc5a))
* Fix reverse mode sometimes remapping most inode numbers to >281474976710656 ([commit](https://github.com/rfjakob/gocryptfs/commit/c9b825c58a9f996379108926754513bca03bb306))
* This version will be called v2.2.0 (instead of v2.2) to comply with
the [Go module versioning](https://golang.org/doc/modules/version-numbers) convention.
Later releases will also follow the convention.
v2.1, 2021-08-18 #### v2.1, 2021-08-18
* `-fido2`: do not request PIN on `gocryptfs -init` fixing `FIDO_ERR_UNSUPPORTED_OPTION` with YubiKey * `-fido2`: do not request PIN on `gocryptfs -init` fixing `FIDO_ERR_UNSUPPORTED_OPTION` with YubiKey
([#571](https://github.com/rfjakob/gocryptfs/issues/571)) ([#571](https://github.com/rfjakob/gocryptfs/issues/571))
* `-sharedstorage`: present stable inode numbers, fixing getcwd failures * `-sharedstorage`: present stable inode numbers, fixing getcwd failures
@ -235,19 +238,19 @@ v2.1, 2021-08-18
* Drop support for Go 1.11 & Go 1.12 ([commit](https://github.com/rfjakob/gocryptfs/commit/a5f88e86d186cdbc67e1efabd7aacf389775e027)) * Drop support for Go 1.11 & Go 1.12 ([commit](https://github.com/rfjakob/gocryptfs/commit/a5f88e86d186cdbc67e1efabd7aacf389775e027))
* You must have Go 1.13 or newer now * You must have Go 1.13 or newer now
v2.0.1, 2021-06-07 #### v2.0.1, 2021-06-07
* Fix symlink creation reporting the wrong size, causing git to report it as modified * Fix symlink creation reporting the wrong size, causing git to report it as modified
([#574](https://github.com/rfjakob/gocryptfs/issues/574)) ([#574](https://github.com/rfjakob/gocryptfs/issues/574))
v2.0, 2021-06-05 #### v2.0, 2021-06-05
* Fix a few [issues discovered by xfstests](https://github.com/rfjakob/fuse-xfstests/wiki/results_2021-05-19) * Fix a few [issues discovered by xfstests](https://github.com/rfjakob/fuse-xfstests/wiki/results_2021-05-19)
* Biggest change: rewrite SEEK_HOLE / SEEK_DATA logic (now emulates 4k alignment) * Biggest change: rewrite SEEK_HOLE / SEEK_DATA logic (now emulates 4k alignment)
v2.0-beta4, 2021-05-15 #### v2.0-beta4, 2021-05-15
* **Make ACLs *actually* work (pass `-acl` to enable)** ([#536](https://github.com/rfjakob/gocryptfs/issues/536)) * **Make ACLs *actually* work (pass `-acl` to enable)** ([#536](https://github.com/rfjakob/gocryptfs/issues/536))
* Blocklist `RENAME_EXCHANGE` and `RENAME_WHITEOUT` (broken as discovered by [fuse-xfstest/gocryptfs-2019-12](https://github.com/rfjakob/fuse-xfstests/tree/gocryptfs-2019-12)) * Blocklist `RENAME_EXCHANGE` and `RENAME_WHITEOUT` (broken as discovered by [fuse-xfstest/gocryptfs-2019-12](https://github.com/rfjakob/fuse-xfstests/tree/gocryptfs-2019-12))
v2.0-beta3, 2021-04-24 #### v2.0-beta3, 2021-04-24
* MANPAGE: Split options into sections acc. to where they apply ([#517](https://github.com/rfjakob/gocryptfs/issues/517)) * MANPAGE: Split options into sections acc. to where they apply ([#517](https://github.com/rfjakob/gocryptfs/issues/517))
* `-idle`: count cwd inside the mount as busy ([#533](https://github.com/rfjakob/gocryptfs/issues/533)) * `-idle`: count cwd inside the mount as busy ([#533](https://github.com/rfjakob/gocryptfs/issues/533))
* Make `gocryptfs.diriv` and `gocryptfs.xxx.name` files world-readable to make encrypted backups easier * Make `gocryptfs.diriv` and `gocryptfs.xxx.name` files world-readable to make encrypted backups easier
@ -266,11 +269,11 @@ v2.0-beta3, 2021-04-24
* Add directory fd caching for 2x - 3x speed boost in small file ops compared to v2.0-beta2 * Add directory fd caching for 2x - 3x speed boost in small file ops compared to v2.0-beta2
([performance numbers](https://github.com/rfjakob/gocryptfs/blob/5cb1e55714aa92a848c0fb5fc3fa7b91625210fe/Documentation/performance.txt#L73)) ([performance numbers](https://github.com/rfjakob/gocryptfs/blob/5cb1e55714aa92a848c0fb5fc3fa7b91625210fe/Documentation/performance.txt#L73))
v2.0-beta2, 2020-11-14 #### v2.0-beta2, 2020-11-14
* Improve [performance](Documentation/performance.txt#L69) * Improve [performance](Documentation/performance.txt#L69)
* Fix [GETATTR panic](https://github.com/rfjakob/gocryptfs/issues/519#issuecomment-718790790) in reverse mode * Fix [GETATTR panic](https://github.com/rfjakob/gocryptfs/issues/519#issuecomment-718790790) in reverse mode
v2.0-beta1, 2020-10-15 #### v2.0-beta1, 2020-10-15
* **Switch to the improved go-fuse [v2 API](https://pkg.go.dev/github.com/hanwen/go-fuse/v2@v2.0.3/fs)** * **Switch to the improved go-fuse [v2 API](https://pkg.go.dev/github.com/hanwen/go-fuse/v2@v2.0.3/fs)**
* This is a big change, a lot of code has been reorganized or rewritten * This is a big change, a lot of code has been reorganized or rewritten
to fit the v2 API model. to fit the v2 API model.
@ -289,7 +292,7 @@ v2.0-beta1, 2020-10-15
([go-fuse #276](https://github.com/hanwen/go-fuse/issues/276), ([go-fuse #276](https://github.com/hanwen/go-fuse/issues/276),
[gocryptfs commit ec74d1d](https://github.com/rfjakob/gocryptfs/commit/ec74d1d2f4217a9a337d1db9902f32ae2aecaf33)) [gocryptfs commit ec74d1d](https://github.com/rfjakob/gocryptfs/commit/ec74d1d2f4217a9a337d1db9902f32ae2aecaf33))
v1.8.0, 2020-05-09 #### v1.8.0, 2020-05-09
* Enable ACL support ([#453](https://github.com/rfjakob/gocryptfs/issues/453)) * Enable ACL support ([#453](https://github.com/rfjakob/gocryptfs/issues/453))
* **Warning 2021-02-07**: This feature is incomplete! Do not use ACLs before gocryptfs v2.0 final! * **Warning 2021-02-07**: This feature is incomplete! Do not use ACLs before gocryptfs v2.0 final!
Reading and writing ACLs works, but they are not enforced or inherited ([#542](https://github.com/rfjakob/gocryptfs/issues/542)) Reading and writing ACLs works, but they are not enforced or inherited ([#542](https://github.com/rfjakob/gocryptfs/issues/542))
@ -314,7 +317,7 @@ v1.8.0, 2020-05-09
* Has been disabled since v1.7 due to issues a third-party module. * Has been disabled since v1.7 due to issues a third-party module.
* Please use FIDO2 instead (gocryptfs v2.0) * Please use FIDO2 instead (gocryptfs v2.0)
v1.7.1, 2019-10-06 #### v1.7.1, 2019-10-06
* Support wild cards in reverse mode via `--exclude-wildcard` * Support wild cards in reverse mode via `--exclude-wildcard`
([#367](https://github.com/rfjakob/gocryptfs/pull/367)). Thanks @ekalin! ([#367](https://github.com/rfjakob/gocryptfs/pull/367)). Thanks @ekalin!
* Create `gocryptfs.diriv` files with 0440 permissions to make it easier to * Create `gocryptfs.diriv` files with 0440 permissions to make it easier to
@ -335,7 +338,7 @@ v1.7.1, 2019-10-06
* tests: use /var/tmp instead of /tmp by default * tests: use /var/tmp instead of /tmp by default
([commit 8c4429](https://github.com/rfjakob/gocryptfs/commit/8c4429408716d9890a98a48c246d616dbfea7e31)) ([commit 8c4429](https://github.com/rfjakob/gocryptfs/commit/8c4429408716d9890a98a48c246d616dbfea7e31))
v1.7, 2019-03-17 #### v1.7, 2019-03-17
* **Fix possible symlink race attacks in forward mode** when using allow_other + plaintextnames * **Fix possible symlink race attacks in forward mode** when using allow_other + plaintextnames
* If you use *both* `-allow_other` *and* `-plaintextnames`, you should upgrade. * If you use *both* `-allow_other` *and* `-plaintextnames`, you should upgrade.
Malicious users could trick gocryptfs into modifying files outside of `CIPHERDIR`, Malicious users could trick gocryptfs into modifying files outside of `CIPHERDIR`,
@ -368,11 +371,11 @@ v1.7, 2019-03-17
* Trezor support has been broken since Sept 2018 due to issues * Trezor support has been broken since Sept 2018 due to issues
in a third-party module ([#261](https://github.com/rfjakob/gocryptfs/issues/261)) in a third-party module ([#261](https://github.com/rfjakob/gocryptfs/issues/261))
v1.6.1, 2018-12-12 #### v1.6.1, 2018-12-12
* Fix "Operation not supported" chmod errors on Go 1.11 * Fix "Operation not supported" chmod errors on Go 1.11
([#271](https://github.com/rfjakob/gocryptfs/issues/271)) ([#271](https://github.com/rfjakob/gocryptfs/issues/271))
v1.6, 2018-08-18 #### v1.6, 2018-08-18
* **Add `-e` / `-exclude` option** for reverse mode * **Add `-e` / `-exclude` option** for reverse mode
([#235](https://github.com/rfjakob/gocryptfs/issues/235), ([#235](https://github.com/rfjakob/gocryptfs/issues/235),
[commit](https://github.com/rfjakob/gocryptfs/commit/ec2fdc19cf9358ae7ba09c528a5807b6b0760f9b)) [commit](https://github.com/rfjakob/gocryptfs/commit/ec2fdc19cf9358ae7ba09c528a5807b6b0760f9b))
@ -386,7 +389,7 @@ v1.6, 2018-08-18
* Fall back to buffered IO even when passed `O_DIRECT` * Fall back to buffered IO even when passed `O_DIRECT`
([commit](https://github.com/rfjakob/gocryptfs/commit/893e41149ed353f355047003b89eeff456990e76)) ([commit](https://github.com/rfjakob/gocryptfs/commit/893e41149ed353f355047003b89eeff456990e76))
v1.5, 2018-06-12 #### v1.5, 2018-06-12
* **Support extended attributes (xattr)** in forward mode * **Support extended attributes (xattr)** in forward mode
([#217](https://github.com/rfjakob/gocryptfs/issues/217)). Older gocryptfs versions ([#217](https://github.com/rfjakob/gocryptfs/issues/217)). Older gocryptfs versions
will ignore the extended attributes. will ignore the extended attributes.
@ -406,7 +409,7 @@ v1.5, 2018-06-12
* Stop printing the help text on a "flag provided but not defined" * Stop printing the help text on a "flag provided but not defined"
error ([commit](https://github.com/rfjakob/gocryptfs/commit/5ad26495fc86527bbfe75ac6b46528d49a373676)) error ([commit](https://github.com/rfjakob/gocryptfs/commit/5ad26495fc86527bbfe75ac6b46528d49a373676))
v1.4.4, 2018-03-18 #### v1.4.4, 2018-03-18
* Overwrite secrets in memory with zeros as soon as possible * Overwrite secrets in memory with zeros as soon as possible
([#211](https://github.com/rfjakob/gocryptfs/issues/211)) ([#211](https://github.com/rfjakob/gocryptfs/issues/211))
* Fix Getdents problems on i386 and mips64le * Fix Getdents problems on i386 and mips64le
@ -419,7 +422,7 @@ v1.4.4, 2018-03-18
[commit](https://github.com/hanwen/go-fuse/commit/a9ddcb8a4b609500fc59c89ccc9ee05f00a5fefd)) [commit](https://github.com/hanwen/go-fuse/commit/a9ddcb8a4b609500fc59c89ccc9ee05f00a5fefd))
* Fix various test issues on MacOS * Fix various test issues on MacOS
v1.4.3, 2018-01-21 #### v1.4.3, 2018-01-21
* **Fix several symlink race attacks** in connection with reverse mode * **Fix several symlink race attacks** in connection with reverse mode
and allow_other. Thanks to @slackner for reporting and helping to fix and allow_other. Thanks to @slackner for reporting and helping to fix
the issues: the issues:
@ -437,7 +440,7 @@ v1.4.3, 2018-01-21
* MacOS: let OSXFuse create the mountpoint if it does not exist * MacOS: let OSXFuse create the mountpoint if it does not exist
([issue #194](https://github.com/rfjakob/gocryptfs/issues/194)) ([issue #194](https://github.com/rfjakob/gocryptfs/issues/194))
v1.4.2, 2017-11-01 #### v1.4.2, 2017-11-01
* Add `Gopkg.toml` file for `dep` vendoring and reproducible builds * Add `Gopkg.toml` file for `dep` vendoring and reproducible builds
([issue #142](https://github.com/rfjakob/gocryptfs/issues/142)) ([issue #142](https://github.com/rfjakob/gocryptfs/issues/142))
* MacOS: deal with `.DS_Store` files inside CIPHERDIR * MacOS: deal with `.DS_Store` files inside CIPHERDIR
@ -450,7 +453,7 @@ v1.4.2, 2017-11-01
* Fix a startup hang when `$PATH` contains the mountpoint * Fix a startup hang when `$PATH` contains the mountpoint
([issue #146](https://github.com/rfjakob/gocryptfs/issues/146)) ([issue #146](https://github.com/rfjakob/gocryptfs/issues/146))
v1.4.1, 2017-08-21 #### v1.4.1, 2017-08-21
* **Use memory pools for buffer handling** ( * **Use memory pools for buffer handling** (
[3c6fe98](https://github.com/rfjakob/gocryptfs/commit/3c6fe98), [3c6fe98](https://github.com/rfjakob/gocryptfs/commit/3c6fe98),
[b2a23e9](https://github.com/rfjakob/gocryptfs/commit/b2a23e9), [b2a23e9](https://github.com/rfjakob/gocryptfs/commit/b2a23e9),
@ -474,7 +477,7 @@ v1.4.1, 2017-08-21
* Enable writing to write-only files * Enable writing to write-only files
([issue #125](https://github.com/rfjakob/gocryptfs/issues/125)) ([issue #125](https://github.com/rfjakob/gocryptfs/issues/125))
v1.4, 2017-06-20 #### v1.4, 2017-06-20
* **Switch to static binary releases** * **Switch to static binary releases**
* From gocryptfs v1.4, I will only release statically-built binaries. * From gocryptfs v1.4, I will only release statically-built binaries.
These support all Linux distributions but cannot use OpenSSL. These support all Linux distributions but cannot use OpenSSL.
@ -499,7 +502,7 @@ v1.4, 2017-06-20
([commit 80516ed](https://github.com/rfjakob/gocryptfs/commit/80516ed3351477793eec882508969b6b29b69b0a)) ([commit 80516ed](https://github.com/rfjakob/gocryptfs/commit/80516ed3351477793eec882508969b6b29b69b0a))
* Add `-info` option to pretty-print infos about a filesystem. * Add `-info` option to pretty-print infos about a filesystem.
v1.3, 2017-04-29 #### v1.3, 2017-04-29
* **Use HKDF to derive separate keys for GCM and EME** * **Use HKDF to derive separate keys for GCM and EME**
* New feature flag: `HKDF` (enabled by default) * New feature flag: `HKDF` (enabled by default)
* This is a forwards-compatible change. gocryptfs v1.3 can mount * This is a forwards-compatible change. gocryptfs v1.3 can mount
@ -522,14 +525,14 @@ v1.3, 2017-04-29
that were compiled without Large File Support. that were compiled without Large File Support.
* Passing "--" now also blocks "-o" parsing * Passing "--" now also blocks "-o" parsing
v1.2.1, 2017-02-26 #### v1.2.1, 2017-02-26
* Add an integrated speed test, `gocryptfs -speed` * Add an integrated speed test, `gocryptfs -speed`
* Limit password size to 1000 bytes and reject trailing garbage after the newline * Limit password size to 1000 bytes and reject trailing garbage after the newline
* Make the test suite work on [Mac OS X](https://github.com/rfjakob/gocryptfs/issues/15) * Make the test suite work on [Mac OS X](https://github.com/rfjakob/gocryptfs/issues/15)
* Handle additional corner cases in `-ctlsock` path sanitization * Handle additional corner cases in `-ctlsock` path sanitization
* Use dedicated exit code 12 on "password incorrect" * Use dedicated exit code 12 on "password incorrect"
v1.2, 2016-12-04 #### v1.2, 2016-12-04
* Add a control socket interface. Allows to encrypt and decrypt filenames. * Add a control socket interface. Allows to encrypt and decrypt filenames.
For details see [backintime#644](https://github.com/bit-team/backintime/issues/644#issuecomment-259835183). For details see [backintime#644](https://github.com/bit-team/backintime/issues/644#issuecomment-259835183).
* New command-line option: `-ctlsock` * New command-line option: `-ctlsock`
@ -548,14 +551,14 @@ v1.2, 2016-12-04
* Preserve owner for symlinks an device files (fixes bug [#64](https://github.com/rfjakob/gocryptfs/issues/64)) * Preserve owner for symlinks an device files (fixes bug [#64](https://github.com/rfjakob/gocryptfs/issues/64))
* Include rendered man page `gocryptfs.1` in the release tarball * Include rendered man page `gocryptfs.1` in the release tarball
v1.1.1, 2016-10-30 #### v1.1.1, 2016-10-30
* Fix a panic on setting file timestamps ([go-fuse#131](https://github.com/hanwen/go-fuse/pull/131)) * Fix a panic on setting file timestamps ([go-fuse#131](https://github.com/hanwen/go-fuse/pull/131))
* Work around an issue in tmpfs that caused a panic in xfstests generic/075 * Work around an issue in tmpfs that caused a panic in xfstests generic/075
([gocryptfs#56](https://github.com/rfjakob/gocryptfs/issues/56)) ([gocryptfs#56](https://github.com/rfjakob/gocryptfs/issues/56))
* Optimize NFS streaming writes * Optimize NFS streaming writes
([commit](https://github.com/rfjakob/gocryptfs/commit/a08d55f42d5b11e265a8617bee16babceebfd026)) ([commit](https://github.com/rfjakob/gocryptfs/commit/a08d55f42d5b11e265a8617bee16babceebfd026))
v1.1, 2016-10-19 #### v1.1, 2016-10-19
* **Add reverse mode ([#19](https://github.com/rfjakob/gocryptfs/issues/19))** * **Add reverse mode ([#19](https://github.com/rfjakob/gocryptfs/issues/19))**
* AES-SIV (RFC5297) encryption to implement deterministic encryption * AES-SIV (RFC5297) encryption to implement deterministic encryption
securely. Uses the excellent securely. Uses the excellent
@ -581,7 +584,7 @@ v1.1, 2016-10-19
* Enable changing the password when you only know the master key * Enable changing the password when you only know the master key
([#28](https://github.com/rfjakob/gocryptfs/issues/28)) ([#28](https://github.com/rfjakob/gocryptfs/issues/28))
v1.0, 2016-07-17 #### v1.0, 2016-07-17
* Deprecate very old filesystems, stage 3/3 * Deprecate very old filesystems, stage 3/3
* Filesystems created by v0.6 can no longer be mounted * Filesystems created by v0.6 can no longer be mounted
* Drop command-line options `-gcmiv128`, `-emenames`, `-diriv`. These * Drop command-line options `-gcmiv128`, `-emenames`, `-diriv`. These
@ -595,7 +598,7 @@ v1.0, 2016-07-17
* Experimental Mac OS X support. See * Experimental Mac OS X support. See
[ticket #15](https://github.com/rfjakob/gocryptfs/issues/15) for details. [ticket #15](https://github.com/rfjakob/gocryptfs/issues/15) for details.
v0.12, 2016-06-19 #### v0.12, 2016-06-19
* Deprecate very old filesystems, stage 2/3 * Deprecate very old filesystems, stage 2/3
* Filesystems created by v0.6 and older can only be mounted read-only * Filesystems created by v0.6 and older can only be mounted read-only
* A [message](https://github.com/rfjakob/gocryptfs/blob/v0.12/internal/configfile/config_file.go#L120) * A [message](https://github.com/rfjakob/gocryptfs/blob/v0.12/internal/configfile/config_file.go#L120)
@ -604,7 +607,7 @@ v0.12, 2016-06-19
* Mounts the filesystem read-only * Mounts the filesystem read-only
* Accept password from stdin as well ([ticket #30](https://github.com/rfjakob/gocryptfs/issues/30)) * Accept password from stdin as well ([ticket #30](https://github.com/rfjakob/gocryptfs/issues/30))
v0.11, 2016-06-10 #### v0.11, 2016-06-10
* Deprecate very old filesystems, stage 1/3 * Deprecate very old filesystems, stage 1/3
* Filesystems created by v0.6 and older can still be mounted but a * Filesystems created by v0.6 and older can still be mounted but a
[warning](https://github.com/rfjakob/gocryptfs/blob/v0.11/internal/configfile/config_file.go#L120) [warning](https://github.com/rfjakob/gocryptfs/blob/v0.11/internal/configfile/config_file.go#L120)
@ -616,7 +619,7 @@ v0.11, 2016-06-10
* Build release binaries with Go 1.6.2 * Build release binaries with Go 1.6.2
* Big speedup for CPUs with AES-NI, see [ticket #23](https://github.com/rfjakob/gocryptfs/issues/23) * Big speedup for CPUs with AES-NI, see [ticket #23](https://github.com/rfjakob/gocryptfs/issues/23)
v0.10, 2016-05-30 #### v0.10, 2016-05-30
* **Replace `spacemonkeygo/openssl` with `stupidgcm`** * **Replace `spacemonkeygo/openssl` with `stupidgcm`**
* gocryptfs now has its own thin wrapper to OpenSSL's GCM implementation * gocryptfs now has its own thin wrapper to OpenSSL's GCM implementation
called `stupidgcm`. called `stupidgcm`.
@ -636,7 +639,7 @@ v0.10, 2016-05-30
* Fix a fsstress [failure](https://github.com/hanwen/go-fuse/issues/106) * Fix a fsstress [failure](https://github.com/hanwen/go-fuse/issues/106)
in the go-fuse library. in the go-fuse library.
v0.9, 2016-04-10 #### v0.9, 2016-04-10
* **Long file name support** * **Long file name support**
* gocryptfs now supports file names up to 255 characters. * gocryptfs now supports file names up to 255 characters.
* This is a forwards-compatible change. gocryptfs v0.9 can mount filesystems * This is a forwards-compatible change. gocryptfs v0.9 can mount filesystems
@ -649,25 +652,25 @@ v0.9, 2016-04-10
* `-d`: Alias for `-debug` * `-d`: Alias for `-debug`
* `-q`: Alias for `-quiet` * `-q`: Alias for `-quiet`
v0.8, 2016-01-23 #### v0.8, 2016-01-23
* Redirect output to syslog when running in the background * Redirect output to syslog when running in the background
* New command-line option: * New command-line option:
* `-memprofile`: Write a memory allocation debugging profile the specified * `-memprofile`: Write a memory allocation debugging profile the specified
file file
v0.7.2, 2016-01-19 #### v0.7.2, 2016-01-19
* **Fix performance issue in small file creation** * **Fix performance issue in small file creation**
* This brings performance on-par with EncFS paranoia mode, with streaming writes * This brings performance on-par with EncFS paranoia mode, with streaming writes
significantly faster significantly faster
* The actual [fix](https://github.com/hanwen/go-fuse/commit/c4b6b7949716d13eec856baffc7b7941ae21778c) * The actual [fix](https://github.com/hanwen/go-fuse/commit/c4b6b7949716d13eec856baffc7b7941ae21778c)
is in the go-fuse library. There are no code changes in gocryptfs. is in the go-fuse library. There are no code changes in gocryptfs.
v0.7.1, 2016-01-09 #### v0.7.1, 2016-01-09
* Make the `build.bash` script compatible with Go 1.3 * Make the `build.bash` script compatible with Go 1.3
* Disable fallocate on OSX (system call not available) * Disable fallocate on OSX (system call not available)
* Introduce pre-built binaries for Fedora 23 and Debian 8 * Introduce pre-built binaries for Fedora 23 and Debian 8
v0.7, 2015-12-20 #### v0.7, 2015-12-20
* **Extend GCM IV size to 128 bit from Go's default of 96 bit** * **Extend GCM IV size to 128 bit from Go's default of 96 bit**
* This pushes back the birthday bound to make IV collisions virtually * This pushes back the birthday bound to make IV collisions virtually
impossible impossible
@ -676,7 +679,7 @@ v0.7, 2015-12-20
* New command-line option: * New command-line option:
* `-gcmiv128`: Use 128-bit GCM IVs (default true) * `-gcmiv128`: Use 128-bit GCM IVs (default true)
v0.6, 2015-12-08 #### v0.6, 2015-12-08
* **Wide-block filename encryption using EME + DirIV** * **Wide-block filename encryption using EME + DirIV**
* EME (ECB-Mix-ECB) provides even better security than CBC as it fixes * EME (ECB-Mix-ECB) provides even better security than CBC as it fixes
the prefix leak. The used Go EME implementation is the prefix leak. The used Go EME implementation is
@ -687,11 +690,11 @@ v0.6, 2015-12-08
* New command-line option: * New command-line option:
* `-emenames`: Enable EME filename encryption (default true) * `-emenames`: Enable EME filename encryption (default true)
v0.5.1, 2015-12-06 #### v0.5.1, 2015-12-06
* Fix a rename regression caused by DirIV and add test case * Fix a rename regression caused by DirIV and add test case
* Use fallocate to guard against out-of-space errors * Use fallocate to guard against out-of-space errors
v0.5, 2015-12-04 #### v0.5, 2015-12-04
* **Stronger filename encryption: DirIV** * **Stronger filename encryption: DirIV**
* Each directory gets a random 128 bit file name IV on creation, * Each directory gets a random 128 bit file name IV on creation,
stored in `gocryptfs.diriv` stored in `gocryptfs.diriv`
@ -707,7 +710,7 @@ v0.5, 2015-12-04
can be used for faster mounting at the cost of lower brute-force can be used for faster mounting at the cost of lower brute-force
resistance. It was mainly added to speed up the automated tests. resistance. It was mainly added to speed up the automated tests.
v0.4, 2015-11-15 #### v0.4, 2015-11-15
* New command-line options: * New command-line options:
* `-plaintextnames`: disables filename encryption, added on user request * `-plaintextnames`: disables filename encryption, added on user request
* `-extpass`: calls an external program for prompting for the password * `-extpass`: calls an external program for prompting for the password
@ -718,15 +721,15 @@ v0.4, 2015-11-15
format changes. The first user is `-plaintextnames`. format changes. The first user is `-plaintextnames`.
* On-disk format 2 * On-disk format 2
v0.3, 2015-11-01 #### v0.3, 2015-11-01
* **Add a random 128 bit file header to authenticate file->block ownership** * **Add a random 128 bit file header to authenticate file->block ownership**
* This is an on-disk-format change * This is an on-disk-format change
* On-disk format 1 * On-disk format 1
v0.2, 2015-10-11 #### v0.2, 2015-10-11
* Replace bash daemonization wrapper with native Go implementation * Replace bash daemonization wrapper with native Go implementation
* Better user feedback on mount failures * Better user feedback on mount failures
v0.1, 2015-10-07 #### v0.1, 2015-10-07
* First release * First release
* On-disk format 0 * On-disk format 0

View File

@ -14,18 +14,18 @@ PLAIN=linux-3.0
SIZE=0 SIZE=0
if [[ -d $PLAIN ]]; then if [[ -d $PLAIN ]]; then
SIZE=$(du -s --apparent-size $PLAIN | cut -f1) SIZE=$(du -s --apparent-size "$PLAIN" | cut -f1)
fi fi
if [[ $SIZE -ne 412334 ]] ; then if [[ $SIZE -ne 412334 ]] ; then
echo "Extracting linux-3.0.tar.gz" echo "Extracting linux-3.0.tar.gz"
rm -Rf $PLAIN rm -Rf "$PLAIN"
tar xf linux-3.0.tar.gz tar xf linux-3.0.tar.gz
fi fi
rm -f $PLAIN/.gocryptfs.reverse.conf rm -f "$PLAIN/.gocryptfs.reverse.conf"
gocryptfs -q -init -reverse -extpass="echo test" -scryptn=10 $PLAIN gocryptfs -q -init -reverse -extpass="echo test" -scryptn=10 "$PLAIN"
MNT=$(mktemp -d /tmp/linux-3.0.reverse.mnt.XXX) MNT=$(mktemp -d /tmp/linux-3.0.reverse.mnt.XXX)
@ -37,7 +37,7 @@ gocryptfs -q -reverse -extpass="echo test" "$PLAIN" "$MNT"
# Execute command, discard all stdout output, print elapsed time # Execute command, discard all stdout output, print elapsed time
# (to stderr, unfortunately). # (to stderr, unfortunately).
function etime { etime() {
# Make the bash builtin "time" print out only the elapse wall clock # Make the bash builtin "time" print out only the elapse wall clock
# seconds # seconds
TIMEFORMAT=%R TIMEFORMAT=%R

View File

@ -7,7 +7,7 @@ cd "$(dirname "$0")"
MYNAME=$(basename "$0") MYNAME=$(basename "$0")
source tests/fuse-unmount.bash source tests/fuse-unmount.bash
function usage { usage() {
echo "Usage: $MYNAME [-encfs] [-openssl=true] [-openssl=false] [-dd] [DIR]" echo "Usage: $MYNAME [-encfs] [-openssl=true] [-openssl=false] [-dd] [DIR]"
} }
@ -104,4 +104,3 @@ if [[ $DD_ONLY -eq 1 ]]; then
else else
./tests/canonical-benchmarks.bash "$MNT" ./tests/canonical-benchmarks.bash "$MNT"
fi fi

View File

@ -12,6 +12,9 @@
cd "$(dirname "$0")" cd "$(dirname "$0")"
# $0 does not work because we may have been sourced
MYNAME=build.bash
# Make sure we have the go binary # Make sure we have the go binary
go version > /dev/null go version > /dev/null
@ -28,7 +31,7 @@ if [[ -d .git ]] ; then
elif [[ -f VERSION ]] ; then elif [[ -f VERSION ]] ; then
GITVERSION=$(cat VERSION) GITVERSION=$(cat VERSION)
else else
echo "Warning: could not determine gocryptfs version" echo "$MYNAME: warning: could not determine gocryptfs version"
GITVERSION="[unknown]" GITVERSION="[unknown]"
fi fi
@ -42,7 +45,7 @@ else
if [[ $FAIL -eq 0 ]]; then if [[ $FAIL -eq 0 ]]; then
GITVERSIONFUSE=$OUT GITVERSIONFUSE=$OUT
else else
echo "Warning: could not determine go-fuse version" echo "$MYNAME: warning: could not determine go-fuse version"
GITVERSIONFUSE="[unknown]" GITVERSIONFUSE="[unknown]"
fi fi
fi fi
@ -56,7 +59,10 @@ fi
# If SOURCE_DATE_EPOCH is set, it overrides BUILDDATE. This is the # If SOURCE_DATE_EPOCH is set, it overrides BUILDDATE. This is the
# standard environment variable for faking the date in reproducible builds. # standard environment variable for faking the date in reproducible builds.
if [[ -n ${SOURCE_DATE_EPOCH:-} ]] ; then if [[ -n ${SOURCE_DATE_EPOCH:-} ]] ; then
BUILDDATE=$(date --utc --date="@${SOURCE_DATE_EPOCH}" +%Y-%m-%d) if ! BUILDDATE=$(date -u --date="@${SOURCE_DATE_EPOCH}" +%Y-%m-%d) ; then
echo "$MYNAME: info: retrying with BSD date syntax..."
BUILDDATE=$(date -u -r "$SOURCE_DATE_EPOCH" +%Y-%m-%d)
fi
fi fi
# Only set GOFLAGS if it is not already set by the user # Only set GOFLAGS if it is not already set by the user

View File

@ -29,7 +29,7 @@ type argContainer struct {
debug, init, zerokey, fusedebug, openssl, passwd, fg, version, debug, init, zerokey, fusedebug, openssl, passwd, fg, version,
plaintextnames, quiet, nosyslog, wpanic, plaintextnames, quiet, nosyslog, wpanic,
longnames, allow_other, reverse, aessiv, nonempty, raw64, longnames, allow_other, reverse, aessiv, nonempty, raw64,
noprealloc, speed, hkdf, serialize_reads, forcedecode, hh, info, noprealloc, speed, hkdf, serialize_reads, hh, info,
sharedstorage, fsck, one_file_system, deterministic_names, sharedstorage, fsck, one_file_system, deterministic_names,
xchacha bool xchacha bool
// Mount options with opposites // Mount options with opposites
@ -172,8 +172,6 @@ func parseCliOpts(osArgs []string) (args argContainer) {
flagSet.BoolVar(&args.speed, "speed", false, "Run crypto speed test") flagSet.BoolVar(&args.speed, "speed", false, "Run crypto speed test")
flagSet.BoolVar(&args.hkdf, "hkdf", true, "Use HKDF as an additional key derivation step") flagSet.BoolVar(&args.hkdf, "hkdf", true, "Use HKDF as an additional key derivation step")
flagSet.BoolVar(&args.serialize_reads, "serialize_reads", false, "Try to serialize read operations") flagSet.BoolVar(&args.serialize_reads, "serialize_reads", false, "Try to serialize read operations")
flagSet.BoolVar(&args.forcedecode, "forcedecode", false, "Force decode of files even if integrity check fails."+
" Requires gocryptfs to be compiled with openssl support and implies -openssl true")
flagSet.BoolVar(&args.hh, "hh", false, "Show this long help text") flagSet.BoolVar(&args.hh, "hh", false, "Show this long help text")
flagSet.BoolVar(&args.info, "info", false, "Display information about CIPHERDIR") flagSet.BoolVar(&args.info, "info", false, "Display information about CIPHERDIR")
flagSet.BoolVar(&args.sharedstorage, "sharedstorage", false, "Make concurrent access to a shared CIPHERDIR safer") flagSet.BoolVar(&args.sharedstorage, "sharedstorage", false, "Make concurrent access to a shared CIPHERDIR safer")
@ -234,7 +232,8 @@ func parseCliOpts(osArgs []string) (args argContainer) {
{ {
var tmp bool var tmp bool
flagSet.BoolVar(&tmp, "nofail", false, "Ignored for /etc/fstab compatibility") flagSet.BoolVar(&tmp, "nofail", false, "Ignored for /etc/fstab compatibility")
flagSet.BoolVar(&tmp, "devrandom", false, "Deprecated (ignored for compatibility)") flagSet.BoolVar(&tmp, "devrandom", false, "Obsolete, ignored for compatibility")
flagSet.BoolVar(&tmp, "forcedecode", false, "Obsolete, ignored for compatibility")
} }
// Actual parsing // Actual parsing
@ -253,7 +252,11 @@ func parseCliOpts(osArgs []string) (args argContainer) {
} }
// "-openssl" needs some post-processing // "-openssl" needs some post-processing
if opensslAuto == "auto" { if opensslAuto == "auto" {
args.openssl = stupidgcm.PreferOpenSSL() if args.xchacha {
args.openssl = stupidgcm.PreferOpenSSLXchacha20poly1305()
} else {
args.openssl = stupidgcm.PreferOpenSSLAES256GCM()
}
} else { } else {
args.openssl, err = strconv.ParseBool(opensslAuto) args.openssl, err = strconv.ParseBool(opensslAuto)
if err != nil { if err != nil {
@ -261,32 +264,6 @@ func parseCliOpts(osArgs []string) (args argContainer) {
os.Exit(exitcodes.Usage) os.Exit(exitcodes.Usage)
} }
} }
// "-forcedecode" only works with openssl. Check compilation and command line parameters
if args.forcedecode {
if stupidgcm.BuiltWithoutOpenssl {
tlog.Fatal.Printf("The -forcedecode flag requires openssl support, but gocryptfs was compiled without it!")
os.Exit(exitcodes.Usage)
}
if args.aessiv {
tlog.Fatal.Printf("The -forcedecode and -aessiv flags are incompatible because they use different crypto libs (openssl vs native Go)")
os.Exit(exitcodes.Usage)
}
if args.reverse {
tlog.Fatal.Printf("The reverse mode and the -forcedecode option are not compatible")
os.Exit(exitcodes.Usage)
}
// Has the user explicitly disabled openssl using "-openssl=false/0"?
if !args.openssl && opensslAuto != "auto" {
tlog.Fatal.Printf("-forcedecode requires openssl, but is disabled via command-line option")
os.Exit(exitcodes.Usage)
}
args.openssl = true
// Try to make it harder for the user to shoot himself in the foot.
args.ro = true
args.allow_other = false
args.ko = "noexec"
}
if len(args.extpass) > 0 && len(args.passfile) != 0 { if len(args.extpass) > 0 && len(args.passfile) != 0 {
tlog.Fatal.Printf("The options -extpass and -passfile cannot be used at the same time") tlog.Fatal.Printf("The options -extpass and -passfile cannot be used at the same time")
os.Exit(exitcodes.Usage) os.Exit(exitcodes.Usage)

View File

@ -119,7 +119,7 @@ func TestParseCliOpts(t *testing.T) {
longnames: true, longnames: true,
raw64: true, raw64: true,
hkdf: true, hkdf: true,
openssl: stupidgcm.PreferOpenSSL(), // depends on CPU and build flags openssl: stupidgcm.PreferOpenSSLAES256GCM(), // depends on CPU and build flags
scryptn: 16, scryptn: 16,
} }

View File

@ -1,6 +1,6 @@
/* /*
Small tool to try to debug unix.Getdents problems on CIFS mounts Small tool to try to debug unix.Getdents problems on CIFS mounts
( https://github.com/rfjakob/gocryptfs/v2/issues/483 ) ( https://github.com/rfjakob/gocryptfs/issues/483 )
Example output: Example output:

View File

@ -1,6 +1,6 @@
/* /*
Small tool to try to debug unix.Getdents problems on CIFS mounts Small tool to try to debug unix.Getdents problems on CIFS mounts
( https://github.com/rfjakob/gocryptfs/v2/issues/483 ) ( https://github.com/rfjakob/gocryptfs/issues/483 )
Example output: Example output:

View File

@ -12,7 +12,7 @@
# Note that pam_mount ignores messages on stdout which is why printing # Note that pam_mount ignores messages on stdout which is why printing
# to stdout is ok. # to stdout is ok.
set -eu set -eu
MYNAME=$(basename $0) MYNAME=$(basename "$0")
if [[ $# -lt 2 || $1 == -* ]]; then if [[ $# -lt 2 || $1 == -* ]]; then
echo "Usage: $MYNAME CIPHERDIR MOUNTPOINT [-o COMMA-SEPARATED-OPTIONS]" >&2 echo "Usage: $MYNAME CIPHERDIR MOUNTPOINT [-o COMMA-SEPARATED-OPTIONS]" >&2
exit 1 exit 1

View File

@ -32,8 +32,8 @@ while true ; do
echo "error: file $PWD/$NEXT already exists" echo "error: file $PWD/$NEXT already exists"
exit 1 exit 1
fi fi
echo -n 2> /dev/null > $NEXT || break echo -n 2> /dev/null > "$NEXT" || break
rm $NEXT rm "$NEXT"
NAME="$NEXT" NAME="$NEXT"
done done
echo "${#NAME}" echo "${#NAME}"
@ -47,8 +47,8 @@ if [[ $QUICK -ne 1 ]]; then
NAME="" NAME=""
while true ; do while true ; do
NEXT="${NAME}x" NEXT="${NAME}x"
mkdir $NEXT 2> /dev/null || break mkdir "$NEXT" 2> /dev/null || break
rmdir $NEXT rmdir "$NEXT"
NAME="$NEXT" NAME="$NEXT"
done done
MAX_DIRNAME=${#NAME} MAX_DIRNAME=${#NAME}

View File

@ -2,7 +2,7 @@
MNT=/mnt/ext4-ramdisk MNT=/mnt/ext4-ramdisk
if mountpoint $MNT ; then if mountpoint "$MNT" ; then
exit 1 exit 1
fi fi

2
go.mod
View File

@ -3,7 +3,7 @@ module github.com/rfjakob/gocryptfs/v2
go 1.16 go 1.16
require ( require (
github.com/hanwen/go-fuse/v2 v2.1.1-0.20210825070001-74a933d6e856 github.com/hanwen/go-fuse/v2 v2.1.1-0.20210825171523-3ab5d95a30ae
github.com/jacobsa/crypto v0.0.0-20190317225127-9f44e2d11115 github.com/jacobsa/crypto v0.0.0-20190317225127-9f44e2d11115
github.com/jacobsa/oglematchers v0.0.0-20150720000706-141901ea67cd // indirect github.com/jacobsa/oglematchers v0.0.0-20150720000706-141901ea67cd // indirect
github.com/jacobsa/oglemock v0.0.0-20150831005832-e94d794d06ff // indirect github.com/jacobsa/oglemock v0.0.0-20150831005832-e94d794d06ff // indirect

4
go.sum
View File

@ -1,7 +1,7 @@
github.com/davecgh/go-spew v1.1.0 h1:ZDRjVQ15GmhC3fiQ8ni8+OwkZQO4DARzQgrnXU1Liz8= github.com/davecgh/go-spew v1.1.0 h1:ZDRjVQ15GmhC3fiQ8ni8+OwkZQO4DARzQgrnXU1Liz8=
github.com/davecgh/go-spew v1.1.0/go.mod h1:J7Y8YcW2NihsgmVo/mv3lAwl/skON4iLHjSsI+c5H38= github.com/davecgh/go-spew v1.1.0/go.mod h1:J7Y8YcW2NihsgmVo/mv3lAwl/skON4iLHjSsI+c5H38=
github.com/hanwen/go-fuse/v2 v2.1.1-0.20210825070001-74a933d6e856 h1:rQb7H5igQ2oIeT+Ul1UtIsGhUiSGeCoyLg84otnHdXU= github.com/hanwen/go-fuse/v2 v2.1.1-0.20210825171523-3ab5d95a30ae h1:4CB6T4YTUVvnro5ba8ju1QCbOuyGAeF3vvKlo50EJ4k=
github.com/hanwen/go-fuse/v2 v2.1.1-0.20210825070001-74a933d6e856/go.mod h1:B1nGE/6RBFyBRC1RRnf23UpwCdyJ31eukw34oAKukAc= github.com/hanwen/go-fuse/v2 v2.1.1-0.20210825171523-3ab5d95a30ae/go.mod h1:B1nGE/6RBFyBRC1RRnf23UpwCdyJ31eukw34oAKukAc=
github.com/jacobsa/crypto v0.0.0-20190317225127-9f44e2d11115 h1:YuDUUFNM21CAbyPOpOP8BicaTD/0klJEKt5p8yuw+uY= github.com/jacobsa/crypto v0.0.0-20190317225127-9f44e2d11115 h1:YuDUUFNM21CAbyPOpOP8BicaTD/0klJEKt5p8yuw+uY=
github.com/jacobsa/crypto v0.0.0-20190317225127-9f44e2d11115/go.mod h1:LadVJg0XuawGk+8L1rYnIED8451UyNxEMdTWCEt5kmU= github.com/jacobsa/crypto v0.0.0-20190317225127-9f44e2d11115/go.mod h1:LadVJg0XuawGk+8L1rYnIED8451UyNxEMdTWCEt5kmU=
github.com/jacobsa/oglematchers v0.0.0-20150720000706-141901ea67cd h1:9GCSedGjMcLZCrusBZuo4tyKLpKUPenUUqi34AkuFmA= github.com/jacobsa/oglematchers v0.0.0-20150720000706-141901ea67cd h1:9GCSedGjMcLZCrusBZuo4tyKLpKUPenUUqi34AkuFmA=

View File

@ -39,7 +39,7 @@ func errExit(err error) {
func prettyPrintHeader(h *contentenc.FileHeader, algo cryptocore.AEADTypeEnum) { func prettyPrintHeader(h *contentenc.FileHeader, algo cryptocore.AEADTypeEnum) {
id := hex.EncodeToString(h.ID) id := hex.EncodeToString(h.ID)
fmt.Printf("Header: Version: %d, Id: %s, assuming %s mode\n", h.Version, id, algo.Name) fmt.Printf("Header: Version: %d, Id: %s, assuming %s mode\n", h.Version, id, algo.Algo)
} }
// printVersion prints a version string like this: // printVersion prints a version string like this:

View File

@ -1,3 +1,3 @@
Header: Version: 2, Id: 8932adf303fe0289679d47fa84d2b241, assuming AES-GCM-256-Go mode Header: Version: 2, Id: 8932adf303fe0289679d47fa84d2b241, assuming AES-GCM-256 mode
Block 0: IV: c8536b4bfd92f5dc3c1e2ac29f116d4a, Tag: 22b20422749b2f4bba67ec7d3bb1ac34, Offset: 18 Len: 4128 Block 0: IV: c8536b4bfd92f5dc3c1e2ac29f116d4a, Tag: 22b20422749b2f4bba67ec7d3bb1ac34, Offset: 18 Len: 4128
Block 1: IV: 2de68f4965779bb137ef2b3c20453556, Tag: 3e8758d6872234b1fffab2504e623467, Offset: 4146 Len: 936 Block 1: IV: 2de68f4965779bb137ef2b3c20453556, Tag: 3e8758d6872234b1fffab2504e623467, Offset: 4146 Len: 936

View File

@ -1,3 +1,3 @@
Header: Version: 2, Id: d839806747918e345633fcdd0988e67c, assuming AES-SIV-512-Go mode Header: Version: 2, Id: d839806747918e345633fcdd0988e67c, assuming AES-SIV-512 mode
Block 0: IV: 1d3ce2b13260f83766ccf9a670478a4b, Tag: 0b6f95bd523b4c93704e15ecc6bef8e7, Offset: 18 Len: 4128 Block 0: IV: 1d3ce2b13260f83766ccf9a670478a4b, Tag: 0b6f95bd523b4c93704e15ecc6bef8e7, Offset: 18 Len: 4128
Block 1: IV: 7eb947d2adf18adf3bed39bbc8052968, Tag: 1a272903e5a987f53f07344840387c20, Offset: 4146 Len: 936 Block 1: IV: 7eb947d2adf18adf3bed39bbc8052968, Tag: 1a272903e5a987f53f07344840387c20, Offset: 4146 Len: 936

24
info.go
View File

@ -1,44 +1,30 @@
package main package main
import ( import (
"encoding/json"
"fmt" "fmt"
"io/ioutil"
"os" "os"
"strings" "strings"
"github.com/rfjakob/gocryptfs/v2/internal/configfile" "github.com/rfjakob/gocryptfs/v2/internal/configfile"
"github.com/rfjakob/gocryptfs/v2/internal/contentenc"
"github.com/rfjakob/gocryptfs/v2/internal/exitcodes" "github.com/rfjakob/gocryptfs/v2/internal/exitcodes"
"github.com/rfjakob/gocryptfs/v2/internal/tlog"
) )
// info pretty-prints the contents of the config file at "filename" for human // info pretty-prints the contents of the config file at "filename" for human
// consumption, stripping out sensitive data. // consumption, stripping out sensitive data.
// This is called when you pass the "-info" option. // This is called when you pass the "-info" option.
func info(filename string) { func info(filename string) {
// Read from disk cf, err := configfile.Load(filename)
js, err := ioutil.ReadFile(filename)
if err != nil { if err != nil {
tlog.Fatal.Printf("Reading config file failed: %v", err) fmt.Printf("Loading config file failed: %v\n", err)
os.Exit(exitcodes.LoadConf)
}
// Unmarshal
var cf configfile.ConfFile
err = json.Unmarshal(js, &cf)
if err != nil {
tlog.Fatal.Printf("Failed to unmarshal config file")
os.Exit(exitcodes.LoadConf)
}
if cf.Version != contentenc.CurrentVersion {
tlog.Fatal.Printf("Unsupported on-disk format %d", cf.Version)
os.Exit(exitcodes.LoadConf) os.Exit(exitcodes.LoadConf)
} }
s := cf.ScryptObject
algo, _ := cf.ContentEncryption()
// Pretty-print // Pretty-print
fmt.Printf("Creator: %s\n", cf.Creator) fmt.Printf("Creator: %s\n", cf.Creator)
fmt.Printf("FeatureFlags: %s\n", strings.Join(cf.FeatureFlags, " ")) fmt.Printf("FeatureFlags: %s\n", strings.Join(cf.FeatureFlags, " "))
fmt.Printf("EncryptedKey: %dB\n", len(cf.EncryptedKey)) fmt.Printf("EncryptedKey: %dB\n", len(cf.EncryptedKey))
s := cf.ScryptObject
fmt.Printf("ScryptObject: Salt=%dB N=%d R=%d P=%d KeyLen=%d\n", fmt.Printf("ScryptObject: Salt=%dB N=%d R=%d P=%d KeyLen=%d\n",
len(s.Salt), s.N, s.R, s.P, s.KeyLen) len(s.Salt), s.N, s.R, s.P, s.KeyLen)
fmt.Printf("contentEncryption: %s\n", algo.Algo) // lowercase because not in JSON
} }

View File

@ -14,6 +14,7 @@ import (
"github.com/rfjakob/gocryptfs/v2/internal/fido2" "github.com/rfjakob/gocryptfs/v2/internal/fido2"
"github.com/rfjakob/gocryptfs/v2/internal/nametransform" "github.com/rfjakob/gocryptfs/v2/internal/nametransform"
"github.com/rfjakob/gocryptfs/v2/internal/readpassword" "github.com/rfjakob/gocryptfs/v2/internal/readpassword"
"github.com/rfjakob/gocryptfs/v2/internal/stupidgcm"
"github.com/rfjakob/gocryptfs/v2/internal/syscallcompat" "github.com/rfjakob/gocryptfs/v2/internal/syscallcompat"
"github.com/rfjakob/gocryptfs/v2/internal/tlog" "github.com/rfjakob/gocryptfs/v2/internal/tlog"
) )
@ -67,6 +68,11 @@ func initDir(args *argContainer) {
tlog.Fatal.Printf("Invalid cipherdir: %v", err) tlog.Fatal.Printf("Invalid cipherdir: %v", err)
os.Exit(exitcodes.CipherDir) os.Exit(exitcodes.CipherDir)
} }
if !args.xchacha && !stupidgcm.CpuHasAES() {
tlog.Info.Printf(tlog.ColorYellow +
"Notice: Your CPU does not have AES acceleration. Consider using -xchacha for better performance." +
tlog.ColorReset)
}
} }
// Choose password for config file // Choose password for config file
if len(args.extpass) == 0 && args.fido2 == "" { if len(args.extpass) == 0 && args.fido2 == "" {

View File

@ -276,7 +276,7 @@ func (cf *ConfFile) WriteFile() error {
err = fd.Sync() err = fd.Sync()
if err != nil { if err != nil {
// This can happen on network drives: FRITZ.NAS mounted on MacOS returns // This can happen on network drives: FRITZ.NAS mounted on MacOS returns
// "operation not supported": https://github.com/rfjakob/gocryptfs/v2/issues/390 // "operation not supported": https://github.com/rfjakob/gocryptfs/issues/390
tlog.Warn.Printf("Warning: fsync failed: %v", err) tlog.Warn.Printf("Warning: fsync failed: %v", err)
// Try sync instead // Try sync instead
syscall.Sync() syscall.Sync()
@ -298,8 +298,8 @@ func getKeyEncrypter(scryptHash []byte, useHKDF bool) *contentenc.ContentEnc {
if useHKDF { if useHKDF {
IVLen = contentenc.DefaultIVBits IVLen = contentenc.DefaultIVBits
} }
cc := cryptocore.New(scryptHash, cryptocore.BackendGoGCM, IVLen, useHKDF, false) cc := cryptocore.New(scryptHash, cryptocore.BackendGoGCM, IVLen, useHKDF)
ce := contentenc.New(cc, 4096, false) ce := contentenc.New(cc, 4096)
return ce return ce
} }

View File

@ -13,7 +13,6 @@ import (
"github.com/hanwen/go-fuse/v2/fuse" "github.com/hanwen/go-fuse/v2/fuse"
"github.com/rfjakob/gocryptfs/v2/internal/cryptocore" "github.com/rfjakob/gocryptfs/v2/internal/cryptocore"
"github.com/rfjakob/gocryptfs/v2/internal/stupidgcm"
"github.com/rfjakob/gocryptfs/v2/internal/tlog" "github.com/rfjakob/gocryptfs/v2/internal/tlog"
) )
@ -41,8 +40,6 @@ type ContentEnc struct {
allZeroBlock []byte allZeroBlock []byte
// All-zero block of size IVBitLen/8, for fast compares // All-zero block of size IVBitLen/8, for fast compares
allZeroNonce []byte allZeroNonce []byte
// Force decode even if integrity check fails (openSSL only)
forceDecode bool
// Ciphertext block "sync.Pool" pool. Always returns cipherBS-sized byte // Ciphertext block "sync.Pool" pool. Always returns cipherBS-sized byte
// slices (usually 4128 bytes). // slices (usually 4128 bytes).
@ -60,9 +57,8 @@ type ContentEnc struct {
} }
// New returns an initialized ContentEnc instance. // New returns an initialized ContentEnc instance.
func New(cc *cryptocore.CryptoCore, plainBS uint64, forceDecode bool) *ContentEnc { func New(cc *cryptocore.CryptoCore, plainBS uint64) *ContentEnc {
tlog.Debug.Printf("contentenc.New: plainBS=%d, forceDecode=%v", tlog.Debug.Printf("contentenc.New: plainBS=%d", plainBS)
plainBS, forceDecode)
if fuse.MAX_KERNEL_WRITE%plainBS != 0 { if fuse.MAX_KERNEL_WRITE%plainBS != 0 {
log.Panicf("unaligned MAX_KERNEL_WRITE=%d", fuse.MAX_KERNEL_WRITE) log.Panicf("unaligned MAX_KERNEL_WRITE=%d", fuse.MAX_KERNEL_WRITE)
@ -81,7 +77,6 @@ func New(cc *cryptocore.CryptoCore, plainBS uint64, forceDecode bool) *ContentEn
cipherBS: cipherBS, cipherBS: cipherBS,
allZeroBlock: make([]byte, cipherBS), allZeroBlock: make([]byte, cipherBS),
allZeroNonce: make([]byte, cc.IVLen), allZeroNonce: make([]byte, cc.IVLen),
forceDecode: forceDecode,
cBlockPool: newBPool(int(cipherBS)), cBlockPool: newBPool(int(cipherBS)),
CReqPool: newBPool(cReqSize), CReqPool: newBPool(cReqSize),
pBlockPool: newBPool(int(plainBS)), pBlockPool: newBPool(int(plainBS)),
@ -111,12 +106,8 @@ func (be *ContentEnc) DecryptBlocks(ciphertext []byte, firstBlockNo uint64, file
var pBlock []byte var pBlock []byte
pBlock, err = be.DecryptBlock(cBlock, blockNo, fileID) pBlock, err = be.DecryptBlock(cBlock, blockNo, fileID)
if err != nil { if err != nil {
if be.forceDecode && err == stupidgcm.ErrAuth {
tlog.Warn.Printf("DecryptBlocks: authentication failure in block #%d, overridden by forcedecode", firstBlockNo)
} else {
break break
} }
}
pBuf.Write(pBlock) pBuf.Write(pBlock)
be.pBlockPool.Put(pBlock) be.pBlockPool.Put(pBlock)
blockNo++ blockNo++
@ -167,7 +158,7 @@ func (be *ContentEnc) DecryptBlock(ciphertext []byte, blockNo uint64, fileID []b
nonce := ciphertext[:be.cryptoCore.IVLen] nonce := ciphertext[:be.cryptoCore.IVLen]
if bytes.Equal(nonce, be.allZeroNonce) { if bytes.Equal(nonce, be.allZeroNonce) {
// Bug in tmpfs? // Bug in tmpfs?
// https://github.com/rfjakob/gocryptfs/v2/issues/56 // https://github.com/rfjakob/gocryptfs/issues/56
// http://www.spinics.net/lists/kernel/msg2370127.html // http://www.spinics.net/lists/kernel/msg2370127.html
return nil, errors.New("all-zero nonce") return nil, errors.New("all-zero nonce")
} }
@ -183,9 +174,6 @@ func (be *ContentEnc) DecryptBlock(ciphertext []byte, blockNo uint64, fileID []b
if err != nil { if err != nil {
tlog.Debug.Printf("DecryptBlock: %s, len=%d", err.Error(), len(ciphertextOrig)) tlog.Debug.Printf("DecryptBlock: %s, len=%d", err.Error(), len(ciphertextOrig))
tlog.Debug.Println(hex.Dump(ciphertextOrig)) tlog.Debug.Println(hex.Dump(ciphertextOrig))
if be.forceDecode && err == stupidgcm.ErrAuth {
return plaintext, err
}
return nil, err return nil, err
} }

View File

@ -23,8 +23,8 @@ func TestSplitRange(t *testing.T) {
testRange{6654, 8945}) testRange{6654, 8945})
key := make([]byte, cryptocore.KeyLen) key := make([]byte, cryptocore.KeyLen)
cc := cryptocore.New(key, cryptocore.BackendGoGCM, DefaultIVBits, true, false) cc := cryptocore.New(key, cryptocore.BackendGoGCM, DefaultIVBits, true)
f := New(cc, DefaultBS, false) f := New(cc, DefaultBS)
for _, r := range ranges { for _, r := range ranges {
parts := f.ExplodePlainRange(r.offset, r.length) parts := f.ExplodePlainRange(r.offset, r.length)
@ -51,8 +51,8 @@ func TestCiphertextRange(t *testing.T) {
testRange{6654, 8945}) testRange{6654, 8945})
key := make([]byte, cryptocore.KeyLen) key := make([]byte, cryptocore.KeyLen)
cc := cryptocore.New(key, cryptocore.BackendGoGCM, DefaultIVBits, true, false) cc := cryptocore.New(key, cryptocore.BackendGoGCM, DefaultIVBits, true)
f := New(cc, DefaultBS, false) f := New(cc, DefaultBS)
for _, r := range ranges { for _, r := range ranges {
@ -74,8 +74,8 @@ func TestCiphertextRange(t *testing.T) {
func TestBlockNo(t *testing.T) { func TestBlockNo(t *testing.T) {
key := make([]byte, cryptocore.KeyLen) key := make([]byte, cryptocore.KeyLen)
cc := cryptocore.New(key, cryptocore.BackendGoGCM, DefaultIVBits, true, false) cc := cryptocore.New(key, cryptocore.BackendGoGCM, DefaultIVBits, true)
f := New(cc, DefaultBS, false) f := New(cc, DefaultBS)
b := f.CipherOffToBlockNo(788) b := f.CipherOffToBlockNo(788)
if b != 0 { if b != 0 {

View File

@ -10,8 +10,8 @@ import (
// TestSizeToSize tests CipherSizeToPlainSize and PlainSizeToCipherSize // TestSizeToSize tests CipherSizeToPlainSize and PlainSizeToCipherSize
func TestSizeToSize(t *testing.T) { func TestSizeToSize(t *testing.T) {
key := make([]byte, cryptocore.KeyLen) key := make([]byte, cryptocore.KeyLen)
cc := cryptocore.New(key, cryptocore.BackendGoGCM, DefaultIVBits, true, false) cc := cryptocore.New(key, cryptocore.BackendGoGCM, DefaultIVBits, true)
ce := New(cc, DefaultBS, false) ce := New(cc, DefaultBS)
const rangeMax = 10000 const rangeMax = 10000

View File

@ -28,25 +28,36 @@ const (
// AEADTypeEnum indicates the type of AEAD backend in use. // AEADTypeEnum indicates the type of AEAD backend in use.
type AEADTypeEnum struct { type AEADTypeEnum struct {
Name string // Algo is the encryption algorithm. Example: "AES-GCM-256"
Algo string
// Lib is the library where Algo is implemented. Either "Go" or "OpenSSL".
Lib string
NonceSize int NonceSize int
} }
// BackendOpenSSL specifies the OpenSSL backend. // String returns something like "AES-GCM-256-OpenSSL"
// "AES-GCM-256-OpenSSL" in gocryptfs -speed. func (a AEADTypeEnum) String() string {
var BackendOpenSSL AEADTypeEnum = AEADTypeEnum{"AES-GCM-256-OpenSSL", 16} return a.Algo + "-" + a.Lib
}
// BackendGoGCM specifies the Go based GCM backend. // BackendOpenSSL specifies the OpenSSL AES-256-GCM backend.
// "AES-GCM-256-OpenSSL" in gocryptfs -speed.
var BackendOpenSSL = AEADTypeEnum{"AES-GCM-256", "OpenSSL", 16}
// BackendGoGCM specifies the Go based AES-256-GCM backend.
// "AES-GCM-256-Go" in gocryptfs -speed. // "AES-GCM-256-Go" in gocryptfs -speed.
var BackendGoGCM AEADTypeEnum = AEADTypeEnum{"AES-GCM-256-Go", 16} var BackendGoGCM = AEADTypeEnum{"AES-GCM-256", "Go", 16}
// BackendAESSIV specifies an AESSIV backend. // BackendAESSIV specifies an AESSIV backend.
// "AES-SIV-512-Go" in gocryptfs -speed. // "AES-SIV-512-Go" in gocryptfs -speed.
var BackendAESSIV AEADTypeEnum = AEADTypeEnum{"AES-SIV-512-Go", siv_aead.NonceSize} var BackendAESSIV = AEADTypeEnum{"AES-SIV-512", "Go", siv_aead.NonceSize}
// BackendXChaCha20Poly1305 specifies XChaCha20-Poly1305-Go. // BackendXChaCha20Poly1305 specifies XChaCha20-Poly1305-Go.
// "XChaCha20-Poly1305-Go" in gocryptfs -speed. // "XChaCha20-Poly1305-Go" in gocryptfs -speed.
var BackendXChaCha20Poly1305 AEADTypeEnum = AEADTypeEnum{"XChaCha20-Poly1305-Go", chacha20poly1305.NonceSizeX} var BackendXChaCha20Poly1305 = AEADTypeEnum{"XChaCha20-Poly1305", "Go", chacha20poly1305.NonceSizeX}
// BackendXChaCha20Poly1305OpenSSL specifies XChaCha20-Poly1305-OpenSSL.
var BackendXChaCha20Poly1305OpenSSL = AEADTypeEnum{"XChaCha20-Poly1305", "OpenSSL", chacha20poly1305.NonceSizeX}
// CryptoCore is the low level crypto implementation. // CryptoCore is the low level crypto implementation.
type CryptoCore struct { type CryptoCore struct {
@ -70,9 +81,9 @@ type CryptoCore struct {
// //
// Note: "key" is either the scrypt hash of the password (when decrypting // Note: "key" is either the scrypt hash of the password (when decrypting
// a config file) or the masterkey (when finally mounting the filesystem). // a config file) or the masterkey (when finally mounting the filesystem).
func New(key []byte, aeadType AEADTypeEnum, IVBitLen int, useHKDF bool, forceDecode bool) *CryptoCore { func New(key []byte, aeadType AEADTypeEnum, IVBitLen int, useHKDF bool) *CryptoCore {
tlog.Debug.Printf("cryptocore.New: key=%d bytes, aeadType=%v, IVBitLen=%d, useHKDF=%v, forceDecode=%v", tlog.Debug.Printf("cryptocore.New: key=%d bytes, aeadType=%v, IVBitLen=%d, useHKDF=%v",
len(key), aeadType, IVBitLen, useHKDF, forceDecode) len(key), aeadType, IVBitLen, useHKDF)
if len(key) != KeyLen { if len(key) != KeyLen {
log.Panicf("Unsupported key length of %d bytes", len(key)) log.Panicf("Unsupported key length of %d bytes", len(key))
@ -117,7 +128,7 @@ func New(key []byte, aeadType AEADTypeEnum, IVBitLen int, useHKDF bool, forceDec
if IVBitLen != 128 { if IVBitLen != 128 {
log.Panicf("stupidgcm only supports 128-bit IVs, you wanted %d", IVBitLen) log.Panicf("stupidgcm only supports 128-bit IVs, you wanted %d", IVBitLen)
} }
aeadCipher = stupidgcm.New(gcmKey, forceDecode) aeadCipher = stupidgcm.NewAES256GCM(gcmKey)
case BackendGoGCM: case BackendGoGCM:
goGcmBlockCipher, err := aes.NewCipher(gcmKey) goGcmBlockCipher, err := aes.NewCipher(gcmKey)
if err != nil { if err != nil {
@ -127,6 +138,8 @@ func New(key []byte, aeadType AEADTypeEnum, IVBitLen int, useHKDF bool, forceDec
if err != nil { if err != nil {
log.Panic(err) log.Panic(err)
} }
default:
log.Panicf("BUG: unhandled case: %v", aeadType)
} }
for i := range gcmKey { for i := range gcmKey {
gcmKey[i] = 0 gcmKey[i] = 0
@ -151,7 +164,7 @@ func New(key []byte, aeadType AEADTypeEnum, IVBitLen int, useHKDF bool, forceDec
for i := range key64 { for i := range key64 {
key64[i] = 0 key64[i] = 0
} }
} else if aeadType == BackendXChaCha20Poly1305 { } else if aeadType == BackendXChaCha20Poly1305 || aeadType == BackendXChaCha20Poly1305OpenSSL {
// We don't support legacy modes with XChaCha20-Poly1305 // We don't support legacy modes with XChaCha20-Poly1305
if IVBitLen != chacha20poly1305.NonceSizeX*8 { if IVBitLen != chacha20poly1305.NonceSizeX*8 {
log.Panicf("XChaCha20-Poly1305 must use 192-bit IVs, you wanted %d", IVBitLen) log.Panicf("XChaCha20-Poly1305 must use 192-bit IVs, you wanted %d", IVBitLen)
@ -160,12 +173,18 @@ func New(key []byte, aeadType AEADTypeEnum, IVBitLen int, useHKDF bool, forceDec
log.Panic("XChaCha20-Poly1305 must use HKDF, but it is disabled") log.Panic("XChaCha20-Poly1305 must use HKDF, but it is disabled")
} }
derivedKey := hkdfDerive(key, hkdfInfoXChaChaPoly1305Content, chacha20poly1305.KeySize) derivedKey := hkdfDerive(key, hkdfInfoXChaChaPoly1305Content, chacha20poly1305.KeySize)
if aeadType == BackendXChaCha20Poly1305 {
aeadCipher, err = chacha20poly1305.NewX(derivedKey) aeadCipher, err = chacha20poly1305.NewX(derivedKey)
} else if aeadType == BackendXChaCha20Poly1305OpenSSL {
aeadCipher = stupidgcm.NewXchacha20poly1305(derivedKey)
} else {
log.Panicf("BUG: unhandled case: %v", aeadType)
}
if err != nil { if err != nil {
log.Panic(err) log.Panic(err)
} }
} else { } else {
log.Panicf("unknown cipher backend %q", aeadType.Name) log.Panicf("unknown cipher backend %q", aeadType)
} }
if aeadCipher.NonceSize()*8 != IVBitLen { if aeadCipher.NonceSize()*8 != IVBitLen {
@ -194,7 +213,7 @@ type wiper interface {
func (c *CryptoCore) Wipe() { func (c *CryptoCore) Wipe() {
be := c.AEADBackend be := c.AEADBackend
if be == BackendOpenSSL || be == BackendAESSIV { if be == BackendOpenSSL || be == BackendAESSIV {
tlog.Debug.Printf("CryptoCore.Wipe: Wiping AEADBackend %s key", be.Name) tlog.Debug.Printf("CryptoCore.Wipe: Wiping AEADBackend %q key", be)
// We don't use "x, ok :=" because we *want* to crash loudly if the // We don't use "x, ok :=" because we *want* to crash loudly if the
// type assertion fails. // type assertion fails.
w := c.AEADCipher.(wiper) w := c.AEADCipher.(wiper)

View File

@ -10,18 +10,18 @@ import (
func TestCryptoCoreNew(t *testing.T) { func TestCryptoCoreNew(t *testing.T) {
key := make([]byte, 32) key := make([]byte, 32)
for _, useHKDF := range []bool{true, false} { for _, useHKDF := range []bool{true, false} {
c := New(key, BackendGoGCM, 96, useHKDF, false) c := New(key, BackendGoGCM, 96, useHKDF)
if c.IVLen != 12 { if c.IVLen != 12 {
t.Fail() t.Fail()
} }
c = New(key, BackendGoGCM, 128, useHKDF, false) c = New(key, BackendGoGCM, 128, useHKDF)
if c.IVLen != 16 { if c.IVLen != 16 {
t.Fail() t.Fail()
} }
if stupidgcm.BuiltWithoutOpenssl { if stupidgcm.BuiltWithoutOpenssl {
continue continue
} }
c = New(key, BackendOpenSSL, 128, useHKDF, false) c = New(key, BackendOpenSSL, 128, useHKDF)
if c.IVLen != 16 { if c.IVLen != 16 {
t.Fail() t.Fail()
} }
@ -37,5 +37,5 @@ func TestNewPanic(t *testing.T) {
}() }()
key := make([]byte, 16) key := make([]byte, 16)
New(key, BackendOpenSSL, 128, true, false) New(key, BackendOpenSSL, 128, true)
} }

View File

@ -25,7 +25,7 @@
// l-wx------. 1 jakob jakob 64 Jan 5 15:54 3 -> /dev/null // l-wx------. 1 jakob jakob 64 Jan 5 15:54 3 -> /dev/null
// lrwx------. 1 jakob jakob 64 Jan 5 15:54 4 -> 'anon_inode:[eventpoll]' // lrwx------. 1 jakob jakob 64 Jan 5 15:54 4 -> 'anon_inode:[eventpoll]'
// //
// See https://github.com/rfjakob/gocryptfs/v2/issues/320 for details. // See https://github.com/rfjakob/gocryptfs/issues/320 for details.
package ensurefds012 package ensurefds012
import ( import (

View File

@ -26,10 +26,6 @@ type Args struct {
ConfigCustom bool ConfigCustom bool
// NoPrealloc disables automatic preallocation before writing // NoPrealloc disables automatic preallocation before writing
NoPrealloc bool NoPrealloc bool
// Try to serialize read operations, "-serialize_reads"
SerializeReads bool
// Force decode even if integrity check fails (openSSL only)
ForceDecode bool
// Exclude is a list of paths to make inaccessible, starting match at // Exclude is a list of paths to make inaccessible, starting match at
// the filesystem root // the filesystem root
Exclude []string Exclude []string
@ -42,7 +38,7 @@ type Args struct {
// Suid is true if the filesystem has been mounted with the "-suid" flag. // Suid is true if the filesystem has been mounted with the "-suid" flag.
// If it is false, we can ignore the GETXATTR "security.capability" calls, // If it is false, we can ignore the GETXATTR "security.capability" calls,
// which are a performance problem for writes. See // which are a performance problem for writes. See
// https://github.com/rfjakob/gocryptfs/v2/issues/515 for details. // https://github.com/rfjakob/gocryptfs/issues/515 for details.
Suid bool Suid bool
// Enable the FUSE kernel_cache option // Enable the FUSE kernel_cache option
KernelCache bool KernelCache bool

View File

@ -20,8 +20,6 @@ import (
"github.com/rfjakob/gocryptfs/v2/internal/contentenc" "github.com/rfjakob/gocryptfs/v2/internal/contentenc"
"github.com/rfjakob/gocryptfs/v2/internal/inomap" "github.com/rfjakob/gocryptfs/v2/internal/inomap"
"github.com/rfjakob/gocryptfs/v2/internal/openfiletable" "github.com/rfjakob/gocryptfs/v2/internal/openfiletable"
"github.com/rfjakob/gocryptfs/v2/internal/serialize_reads"
"github.com/rfjakob/gocryptfs/v2/internal/stupidgcm"
"github.com/rfjakob/gocryptfs/v2/internal/syscallcompat" "github.com/rfjakob/gocryptfs/v2/internal/syscallcompat"
"github.com/rfjakob/gocryptfs/v2/internal/tlog" "github.com/rfjakob/gocryptfs/v2/internal/tlog"
) )
@ -209,17 +207,10 @@ func (f *File) doRead(dst []byte, off uint64, length uint64) ([]byte, syscall.Er
plaintext, err := f.contentEnc.DecryptBlocks(ciphertext, firstBlockNo, fileID) plaintext, err := f.contentEnc.DecryptBlocks(ciphertext, firstBlockNo, fileID)
f.rootNode.contentEnc.CReqPool.Put(ciphertext) f.rootNode.contentEnc.CReqPool.Put(ciphertext)
if err != nil { if err != nil {
if f.rootNode.args.ForceDecode && err == stupidgcm.ErrAuth {
// We do not have the information which block was corrupt here anymore,
// but DecryptBlocks() has already logged it anyway.
tlog.Warn.Printf("doRead %d: off=%d len=%d: returning corrupt data due to forcedecode",
f.qIno.Ino, off, length)
} else {
curruptBlockNo := firstBlockNo + f.contentEnc.PlainOffToBlockNo(uint64(len(plaintext))) curruptBlockNo := firstBlockNo + f.contentEnc.PlainOffToBlockNo(uint64(len(plaintext)))
tlog.Warn.Printf("doRead %d: corrupt block #%d: %v", f.qIno.Ino, curruptBlockNo, err) tlog.Warn.Printf("doRead %d: corrupt block #%d: %v", f.qIno.Ino, curruptBlockNo, err)
return nil, syscall.EIO return nil, syscall.EIO
} }
}
// Crop down to the relevant part // Crop down to the relevant part
var out []byte var out []byte
@ -252,13 +243,7 @@ func (f *File) Read(ctx context.Context, buf []byte, off int64) (resultData fuse
defer f.fileTableEntry.ContentLock.RUnlock() defer f.fileTableEntry.ContentLock.RUnlock()
tlog.Debug.Printf("ino%d: FUSE Read: offset=%d length=%d", f.qIno.Ino, off, len(buf)) tlog.Debug.Printf("ino%d: FUSE Read: offset=%d length=%d", f.qIno.Ino, off, len(buf))
if f.rootNode.args.SerializeReads {
serialize_reads.Wait(off, len(buf))
}
out, errno := f.doRead(buf[:0], uint64(off), uint64(len(buf))) out, errno := f.doRead(buf[:0], uint64(off), uint64(len(buf)))
if f.rootNode.args.SerializeReads {
serialize_reads.Done()
}
if errno != 0 { if errno != 0 {
return nil, errno return nil, errno
} }

View File

@ -24,7 +24,7 @@ var xattrNameIV = []byte("xattr_name_iv_xx")
var xattrStorePrefix = "user.gocryptfs." var xattrStorePrefix = "user.gocryptfs."
// We get one read of this xattr for each write - // We get one read of this xattr for each write -
// see https://github.com/rfjakob/gocryptfs/v2/issues/515 for details. // see https://github.com/rfjakob/gocryptfs/issues/515 for details.
var xattrCapability = "security.capability" var xattrCapability = "security.capability"
// isAcl returns true if the attribute name is for storing ACLs // isAcl returns true if the attribute name is for storing ACLs
@ -41,7 +41,7 @@ func (n *Node) Getxattr(ctx context.Context, attr string, dest []byte) (uint32,
rn := n.rootNode() rn := n.rootNode()
// If we are not mounted with -suid, reading the capability xattr does not // If we are not mounted with -suid, reading the capability xattr does not
// make a lot of sense, so reject the request and gain a massive speedup. // make a lot of sense, so reject the request and gain a massive speedup.
// See https://github.com/rfjakob/gocryptfs/v2/issues/515 . // See https://github.com/rfjakob/gocryptfs/issues/515 .
if !rn.args.Suid && attr == xattrCapability { if !rn.args.Suid && attr == xattrCapability {
// Returning EOPNOTSUPP is what we did till // Returning EOPNOTSUPP is what we did till
// ca9e912a28b901387e1dbb85f6c531119f2d5ef2 "fusefrontend: drop xattr user namespace restriction" // ca9e912a28b901387e1dbb85f6c531119f2d5ef2 "fusefrontend: drop xattr user namespace restriction"

View File

@ -11,7 +11,6 @@ import (
"github.com/rfjakob/gocryptfs/v2/internal/contentenc" "github.com/rfjakob/gocryptfs/v2/internal/contentenc"
"github.com/rfjakob/gocryptfs/v2/internal/inomap" "github.com/rfjakob/gocryptfs/v2/internal/inomap"
"github.com/rfjakob/gocryptfs/v2/internal/nametransform" "github.com/rfjakob/gocryptfs/v2/internal/nametransform"
"github.com/rfjakob/gocryptfs/v2/internal/serialize_reads"
"github.com/rfjakob/gocryptfs/v2/internal/syscallcompat" "github.com/rfjakob/gocryptfs/v2/internal/syscallcompat"
"github.com/rfjakob/gocryptfs/v2/internal/tlog" "github.com/rfjakob/gocryptfs/v2/internal/tlog"
) )
@ -63,21 +62,28 @@ type RootNode struct {
} }
func NewRootNode(args Args, c *contentenc.ContentEnc, n *nametransform.NameTransform) *RootNode { func NewRootNode(args Args, c *contentenc.ContentEnc, n *nametransform.NameTransform) *RootNode {
if args.SerializeReads { var rootDev uint64
serialize_reads.InitSerializer() var st syscall.Stat_t
if err := syscall.Stat(args.Cipherdir, &st); err != nil {
tlog.Warn.Printf("Could not stat backing directory %q: %v", args.Cipherdir, err)
} else {
rootDev = uint64(st.Dev)
} }
if len(args.Exclude) > 0 { if len(args.Exclude) > 0 {
tlog.Warn.Printf("Forward mode does not support -exclude") tlog.Warn.Printf("Forward mode does not support -exclude")
} }
ivLen := nametransform.DirIVLen ivLen := nametransform.DirIVLen
if args.PlaintextNames { if args.PlaintextNames {
ivLen = 0 ivLen = 0
} }
rn := &RootNode{ rn := &RootNode{
args: args, args: args,
nameTransform: n, nameTransform: n,
contentEnc: c, contentEnc: c,
inoMap: inomap.New(), inoMap: inomap.New(rootDev),
dirCache: dirCache{ivLen: ivLen}, dirCache: dirCache{ivLen: ivLen},
quirks: syscallcompat.DetectQuirks(args.Cipherdir), quirks: syscallcompat.DetectQuirks(args.Cipherdir),
} }

View File

@ -17,8 +17,8 @@ import (
func newTestFS(args Args) *RootNode { func newTestFS(args Args) *RootNode {
// Init crypto backend // Init crypto backend
key := make([]byte, cryptocore.KeyLen) key := make([]byte, cryptocore.KeyLen)
cCore := cryptocore.New(key, cryptocore.BackendGoGCM, contentenc.DefaultIVBits, true, false) cCore := cryptocore.New(key, cryptocore.BackendGoGCM, contentenc.DefaultIVBits, true)
cEnc := contentenc.New(cCore, contentenc.DefaultBS, false) cEnc := contentenc.New(cCore, contentenc.DefaultBS)
n := nametransform.New(cCore.EMECipher, true, true, nil, false) n := nametransform.New(cCore.EMECipher, true, true, nil, false)
rn := NewRootNode(args, cEnc, n) rn := NewRootNode(args, cEnc, n)
oneSec := time.Second oneSec := time.Second

View File

@ -2,10 +2,13 @@ package fusefrontend_reverse
import ( import (
"log" "log"
"os"
"path/filepath" "path/filepath"
"strings" "strings"
"syscall" "syscall"
"github.com/rfjakob/gocryptfs/v2/internal/exitcodes"
"github.com/rfjakob/gocryptfs/v2/internal/tlog" "github.com/rfjakob/gocryptfs/v2/internal/tlog"
"golang.org/x/sys/unix" "golang.org/x/sys/unix"
@ -46,12 +49,14 @@ type RootNode struct {
// ReverseFS provides an encrypted view. // ReverseFS provides an encrypted view.
func NewRootNode(args fusefrontend.Args, c *contentenc.ContentEnc, n *nametransform.NameTransform) *RootNode { func NewRootNode(args fusefrontend.Args, c *contentenc.ContentEnc, n *nametransform.NameTransform) *RootNode {
var rootDev uint64 var rootDev uint64
if args.OneFileSystem {
var st syscall.Stat_t var st syscall.Stat_t
err := syscall.Stat(args.Cipherdir, &st) if err := syscall.Stat(args.Cipherdir, &st); err != nil {
if err != nil { tlog.Warn.Printf("Could not stat backing directory %q: %v", args.Cipherdir, err)
log.Panicf("Could not stat backing directory %q: %v", args.Cipherdir, err) if args.OneFileSystem {
tlog.Fatal.Printf("This is a fatal error in combination with -one-file-system")
os.Exit(exitcodes.CipherDir)
} }
} else {
rootDev = uint64(st.Dev) rootDev = uint64(st.Dev)
} }
@ -59,7 +64,7 @@ func NewRootNode(args fusefrontend.Args, c *contentenc.ContentEnc, n *nametransf
args: args, args: args,
nameTransform: n, nameTransform: n,
contentEnc: c, contentEnc: c,
inoMap: inomap.New(), inoMap: inomap.New(rootDev),
rootDev: rootDev, rootDev: rootDev,
} }
if len(args.Exclude) > 0 || len(args.ExcludeWildcard) > 0 || len(args.ExcludeFrom) > 0 { if len(args.Exclude) > 0 || len(args.ExcludeWildcard) > 0 || len(args.ExcludeFrom) > 0 {

View File

@ -49,13 +49,22 @@ type InoMap struct {
} }
// New returns a new InoMap. // New returns a new InoMap.
func New() *InoMap { // Inode numbers on device `rootDev` will be passed through as-is.
return &InoMap{ // If `rootDev` is zero, the first Translate() call decides the effective
// rootDev.
func New(rootDev uint64) *InoMap {
m := &InoMap{
namespaceMap: make(map[namespaceData]uint16), namespaceMap: make(map[namespaceData]uint16),
namespaceNext: 0, namespaceNext: 0,
spillMap: make(map[QIno]uint64), spillMap: make(map[QIno]uint64),
spillNext: 0, spillNext: 0,
} }
if rootDev > 0 {
// Reserve namespace 0 for rootDev
m.namespaceMap[namespaceData{rootDev, 0}] = 0
m.namespaceNext = 1
}
return m
} }
var spillWarn sync.Once var spillWarn sync.Once

View File

@ -6,7 +6,7 @@ import (
) )
func TestTranslate(t *testing.T) { func TestTranslate(t *testing.T) {
m := New() m := New(0)
q := QIno{Ino: 1} q := QIno{Ino: 1}
out := m.Translate(q) out := m.Translate(q)
if out != 1 { if out != 1 {
@ -25,12 +25,7 @@ func TestTranslate(t *testing.T) {
func TestTranslateStress(t *testing.T) { func TestTranslateStress(t *testing.T) {
const baseDev = 12345 const baseDev = 12345
m := New() m := New(baseDev)
// Make sure baseDev gets namespace id zero
var q QIno
q.Dev = baseDev
m.Translate(q)
var wg sync.WaitGroup var wg sync.WaitGroup
wg.Add(4) wg.Add(4)
@ -100,7 +95,7 @@ func TestTranslateStress(t *testing.T) {
} }
func TestSpill(t *testing.T) { func TestSpill(t *testing.T) {
m := New() m := New(0)
var q QIno var q QIno
q.Ino = maxPassthruIno + 1 q.Ino = maxPassthruIno + 1
out1 := m.Translate(q) out1 := m.Translate(q)
@ -119,7 +114,7 @@ func TestSpill(t *testing.T) {
// TestUniqueness checks that unique (Dev, Flags, Ino) tuples get unique inode // TestUniqueness checks that unique (Dev, Flags, Ino) tuples get unique inode
// numbers // numbers
func TestUniqueness(t *testing.T) { func TestUniqueness(t *testing.T) {
m := New() m := New(0)
var q QIno var q QIno
outMap := make(map[uint64]struct{}) outMap := make(map[uint64]struct{})
for q.Dev = 0; q.Dev < 10; q.Dev++ { for q.Dev = 0; q.Dev < 10; q.Dev++ {
@ -141,7 +136,7 @@ func TestUniqueness(t *testing.T) {
} }
func BenchmarkTranslateSingleDev(b *testing.B) { func BenchmarkTranslateSingleDev(b *testing.B) {
m := New() m := New(0)
var q QIno var q QIno
for n := 0; n < b.N; n++ { for n := 0; n < b.N; n++ {
q.Ino = uint64(n % 1000) q.Ino = uint64(n % 1000)
@ -150,7 +145,7 @@ func BenchmarkTranslateSingleDev(b *testing.B) {
} }
func BenchmarkTranslateManyDevs(b *testing.B) { func BenchmarkTranslateManyDevs(b *testing.B) {
m := New() m := New(0)
var q QIno var q QIno
for n := 0; n < b.N; n++ { for n := 0; n < b.N; n++ {
q.Dev = uint64(n % 10) q.Dev = uint64(n % 10)

View File

@ -68,7 +68,7 @@ func WriteDirIVAt(dirfd int) error {
iv := cryptocore.RandBytes(DirIVLen) iv := cryptocore.RandBytes(DirIVLen)
// 0400 permissions: gocryptfs.diriv should never be modified after creation. // 0400 permissions: gocryptfs.diriv should never be modified after creation.
// Don't use "ioutil.WriteFile", it causes trouble on NFS: // Don't use "ioutil.WriteFile", it causes trouble on NFS:
// https://github.com/rfjakob/gocryptfs/v2/commit/7d38f80a78644c8ec4900cc990bfb894387112ed // https://github.com/rfjakob/gocryptfs/commit/7d38f80a78644c8ec4900cc990bfb894387112ed
fd, err := syscallcompat.Openat(dirfd, DirIVFilename, os.O_WRONLY|os.O_CREATE|os.O_EXCL, dirivPerms) fd, err := syscallcompat.Openat(dirfd, DirIVFilename, os.O_WRONLY|os.O_CREATE|os.O_EXCL, dirivPerms)
if err != nil { if err != nil {
tlog.Warn.Printf("WriteDirIV: Openat: %v", err) tlog.Warn.Printf("WriteDirIV: Openat: %v", err)

View File

@ -6,14 +6,14 @@ const (
// never chmod'ed or chown'ed. // never chmod'ed or chown'ed.
// //
// Group-readable so the FS can be mounted by several users in the same group // Group-readable so the FS can be mounted by several users in the same group
// (see https://github.com/rfjakob/gocryptfs/v2/issues/387 ). // (see https://github.com/rfjakob/gocryptfs/issues/387 ).
// //
// Note that gocryptfs.conf is still created with 0400 permissions so the // Note that gocryptfs.conf is still created with 0400 permissions so the
// owner must explicitly chmod it to permit access. // owner must explicitly chmod it to permit access.
// //
// World-readable so an encrypted directory can be copied by the non-root // World-readable so an encrypted directory can be copied by the non-root
// owner when gocryptfs is running as root // owner when gocryptfs is running as root
// ( https://github.com/rfjakob/gocryptfs/v2/issues/539 ). // ( https://github.com/rfjakob/gocryptfs/issues/539 ).
dirivPerms = 0444 dirivPerms = 0444
// Permissions for gocryptfs.longname.[sha256].name files. // Permissions for gocryptfs.longname.[sha256].name files.

View File

@ -1,150 +0,0 @@
package serialize_reads
import (
"log"
"sync"
"time"
"github.com/rfjakob/gocryptfs/v2/internal/tlog"
)
// serializerState is used by the Wait and Done functions
type serializerState struct {
// we get submissions through the "input" channel
input chan *submission
// q = Queue
q []*submission
// wg is used to wait for the read to complete before unblocking the next
wg sync.WaitGroup
}
// Wait places the caller into a queue and blocks
func Wait(offset int64, size int) {
serializer.wait(offset, size)
}
// Done signals that the read operation has finished
func Done() {
serializer.wg.Done()
}
type submission struct {
// "ch" is closed by "eventLoop" once it wants to unblock the caller
ch chan struct{}
// submissions are prioritized by offset (lowest offset gets unblocked first)
offset int64
// size will be used in the future to detect consecutive read requests. These
// can be unblocked immediately.
size int
}
func (sr *serializerState) wait(offset int64, size int) {
ch := make(chan struct{})
sb := &submission{
ch: ch,
offset: offset,
size: size,
}
// Send our submission
sr.input <- sb
// Wait till we get unblocked
<-ch
}
// push returns true if the queue is full after the element has been stored.
// It panics if it did not have space to store the element.
func (sr *serializerState) push(sb *submission) (full bool) {
free := 0
stored := false
for i, v := range sr.q {
if v != nil {
continue
}
if !stored {
sr.q[i] = sb
stored = true
continue
}
free++
}
if !stored {
// This should never happen because eventLoop checks if the queue got full
log.Panic("BUG: unhandled queue overflow")
}
if free == 0 {
return true
}
return false
}
// pop the submission with the lowest offset off the queue
func (sr *serializerState) pop() *submission {
var winner *submission
var winnerIndex int
for i, v := range sr.q {
if v == nil {
continue
}
if winner == nil {
winner = v
winnerIndex = i
continue
}
if v.offset < winner.offset {
winner = v
winnerIndex = i
}
}
if winner == nil {
return nil
}
sr.q[winnerIndex] = nil
return winner
}
func (sr *serializerState) eventLoop() {
sr.input = make(chan *submission)
empty := true
for {
if empty {
// If the queue is empty we block on the channel to conserve CPU
sb := <-sr.input
sr.push(sb)
empty = false
}
select {
case sb := <-sr.input:
full := sr.push(sb)
if full {
// Queue is full, unblock the new request immediately
tlog.Warn.Printf("serialize_reads: queue full, forcing unblock")
sr.unblockOne()
}
case <-time.After(time.Microsecond * 500):
// Looks like we have waited out all concurrent requests.
empty = sr.unblockOne()
}
}
}
// Unblock a submission and wait for completion
func (sr *serializerState) unblockOne() (empty bool) {
winner := sr.pop()
if winner == nil {
return true
}
sr.wg.Add(1)
close(winner.ch)
sr.wg.Wait()
return false
}
var serializer serializerState
// InitSerializer sets up the internal serializer state and starts the event loop.
// Called by fusefrontend.NewFS.
func InitSerializer() {
serializer.input = make(chan *submission)
serializer.q = make([]*submission, 10)
go serializer.eventLoop()
}

54
internal/speed/cpuinfo.go Normal file
View File

@ -0,0 +1,54 @@
package speed
import (
"io/ioutil"
"os"
"runtime"
"strings"
)
// cpuModelName returns the "model name" acc. to /proc/cpuinfo, or ""
// on error.
//
// Examples: On my desktop PC:
//
// $ grep "model name" /proc/cpuinfo
// model name : Intel(R) Core(TM) i5-3470 CPU @ 3.20GHz
//
// --> Returns "Intel(R) Core(TM) i5-3470 CPU @ 3.20GHz".
//
// On a Raspberry Pi 4:
//
// $ grep "model name" /proc/cpuinfo
// (empty)
// $ grep Hardware /proc/cpuinfo
// Hardware : BCM2835
//
// --> Returns "BCM2835"
func cpuModelName() string {
if runtime.GOOS != "linux" {
return ""
}
f, err := os.Open("/proc/cpuinfo")
if err != nil {
return ""
}
content, err := ioutil.ReadAll(f)
if err != nil {
return ""
}
lines := strings.Split(string(content), "\n")
// Look for "model name", then for "Hardware" (arm devices don't have "model name")
for _, want := range []string{"model name", "Hardware"} {
for _, line := range lines {
if strings.HasPrefix(line, want) {
parts := strings.SplitN(line, ":", 2)
if len(parts) != 2 {
continue
}
return strings.TrimSpace(parts[1])
}
}
}
return ""
}

View File

@ -27,18 +27,29 @@ const blockSize = 4096
// Run - run the speed the test and print the results. // Run - run the speed the test and print the results.
func Run() { func Run() {
cpu := cpuModelName()
if cpu == "" {
cpu = "unknown"
}
aes := "; no AES acceleration"
if stupidgcm.CpuHasAES() {
aes = "; with AES acceleration"
}
fmt.Printf("cpu: %s%s\n", cpu, aes)
bTable := []struct { bTable := []struct {
name string name string
f func(*testing.B) f func(*testing.B)
preferred bool preferred bool
}{ }{
{name: cryptocore.BackendOpenSSL.Name, f: bStupidGCM, preferred: stupidgcm.PreferOpenSSL()}, {name: cryptocore.BackendOpenSSL.String(), f: bStupidGCM, preferred: stupidgcm.PreferOpenSSLAES256GCM()},
{name: cryptocore.BackendGoGCM.Name, f: bGoGCM, preferred: !stupidgcm.PreferOpenSSL()}, {name: cryptocore.BackendGoGCM.String(), f: bGoGCM, preferred: !stupidgcm.PreferOpenSSLAES256GCM()},
{name: cryptocore.BackendAESSIV.Name, f: bAESSIV, preferred: false}, {name: cryptocore.BackendAESSIV.String(), f: bAESSIV, preferred: false},
{name: cryptocore.BackendXChaCha20Poly1305.Name, f: bChacha20poly1305, preferred: false}, {name: cryptocore.BackendXChaCha20Poly1305OpenSSL.String(), f: bStupidXchacha, preferred: stupidgcm.PreferOpenSSLXchacha20poly1305()},
{name: cryptocore.BackendXChaCha20Poly1305.String(), f: bXchacha20poly1305, preferred: !stupidgcm.PreferOpenSSLXchacha20poly1305()},
} }
for _, b := range bTable { for _, b := range bTable {
fmt.Printf("%-20s\t", b.name) fmt.Printf("%-26s\t", b.name)
mbs := mbPerSec(testing.Benchmark(b.f)) mbs := mbPerSec(testing.Benchmark(b.f))
if mbs > 0 { if mbs > 0 {
fmt.Printf("%7.2f MB/s", mbs) fmt.Printf("%7.2f MB/s", mbs)
@ -47,10 +58,8 @@ func Run() {
} }
if b.preferred { if b.preferred {
fmt.Printf("\t(selected in auto mode)\n") fmt.Printf("\t(selected in auto mode)\n")
} else if b.name == cryptocore.BackendXChaCha20Poly1305.Name {
fmt.Printf("\t(use via -xchacha flag)\n")
} else { } else {
fmt.Printf("\t\n") fmt.Printf("\n")
} }
} }
} }
@ -72,35 +81,55 @@ func randBytes(n int) []byte {
return b return b
} }
// bEncrypt benchmarks the encryption speed of cipher "c"
func bEncrypt(b *testing.B, c cipher.AEAD) {
authData := randBytes(adLen)
iv := randBytes(c.NonceSize())
in := make([]byte, blockSize)
dst := make([]byte, len(in)+len(iv)+c.Overhead())
copy(dst, iv)
b.SetBytes(int64(len(in)))
b.ResetTimer()
for i := 0; i < b.N; i++ {
// Reset dst buffer
dst = dst[:len(iv)]
// Encrypt and append to nonce
c.Seal(dst, iv, in, authData)
}
}
func bDecrypt(b *testing.B, c cipher.AEAD) {
authData := randBytes(adLen)
iv := randBytes(c.NonceSize())
plain := randBytes(blockSize)
ciphertext := c.Seal(iv, iv, plain, authData)
b.SetBytes(int64(len(plain)))
b.ResetTimer()
for i := 0; i < b.N; i++ {
// Reset plain buffer
plain = plain[:0]
// Decrypt
_, err := c.Open(plain, iv, ciphertext[c.NonceSize():], authData)
if err != nil {
b.Fatal(err)
}
}
}
// bStupidGCM benchmarks stupidgcm's openssl GCM // bStupidGCM benchmarks stupidgcm's openssl GCM
func bStupidGCM(b *testing.B) { func bStupidGCM(b *testing.B) {
if stupidgcm.BuiltWithoutOpenssl { if stupidgcm.BuiltWithoutOpenssl {
b.Skip("openssl has been disabled at compile-time") b.Skip("openssl has been disabled at compile-time")
} }
key := randBytes(32) bEncrypt(b, stupidgcm.NewAES256GCM(randBytes(32)))
authData := randBytes(adLen)
iv := randBytes(16)
in := make([]byte, blockSize)
b.SetBytes(int64(len(in)))
sGCM := stupidgcm.New(key, false)
b.ResetTimer()
for i := 0; i < b.N; i++ {
// Encrypt and append to nonce
sGCM.Seal(iv, iv, in, authData)
}
} }
// bGoGCM benchmarks Go stdlib GCM // bGoGCM benchmarks Go stdlib GCM
func bGoGCM(b *testing.B) { func bGoGCM(b *testing.B) {
key := randBytes(32) gAES, err := aes.NewCipher(randBytes(32))
authData := randBytes(adLen)
iv := randBytes(16)
in := make([]byte, blockSize)
b.SetBytes(int64(len(in)))
gAES, err := aes.NewCipher(key)
if err != nil { if err != nil {
b.Fatal(err) b.Fatal(err)
} }
@ -108,42 +137,25 @@ func bGoGCM(b *testing.B) {
if err != nil { if err != nil {
b.Fatal(err) b.Fatal(err)
} }
bEncrypt(b, gGCM)
b.ResetTimer()
for i := 0; i < b.N; i++ {
// Encrypt and append to nonce
gGCM.Seal(iv, iv, in, authData)
}
} }
// bAESSIV benchmarks AES-SIV from github.com/jacobsa/crypto/siv // bAESSIV benchmarks AES-SIV from github.com/jacobsa/crypto/siv
func bAESSIV(b *testing.B) { func bAESSIV(b *testing.B) {
key := randBytes(64) c := siv_aead.New(randBytes(64))
authData := randBytes(adLen) bEncrypt(b, c)
iv := randBytes(16)
in := make([]byte, blockSize)
b.SetBytes(int64(len(in)))
gGCM := siv_aead.New(key)
b.ResetTimer()
for i := 0; i < b.N; i++ {
// Encrypt and append to nonce
gGCM.Seal(iv, iv, in, authData)
}
} }
// bChacha20poly1305 benchmarks XChaCha20 from golang.org/x/crypto/chacha20poly1305 // bXchacha20poly1305 benchmarks XChaCha20 from golang.org/x/crypto/chacha20poly1305
func bChacha20poly1305(b *testing.B) { func bXchacha20poly1305(b *testing.B) {
key := randBytes(32) c, _ := chacha20poly1305.NewX(randBytes(32))
authData := randBytes(adLen) bEncrypt(b, c)
iv := randBytes(chacha20poly1305.NonceSizeX) }
in := make([]byte, blockSize)
b.SetBytes(int64(len(in))) // bStupidXchacha benchmarks OpenSSL XChaCha20
c, _ := chacha20poly1305.NewX(key) func bStupidXchacha(b *testing.B) {
if stupidgcm.BuiltWithoutOpenssl {
b.ResetTimer() b.Skip("openssl has been disabled at compile-time")
for i := 0; i < b.N; i++ { }
// Encrypt and append to nonce bEncrypt(b, stupidgcm.NewXchacha20poly1305(randBytes(32)))
c.Seal(iv, iv, in, authData)
}
} }

View File

@ -1,5 +1,16 @@
package speed package speed
import (
"crypto/aes"
"crypto/cipher"
"testing"
"golang.org/x/crypto/chacha20poly1305"
"github.com/rfjakob/gocryptfs/v2/internal/siv_aead"
"github.com/rfjakob/gocryptfs/v2/internal/stupidgcm"
)
/* /*
Make the "-speed" benchmarks also accessible to the standard test system. Make the "-speed" benchmarks also accessible to the standard test system.
Example run: Example run:
@ -12,18 +23,62 @@ PASS
ok github.com/rfjakob/gocryptfs/v2/internal/speed 6.022s ok github.com/rfjakob/gocryptfs/v2/internal/speed 6.022s
*/ */
import (
"testing"
)
func BenchmarkStupidGCM(b *testing.B) { func BenchmarkStupidGCM(b *testing.B) {
bStupidGCM(b) bStupidGCM(b)
} }
func BenchmarkStupidGCMDecrypt(b *testing.B) {
if stupidgcm.BuiltWithoutOpenssl {
b.Skip("openssl has been disabled at compile-time")
}
bDecrypt(b, stupidgcm.NewAES256GCM(randBytes(32)))
}
func BenchmarkGoGCM(b *testing.B) { func BenchmarkGoGCM(b *testing.B) {
bGoGCM(b) bGoGCM(b)
} }
func BenchmarkGoGCMDecrypt(b *testing.B) {
gAES, err := aes.NewCipher(randBytes(32))
if err != nil {
b.Fatal(err)
}
gGCM, err := cipher.NewGCMWithNonceSize(gAES, 16)
if err != nil {
b.Fatal(err)
}
bDecrypt(b, gGCM)
}
func BenchmarkAESSIV(b *testing.B) { func BenchmarkAESSIV(b *testing.B) {
bAESSIV(b) bAESSIV(b)
} }
func BenchmarkAESSIVDecrypt(b *testing.B) {
bEncrypt(b, siv_aead.New(randBytes(64)))
}
func BenchmarkXchacha(b *testing.B) {
bXchacha20poly1305(b)
}
func BenchmarkXchachaDecrypt(b *testing.B) {
c, _ := chacha20poly1305.NewX(randBytes(32))
bDecrypt(b, c)
}
func BenchmarkStupidXchacha(b *testing.B) {
bStupidXchacha(b)
}
func BenchmarkStupidXchachaDecrypt(b *testing.B) {
bDecrypt(b, stupidgcm.NewXchacha20poly1305(randBytes(32)))
}
func BenchmarkStupidChacha(b *testing.B) {
bEncrypt(b, stupidgcm.NewChacha20poly1305(randBytes(32)))
}
func BenchmarkStupidChachaDecrypt(b *testing.B) {
bDecrypt(b, stupidgcm.NewChacha20poly1305(randBytes(32)))
}

1
internal/stupidgcm/.gitignore vendored Normal file
View File

@ -0,0 +1 @@
*.o

View File

@ -0,0 +1,18 @@
.PHONY: test
test: gcc
# All three ways of building this must work
go build
go build -tags without_openssl
CGO_ENABLED=0 go build -tags without_openssl
# Likewise, all three ways of testing this must work
go test -v
go test -v -tags without_openssl
CGO_ENABLED=0 go test -v -tags without_openssl
.PHONY: gcc
gcc:
gcc -Wall -Wextra -Wformat-security -Wconversion -lcrypto -c *.c
.PHONY: format
format:
clang-format --style=WebKit -i *.c *.h

View File

@ -0,0 +1,54 @@
// +build !without_openssl
package stupidgcm
import (
"crypto/cipher"
"log"
"golang.org/x/crypto/chacha20poly1305"
)
/*
#include <openssl/evp.h>
*/
import "C"
type stupidChacha20poly1305 struct {
stupidAEADCommon
}
// Verify that we satisfy the cipher.AEAD interface
var _ cipher.AEAD = &stupidChacha20poly1305{}
// _EVP_chacha20_poly1305 caches C.EVP_chacha20_poly1305() to avoid the Cgo call
// overhead for each instantiation of NewChacha20poly1305.
var _EVP_chacha20_poly1305 *C.EVP_CIPHER
func init() {
_EVP_chacha20_poly1305 = C.EVP_chacha20_poly1305()
}
// NewChacha20poly1305 returns a new instance of the OpenSSL ChaCha20-Poly1305 AEAD
// cipher ( https://www.openssl.org/docs/man1.1.1/man3/EVP_chacha20_poly1305.html ).
//
// gocryptfs only uses ChaCha20-Poly1305 as a building block for OpenSSL
// XChaCha20-Poly1305. This function is hot because it gets called once for each
// block by XChaCha20-Poly1305.
//
// Only 32-bytes keys and 12-byte IVs are supported.
func NewChacha20poly1305(key []byte) cipher.AEAD {
if len(key) != chacha20poly1305.KeySize {
log.Panicf("Only %d-byte keys are supported, you passed %d bytes", chacha20poly1305.KeySize, len(key))
}
// private copy
key2 := make([]byte, chacha20poly1305.KeySize)
copy(key2, key)
return &stupidChacha20poly1305{
stupidAEADCommon{
key: key2,
openSSLEVPCipher: _EVP_chacha20_poly1305,
nonceSize: chacha20poly1305.NonceSize,
},
}
}

View File

@ -0,0 +1,20 @@
// +build !without_openssl
package stupidgcm
import (
"testing"
"golang.org/x/crypto/chacha20poly1305"
)
func TestStupidChacha20poly1305(t *testing.T) {
key := randBytes(32)
c := NewChacha20poly1305(key)
ref, err := chacha20poly1305.New(key)
if err != nil {
t.Fatal(err)
}
testCiphers(t, c, ref)
}

View File

@ -0,0 +1,70 @@
// +build !without_openssl
package stupidgcm
import (
"log"
)
/*
#include <openssl/evp.h>
*/
import "C"
type stupidAEADCommon struct {
wiped bool
key []byte
openSSLEVPCipher *C.EVP_CIPHER
nonceSize int
}
// Overhead returns the number of bytes that are added for authentication.
//
// Part of the cipher.AEAD interface.
func (c *stupidAEADCommon) Overhead() int {
return tagLen
}
// NonceSize returns the required size of the nonce / IV
//
// Part of the cipher.AEAD interface.
func (c *stupidAEADCommon) NonceSize() int {
return c.nonceSize
}
// Seal encrypts "in" using "iv" and "authData" and append the result to "dst"
//
// Part of the cipher.AEAD interface.
func (c *stupidAEADCommon) Seal(dst, iv, in, authData []byte) []byte {
return openSSLSeal(c, dst, iv, in, authData)
}
// Open decrypts "in" using "iv" and "authData" and append the result to "dst"
//
// Part of the cipher.AEAD interface.
func (c *stupidAEADCommon) Open(dst, iv, in, authData []byte) ([]byte, error) {
return openSSLOpen(c, dst, iv, in, authData)
}
// Wipe tries to wipe the key from memory by overwriting it with zeros.
//
// This is not bulletproof due to possible GC copies, but
// still raises the bar for extracting the key.
func (c *stupidAEADCommon) Wipe() {
key := c.key
c.wiped = true
c.key = nil
for i := range key {
key[i] = 0
}
}
func (c *stupidAEADCommon) Wiped() bool {
if c.wiped {
return true
}
if len(c.key) != keyLen {
log.Panicf("wrong key length %d", len(c.key))
}
return false
}

View File

@ -0,0 +1,282 @@
// +build cgo,!without_openssl
package stupidgcm
import (
"bytes"
"crypto/cipher"
"crypto/rand"
"encoding/hex"
"log"
"sync"
"testing"
)
func testCiphers(t *testing.T, our cipher.AEAD, ref cipher.AEAD) {
t.Run("testEncryptDecrypt", func(t *testing.T) { testEncryptDecrypt(t, our, ref) })
t.Run("testInplaceSeal", func(t *testing.T) { testInplaceSeal(t, our, ref) })
t.Run("testInplaceOpen", func(t *testing.T) { testInplaceOpen(t, our, ref) })
t.Run("testCorruption_c1", func(t *testing.T) { testCorruption(t, our) })
t.Run("testCorruption_c2", func(t *testing.T) { testCorruption(t, ref) })
t.Run("testConcurrency", func(t *testing.T) { testConcurrency(t, our, ref) })
t.Run("testOpenAllZero_our", func(t *testing.T) { testOpenAllZero(t, our) })
t.Run("testOpenAllZero_ref", func(t *testing.T) { testOpenAllZero(t, ref) })
t.Run("testWipe", func(t *testing.T) { testWipe(t, our) })
}
// testEncryptDecrypt encrypts and decrypts using both stupidgcm and Go's built-in
// GCM implementation and verifies that the results are identical.
func testEncryptDecrypt(t *testing.T, c1 cipher.AEAD, c2 cipher.AEAD) {
if c1.NonceSize() != c2.NonceSize() {
t.Fatal("different NonceSize")
}
if c1.Overhead() != c2.Overhead() {
t.Fatal("different Overhead")
}
authData := randBytes(24)
iv := randBytes(c1.NonceSize())
dst := make([]byte, 71) // 71 = arbitrary length
// Check all block sizes from nil to 0 to 5000
for i := -1; i < 5000; i++ {
// stays nil at i == -1
var in []byte
if i >= 0 {
in = make([]byte, i)
}
c1out := c1.Seal(dst, iv, in, authData)
c2out := c2.Seal(dst, iv, in, authData)
// Ciphertext must be identical to Go GCM
if !bytes.Equal(c1out, c2out) {
t.Fatalf("Compare failed for encryption, size %d", i)
t.Log("c1out:")
t.Log("\n" + hex.Dump(c1out))
t.Log("c2out:")
t.Log("\n" + hex.Dump(c2out))
}
c1out2, sErr := c1.Open(dst, iv, c1out[len(dst):], authData)
if sErr != nil {
t.Fatal(sErr)
}
c2out2, gErr := c2.Open(dst, iv, c2out[len(dst):], authData)
if gErr != nil {
t.Fatal(gErr)
}
// Plaintext must be identical to Go GCM
if !bytes.Equal(c1out2, c2out2) {
t.Fatalf("Compare failed for decryption, size %d", i)
}
}
}
// testConcurrency verifies that we don't corrupt data when called concurrently
func testConcurrency(t *testing.T, c1 cipher.AEAD, c2 cipher.AEAD) {
const loopCount = 2
const goroutineCount = 4
for h := 0; h < loopCount; h++ {
var wg sync.WaitGroup
for i := 0; i < goroutineCount; i++ {
wg.Add(1)
go func() {
defer wg.Done()
testEncryptDecrypt(t, c1, c2)
}()
wg.Wait()
}
}
}
// testInplaceSeal:
// Seal re-uses the "dst" buffer it is large enough.
// Check that this works correctly by testing different "dst" capacities from
// 5000 to 16 and "in" lengths from 1 to 5000.
func testInplaceSeal(t *testing.T, c1 cipher.AEAD, c2 cipher.AEAD) {
authData := randBytes(24)
iv := randBytes(c1.NonceSize())
max := 5016
// Check all block sizes from 0 to 5000
for i := 0; i < max-16; i++ {
in := make([]byte, i)
dst := make([]byte, max-i)
dst = dst[:16]
c1out := c1.Seal(dst, iv, in, authData)
dst2 := make([]byte, 16)
c2out := c2.Seal(dst2, iv, in, authData)
// Ciphertext must be identical to Go GCM
if !bytes.Equal(c1out, c2out) {
t.Fatalf("Compare failed for encryption, size %d", i)
t.Log("sOut:")
t.Log("\n" + hex.Dump(c1out))
t.Log("gOut:")
t.Log("\n" + hex.Dump(c2out))
}
}
}
// testInplaceOpen - Open re-uses the "dst" buffer it is large enough.
// Check that this works correctly by testing different "dst" capacities from
// 5000 to 16 and "in" lengths from 1 to 5000.
func testInplaceOpen(t *testing.T, c1 cipher.AEAD, c2 cipher.AEAD) {
authData := randBytes(24)
iv := randBytes(c1.NonceSize())
max := 5016
// Check all block sizes from 1 to 5000
for i := 1; i < max-c1.NonceSize(); i++ {
in := make([]byte, i)
c2ciphertext := c2.Seal(iv, iv, in, authData)
dst := make([]byte, max-i)
// sPlaintext ... stupidgcm plaintext
c1plaintext, err := c1.Open(dst[:0], iv, c2ciphertext[c1.NonceSize():], authData)
if err != nil {
t.Fatal(err)
}
// Plaintext must be identical to Go GCM
if !bytes.Equal(in, c1plaintext) {
t.Fatalf("Compare failed, i=%d", i)
}
}
}
// testCorruption verifies that changes in the ciphertext result in a decryption
// error
func testCorruption(t *testing.T, c cipher.AEAD) {
authData := randBytes(24)
iv := randBytes(c.NonceSize())
in := make([]byte, 354)
out := c.Seal(nil, iv, in, authData)
out2, sErr := c.Open(nil, iv, out, authData)
if sErr != nil {
t.Fatal(sErr)
}
if !bytes.Equal(in, out2) {
t.Fatalf("Compare failed")
}
// Corrupt first byte
out[0]++
out2, sErr = c.Open(nil, iv, out, authData)
if sErr == nil || out2 != nil {
t.Fatalf("Should have gotten error")
}
out[0]--
// Corrupt last byte
out[len(out)-1]++
out2, sErr = c.Open(nil, iv, out, authData)
if sErr == nil || out2 != nil {
t.Fatalf("Should have gotten error")
}
out[len(out)-1]--
// Append one byte
out = append(out, 0)
out2, sErr = c.Open(nil, iv, out, authData)
if sErr == nil || out2 != nil {
t.Fatalf("Should have gotten error")
}
}
// testOpenAllZero tests that we do not crash for nil or any block size of zeros
func testOpenAllZero(t *testing.T, c cipher.AEAD) {
authData := make([]byte, 24)
iv := make([]byte, c.NonceSize())
for i := -1; i < 5000; i++ {
// stays nil at i == -1
var cipher []byte
if i >= 0 {
cipher = make([]byte, i)
}
plain, err := c.Open(nil, iv, cipher, authData)
if err == nil {
t.Error("should have gotten error, but did not")
}
if len(plain) > 0 {
t.Errorf("should not have received data, but got %d bytes", len(plain))
}
}
}
func testWipe(t *testing.T, c cipher.AEAD) {
switch c2 := c.(type) {
case *stupidGCM:
c2.Wipe()
if !c2.Wiped() {
t.Error("c2.wiped is not set")
}
for _, v := range c2.key {
if v != 0 {
t.Fatal("c2._key is not zeroed")
}
}
case *stupidChacha20poly1305:
c2.Wipe()
if !c2.Wiped() {
t.Error("c2.wiped is not set")
}
for _, v := range c2.key {
if v != 0 {
t.Fatal("c2._key is not zeroed")
}
}
case *stupidXchacha20poly1305:
c2.Wipe()
if !c2.wiped {
t.Error("c2.wiped is not set")
}
for _, v := range c2.key {
if v != 0 {
t.Fatal("c2.key is not zeroed")
}
}
default:
t.Fatalf("BUG: unhandled type %T", c2)
}
}
// Get "n" random bytes from /dev/urandom or panic
func randBytes(n int) []byte {
b := make([]byte, n)
_, err := rand.Read(b)
if err != nil {
log.Panic("Failed to read random bytes: " + err.Error())
}
return b
}
/*
BenchmarkCCall benchmarks the overhead of calling from Go into C.
Looks like things improved a bit compared to
https://www.cockroachlabs.com/blog/the-cost-and-complexity-of-cgo/
where they measured 171ns/op:
$ go test -bench .
goos: linux
goarch: amd64
pkg: github.com/rfjakob/gocryptfs/v2/internal/stupidgcm
cpu: Intel(R) Core(TM) i5-3470 CPU @ 3.20GHz
BenchmarkCCall-4 13989364 76.72 ns/op
PASS
ok github.com/rfjakob/gocryptfs/v2/internal/stupidgcm 1.735s
*/
func BenchmarkCCall(b *testing.B) {
for i := 0; i < b.N; i++ {
noopCFunction()
}
}

59
internal/stupidgcm/doc.go Normal file
View File

@ -0,0 +1,59 @@
// Package stupidgcm wraps OpenSSL to provide a cipher.AEAD interface for
// authenticated encryption algorithms.
//
// The supported algorithms are:
//
// (1) AES-GCM-256 (OpenSSL EVP_aes_256_gcm)
//
// (2) ChaCha20-Poly1305 (OpenSSL EVP_chacha20_poly1305)
//
// (3) XChaCha20-Poly1305 (OpenSSL EVP_chacha20_poly1305 + Go HChaCha20)
//
// The golang.org/x/crypto libraries provides implementations for all algorithms,
// and the test suite verifies that the implementation in this package gives
// the exact same results.
//
// However, OpenSSL has optimized assembly for almost all platforms, which Go
// does not. Example for a 32-bit ARM device (Odroid XU4):
//
// $ gocrypts -speed
// gocryptfs v2.1-68-gedf9d4c.stupidchacha; go-fuse v2.1.1-0.20210825171523-3ab5d95a30ae; 2021-09-04 go1.16.7 linux/arm
// AES-GCM-256-OpenSSL 56.84 MB/s (selected in auto mode)
// AES-GCM-256-Go 16.61 MB/s
// AES-SIV-512-Go 16.49 MB/s
// XChaCha20-Poly1305-Go 39.08 MB/s (use via -xchacha flag)
// XChaCha20-Poly1305-OpenSSL 141.82 MB/s
//
// This package is "stupid" in the sense that it only supports a narrow set of
// key- and iv-lengths, and panics if it does not like what you pass it.
// See the constructor functions for which restrictions apply for each algorithm.
// Also, it is only tested for block lengths up to 5000 bytes, because this is
// what gocryptfs uses.
//
// Corrupt ciphertexts never cause a panic. Instead, ErrAuth is returned on
// decryption.
//
// XChaCha20-Poly1305
//
// The XChaCha20-Poly1305 implementation is more complicated than the others,
// because OpenSSL does not support XChaCha20-Poly1305 directly. Follow
// https://github.com/openssl/openssl/issues/5523 to get notified when it is
// accepted into OpenSSL.
//
// Fortunately, XChaCha20-Poly1305 is just ChaCha20-Poly1305 with some key+iv
// mixing using HChaCha20 in front:
//
// key (32 bytes), iv (24 bytes)
// |
// v
// HChaCha20 (provided by golang.org/x/crypto/chacha20)
// |
// v
// key2 (32 bytes), iv2 (16 bytes)
// |
// v
// ChaCha20-Poly1305 (OpenSSL EVP_chacha20_poly1305)
//
// As HChaCha20 is very fast, XChaCha20-Poly1305 gets almost the same throughput
// as ChaCha20-Poly1305 (for 4kiB blocks).
package stupidgcm

41
internal/stupidgcm/gcm.go Normal file
View File

@ -0,0 +1,41 @@
// +build !without_openssl
package stupidgcm
// #include <openssl/evp.h>
import "C"
import (
"crypto/cipher"
"log"
)
const (
// BuiltWithoutOpenssl indicates if openssl been disabled at compile-time
BuiltWithoutOpenssl = false
keyLen = 32
ivLen = 16
tagLen = 16
)
type stupidGCM struct {
stupidAEADCommon
}
// NewAES256GCM returns a new AES-256-GCM cipher that satisfies the cipher.AEAD interface.
//
// Only 32-bytes keys and 16-byte IVs are supported.
func NewAES256GCM(keyIn []byte) cipher.AEAD {
if len(keyIn) != keyLen {
log.Panicf("Only %d-byte keys are supported", keyLen)
}
return &stupidGCM{
stupidAEADCommon{
// Create a private copy of the key
key: append([]byte{}, keyIn...),
openSSLEVPCipher: C.EVP_aes_256_gcm(),
nonceSize: ivLen,
},
}
}

View File

@ -0,0 +1,28 @@
// +build !without_openssl
// We compare against Go's built-in GCM implementation. Since stupidgcm only
// supports 128-bit IVs and Go only supports that from 1.5 onward, we cannot
// run these tests on older Go versions.
package stupidgcm
import (
"crypto/aes"
"crypto/cipher"
"testing"
)
func TestStupidGCM(t *testing.T) {
key := randBytes(32)
sGCM := NewAES256GCM(key)
gAES, err := aes.NewCipher(key)
if err != nil {
t.Fatal(err)
}
gGCM, err := cipher.NewGCMWithNonceSize(gAES, 16)
if err != nil {
t.Fatal(err)
}
testCiphers(t, sGCM, gGCM)
}

View File

@ -0,0 +1,123 @@
// +build !without_openssl
package stupidgcm
import (
"fmt"
"log"
)
/*
#include "openssl_aead.h"
#cgo pkg-config: libcrypto
*/
import "C"
func openSSLSeal(a *stupidAEADCommon, dst, iv, in, authData []byte) []byte {
if a.Wiped() {
log.Panic("BUG: tried to use wiped key")
}
if len(iv) != a.NonceSize() {
log.Panicf("Only %d-byte IVs are supported, you passed %d bytes", a.NonceSize(), len(iv))
}
// If the "dst" slice is large enough we can use it as our output buffer
outLen := len(in) + tagLen
var buf []byte
inplace := false
if cap(dst)-len(dst) >= outLen {
inplace = true
buf = dst[len(dst) : len(dst)+outLen]
} else {
buf = make([]byte, outLen)
}
res := int(C.openssl_aead_seal(a.openSSLEVPCipher,
slicePointerOrNull(in),
C.int(len(in)),
(*C.uchar)(&authData[0]),
C.int(len(authData)),
(*C.uchar)(&a.key[0]),
C.int(len(a.key)),
(*C.uchar)(&iv[0]),
C.int(len(iv)),
(*C.uchar)(&buf[0]),
C.int(len(buf))))
if res != outLen {
log.Panicf("expected length %d, got %d", outLen, res)
}
if inplace {
return dst[:len(dst)+outLen]
}
return append(dst, buf...)
}
func openSSLOpen(a *stupidAEADCommon, dst, iv, in, authData []byte) ([]byte, error) {
if a.Wiped() {
log.Panic("BUG: tried to use wiped key")
}
if len(iv) != a.NonceSize() {
log.Panicf("Only %d-byte IVs are supported, you passed %d bytes", a.NonceSize(), len(iv))
}
if len(in) < tagLen {
return nil, fmt.Errorf("stupidChacha20poly1305: input data too short (%d bytes)", len(in))
}
// If the "dst" slice is large enough we can use it as our output buffer
outLen := len(in) - tagLen
var buf []byte
inplace := false
if cap(dst)-len(dst) >= outLen {
inplace = true
buf = dst[len(dst) : len(dst)+outLen]
} else {
buf = make([]byte, len(in)-tagLen)
}
ciphertext := in[:len(in)-tagLen]
tag := in[len(in)-tagLen:]
res := int(C.openssl_aead_open(a.openSSLEVPCipher,
slicePointerOrNull(ciphertext),
C.int(len(ciphertext)),
(*C.uchar)(&authData[0]),
C.int(len(authData)),
(*C.uchar)(&tag[0]),
C.int(len(tag)),
(*C.uchar)(&a.key[0]),
C.int(len(a.key)),
(*C.uchar)(&iv[0]),
C.int(len(iv)),
slicePointerOrNull(buf),
C.int(len(buf))))
if res < 0 {
return nil, ErrAuth
}
if res != outLen {
log.Panicf("unexpected length %d", res)
}
if inplace {
return dst[:len(dst)+outLen], nil
}
return append(dst, buf...), nil
}
// slicePointerOrNull returns a C pointer to the beginning of the byte slice,
// or NULL if the byte slice is empty. This is useful for slices that can be
// empty, otherwise you can directly use "(*C.uchar)(&s[0])".
func slicePointerOrNull(s []byte) (ptr *C.uchar) {
if len(s) == 0 {
return
}
return (*C.uchar)(&s[0])
}
// This functions exists to benchmark the C call overhead from Go.
// See BenchmarkCCall for resuts.
func noopCFunction() {
C.noop_c_function()
}

View File

@ -0,0 +1,185 @@
// +build !without_openssl
#include "openssl_aead.h"
#include <openssl/evp.h>
#include <stdio.h>
//#cgo pkg-config: libcrypto
static void panic(const char* const msg)
{
fprintf(stderr, "panic in C code: %s\n", msg);
__builtin_trap();
}
// We only support 16-byte tags
static const int supportedTagLen = 16;
// https://wiki.openssl.org/index.php/EVP_Authenticated_Encryption_and_Decryption#Authenticated_Encryption_using_GCM_mode
int openssl_aead_seal(
const EVP_CIPHER* evpCipher,
const unsigned char* const plaintext,
const int plaintextLen,
const unsigned char* const authData,
const int authDataLen,
const unsigned char* const key,
const int keyLen,
const unsigned char* const iv,
const int ivLen,
unsigned char* const ciphertext,
const int ciphertextBufLen)
{
// Create scratch space "ctx"
EVP_CIPHER_CTX* ctx = EVP_CIPHER_CTX_new();
if (!ctx) {
panic("EVP_CIPHER_CTX_new failed");
}
// Set cipher
if (EVP_EncryptInit_ex(ctx, evpCipher, NULL, NULL, NULL) != 1) {
panic("EVP_EncryptInit_ex set cipher failed");
}
// Check keyLen by trying to set it (fails if keyLen != 32)
if (EVP_CIPHER_CTX_set_key_length(ctx, keyLen) != 1) {
panic("keyLen mismatch");
}
// Set IV length so we do not depend on the default
if (EVP_CIPHER_CTX_ctrl(ctx, EVP_CTRL_AEAD_SET_IVLEN, ivLen, NULL) != 1) {
panic("EVP_CTRL_AEAD_SET_IVLEN failed");
}
// Set key and IV
if (EVP_EncryptInit_ex(ctx, NULL, NULL, key, iv) != 1) {
panic("EVP_EncryptInit_ex set key & iv failed");
}
// Provide authentication data
int outLen = 0;
if (EVP_EncryptUpdate(ctx, NULL, &outLen, authData, authDataLen) != 1) {
panic("EVP_EncryptUpdate authData failed");
}
if (outLen != authDataLen) {
panic("EVP_EncryptUpdate authData: unexpected length");
}
// Encrypt "plaintext" into "ciphertext"
if (plaintextLen > ciphertextBufLen) {
panic("plaintext overflows output buffer");
}
if (EVP_EncryptUpdate(ctx, ciphertext, &outLen, plaintext, plaintextLen) != 1) {
panic("EVP_EncryptUpdate ciphertext failed");
}
if (outLen != plaintextLen) {
panic("EVP_EncryptUpdate ciphertext: unexpected length");
}
int ciphertextLen = outLen;
// Finalise encryption
// Normally ciphertext bytes may be written at this stage, but this does not occur in GCM mode
if (EVP_EncryptFinal_ex(ctx, ciphertext + plaintextLen, &outLen) != 1) {
panic("EVP_EncryptFinal_ex failed");
}
if (outLen != 0) {
panic("EVP_EncryptFinal_ex: unexpected length");
}
// Get MAC tag and append it to the ciphertext
if (ciphertextLen + supportedTagLen > ciphertextBufLen) {
panic("tag overflows output buffer");
}
if (EVP_CIPHER_CTX_ctrl(ctx, EVP_CTRL_AEAD_GET_TAG, supportedTagLen, ciphertext + plaintextLen) != 1) {
panic("EVP_CTRL_AEAD_GET_TAG failed");
}
ciphertextLen += supportedTagLen;
// Free scratch space
EVP_CIPHER_CTX_free(ctx);
return ciphertextLen;
}
int openssl_aead_open(
const EVP_CIPHER* evpCipher,
const unsigned char* const ciphertext,
const int ciphertextLen,
const unsigned char* const authData,
const int authDataLen,
unsigned char* const tag,
const int tagLen,
const unsigned char* const key,
const int keyLen,
const unsigned char* const iv,
const int ivLen,
unsigned char* const plaintext,
const int plaintextBufLen)
{
// Create scratch space "ctx"
EVP_CIPHER_CTX* ctx = EVP_CIPHER_CTX_new();
if (!ctx) {
panic("EVP_CIPHER_CTX_new failed");
}
// Set cipher
if (EVP_DecryptInit_ex(ctx, evpCipher, NULL, NULL, NULL) != 1) {
panic("EVP_DecryptInit_ex set cipher failed");
}
// Check keyLen by trying to set it (fails if keyLen != 32)
if (EVP_CIPHER_CTX_set_key_length(ctx, keyLen) != 1) {
panic("keyLen mismatch");
}
// Set IV length so we do not depend on the default
if (EVP_CIPHER_CTX_ctrl(ctx, EVP_CTRL_AEAD_SET_IVLEN, ivLen, NULL) != 1) {
panic("EVP_CTRL_AEAD_SET_IVLEN failed");
}
// Set key and IV
if (EVP_DecryptInit_ex(ctx, NULL, NULL, key, iv) != 1) {
panic("EVP_DecryptInit_ex set key & iv failed");
}
// Provide authentication data
int outLen = 0;
if (EVP_DecryptUpdate(ctx, NULL, &outLen, authData, authDataLen) != 1) {
panic("EVP_DecryptUpdate authData failed");
}
if (outLen != authDataLen) {
panic("EVP_DecryptUpdate authData: unexpected length");
}
// Decrypt "ciphertext" into "plaintext"
if (ciphertextLen > plaintextBufLen) {
panic("ciphertextLen overflows output buffer");
}
if (EVP_DecryptUpdate(ctx, plaintext, &outLen, ciphertext, ciphertextLen) != 1) {
panic("EVP_DecryptUpdate failed");
}
int plaintextLen = outLen;
// Check tag
if (tagLen != supportedTagLen) {
panic("unsupported tag length");
}
if (EVP_CIPHER_CTX_ctrl(ctx, EVP_CTRL_AEAD_SET_TAG, tagLen, tag) != 1) {
panic("EVP_CTRL_AEAD_SET_TAG failed");
}
if (EVP_DecryptFinal_ex(ctx, plaintext + plaintextLen, &outLen) != 1) {
// authentication failed
return -1;
}
if (outLen != 0) {
panic("EVP_EncryptFinal_ex: unexpected length");
}
/* Clean up */
EVP_CIPHER_CTX_free(ctx);
return plaintextLen;
}
// This functions exists to benchmark the C call overhead from Go.
void noop_c_function(void) {
return;
}

View File

@ -0,0 +1,31 @@
#include <openssl/evp.h>
int openssl_aead_seal(
const EVP_CIPHER* evpCipher,
const unsigned char* const plaintext,
const int plaintextLen,
const unsigned char* const authData,
const int authDataLen,
const unsigned char* const key,
const int keyLen,
const unsigned char* const iv,
const int ivLen,
unsigned char* const ciphertext,
const int ciphertextBufLen);
int openssl_aead_open(
const EVP_CIPHER* evpCipher,
const unsigned char* const ciphertext,
const int ciphertextLen,
const unsigned char* const authData,
const int authDataLen,
unsigned char* const tag,
const int tagLen,
const unsigned char* const key,
const int keyLen,
const unsigned char* const iv,
const int ivLen,
unsigned char* const plaintext,
const int plaintextBufLen);
void noop_c_function(void);

View File

@ -6,7 +6,8 @@ import (
"golang.org/x/sys/cpu" "golang.org/x/sys/cpu"
) )
// PreferOpenSSL tells us if OpenSSL is faster than Go GCM on this machine. // PreferOpenSSLAES256GCM tells us if OpenSSL AES-256-GCM is faster than Go stdlib
// on this machine.
// //
// Go GCM is only faster if the CPU either: // Go GCM is only faster if the CPU either:
// //
@ -14,22 +15,46 @@ import (
// 2) Is ARM64 && has AES instructions && Go is v1.11 or higher // 2) Is ARM64 && has AES instructions && Go is v1.11 or higher
// (commit https://github.com/golang/go/commit/4f1f503373cda7160392be94e3849b0c9b9ebbda) // (commit https://github.com/golang/go/commit/4f1f503373cda7160392be94e3849b0c9b9ebbda)
// //
// See https://github.com/rfjakob/gocryptfs/v2/wiki/CPU-Benchmarks // See https://github.com/rfjakob/gocryptfs/wiki/CPU-Benchmarks
// for benchmarks. // for benchmarks.
func PreferOpenSSL() bool { func PreferOpenSSLAES256GCM() bool {
if BuiltWithoutOpenssl { if BuiltWithoutOpenssl {
return false return false
} }
// Safe to call on other architectures - will just read false. // If the CPU has AES acceleration, Go stdlib is faster
if cpu.X86.HasAES || cpu.ARM64.HasAES { if CpuHasAES() {
// Go stdlib is probably faster
return false return false
} }
// On the Apple M1, Go stdlib is faster than OpenSSL, despite cpu.ARM64.HasAES // Otherwise OpenSSL is probably faster
// reading false: https://github.com/rfjakob/gocryptfs/v2/issues/556#issuecomment-848079309
if runtime.GOOS == "darwin" && runtime.GOARCH == "arm64" {
return false
}
// OpenSSL is probably faster
return true return true
} }
// PreferOpenSSLXchacha20poly1305 returns true if OpenSSL Xchacha20poly1305 is
// faster than Go stdlib on this machine.
func PreferOpenSSLXchacha20poly1305() bool {
if BuiltWithoutOpenssl {
return false
}
// Go x/crypto has optimized assembly for amd64:
// https://github.com/golang/crypto/blob/master/chacha20poly1305/chacha20poly1305_amd64.s
if runtime.GOARCH == "amd64" {
return false
}
// On arm64 and arm, OpenSSL is faster. Probably everwhere else too.
return true
}
// CpuHasAES tells you if the CPU we are running has AES acceleration that is
// usable by the Go crypto library.
func CpuHasAES() bool {
// Safe to call on other architectures - will just read false.
if cpu.X86.HasAES || cpu.ARM64.HasAES {
return true
}
// On the Apple M1, the CPU has AES acceleration, despite cpu.ARM64.HasAES
// reading false: https://github.com/rfjakob/gocryptfs/issues/556#issuecomment-848079309
if runtime.GOOS == "darwin" && runtime.GOARCH == "arm64" {
return true
}
return false
}

View File

@ -1,249 +0,0 @@
// +build !without_openssl
// Package stupidgcm is a thin wrapper for OpenSSL's GCM encryption and
// decryption functions. It only support 32-byte keys and 16-bit IVs.
package stupidgcm
// #include <openssl/evp.h>
// #cgo pkg-config: libcrypto
import "C"
import (
"crypto/cipher"
"fmt"
"log"
"unsafe"
)
const (
// BuiltWithoutOpenssl indicates if openssl been disabled at compile-time
BuiltWithoutOpenssl = false
keyLen = 32
ivLen = 16
tagLen = 16
)
// StupidGCM implements the cipher.AEAD interface
type StupidGCM struct {
key []byte
forceDecode bool
}
// Verify that we satisfy the cipher.AEAD interface
var _ cipher.AEAD = &StupidGCM{}
// New returns a new cipher.AEAD implementation..
func New(keyIn []byte, forceDecode bool) cipher.AEAD {
if len(keyIn) != keyLen {
log.Panicf("Only %d-byte keys are supported", keyLen)
}
// Create a private copy of the key
key := append([]byte{}, keyIn...)
return &StupidGCM{key: key, forceDecode: forceDecode}
}
// NonceSize returns the required size of the nonce / IV.
func (g *StupidGCM) NonceSize() int {
return ivLen
}
// Overhead returns the number of bytes that are added for authentication.
func (g *StupidGCM) Overhead() int {
return tagLen
}
// Seal encrypts "in" using "iv" and "authData" and append the result to "dst"
func (g *StupidGCM) Seal(dst, iv, in, authData []byte) []byte {
if len(iv) != ivLen {
log.Panicf("Only %d-byte IVs are supported", ivLen)
}
if len(in) == 0 {
log.Panic("Zero-length input data is not supported")
}
if len(g.key) != keyLen {
log.Panicf("Wrong key length: %d. Key has been wiped?", len(g.key))
}
// If the "dst" slice is large enough we can use it as our output buffer
outLen := len(in) + tagLen
var buf []byte
inplace := false
if cap(dst)-len(dst) >= outLen {
inplace = true
buf = dst[len(dst) : len(dst)+outLen]
} else {
buf = make([]byte, outLen)
}
// https://wiki.openssl.org/index.php/EVP_Authenticated_Encryption_and_Decryption#Authenticated_Encryption_using_GCM_mode
// Create scratch space "context"
ctx := C.EVP_CIPHER_CTX_new()
if ctx == nil {
log.Panic("EVP_CIPHER_CTX_new failed")
}
// Set cipher to AES-256
if C.EVP_EncryptInit_ex(ctx, C.EVP_aes_256_gcm(), nil, nil, nil) != 1 {
log.Panic("EVP_EncryptInit_ex I failed")
}
// Use 16-byte IV
if C.EVP_CIPHER_CTX_ctrl(ctx, C.EVP_CTRL_GCM_SET_IVLEN, ivLen, nil) != 1 {
log.Panic("EVP_CIPHER_CTX_ctrl EVP_CTRL_GCM_SET_IVLEN failed")
}
// Set key and IV
if C.EVP_EncryptInit_ex(ctx, nil, nil, (*C.uchar)(&g.key[0]), (*C.uchar)(&iv[0])) != 1 {
log.Panic("EVP_EncryptInit_ex II failed")
}
// Provide authentication data
var resultLen C.int
if C.EVP_EncryptUpdate(ctx, nil, &resultLen, (*C.uchar)(&authData[0]), C.int(len(authData))) != 1 {
log.Panic("EVP_EncryptUpdate authData failed")
}
if int(resultLen) != len(authData) {
log.Panicf("Unexpected length %d", resultLen)
}
// Encrypt "in" into "buf"
if C.EVP_EncryptUpdate(ctx, (*C.uchar)(&buf[0]), &resultLen, (*C.uchar)(&in[0]), C.int(len(in))) != 1 {
log.Panic("EVP_EncryptUpdate failed")
}
if int(resultLen) != len(in) {
log.Panicf("Unexpected length %d", resultLen)
}
// Finalise encryption
// Because GCM is a stream encryption, this will not write out any data.
dummy := make([]byte, 16)
if C.EVP_EncryptFinal_ex(ctx, (*C.uchar)(&dummy[0]), &resultLen) != 1 {
log.Panic("EVP_EncryptFinal_ex failed")
}
if resultLen != 0 {
log.Panicf("Unexpected length %d", resultLen)
}
// Get GMAC tag and append it to the ciphertext in "buf"
if C.EVP_CIPHER_CTX_ctrl(ctx, C.EVP_CTRL_GCM_GET_TAG, tagLen, (unsafe.Pointer)(&buf[len(in)])) != 1 {
log.Panic("EVP_CIPHER_CTX_ctrl EVP_CTRL_GCM_GET_TAG failed")
}
// Free scratch space
C.EVP_CIPHER_CTX_free(ctx)
if inplace {
return dst[:len(dst)+outLen]
}
return append(dst, buf...)
}
// Open decrypts "in" using "iv" and "authData" and append the result to "dst"
func (g *StupidGCM) Open(dst, iv, in, authData []byte) ([]byte, error) {
if len(iv) != ivLen {
log.Panicf("Only %d-byte IVs are supported", ivLen)
}
if len(g.key) != keyLen {
log.Panicf("Wrong key length: %d. Key has been wiped?", len(g.key))
}
if len(in) <= tagLen {
return nil, fmt.Errorf("stupidgcm: input data too short (%d bytes)", len(in))
}
// If the "dst" slice is large enough we can use it as our output buffer
outLen := len(in) - tagLen
var buf []byte
inplace := false
if cap(dst)-len(dst) >= outLen {
inplace = true
buf = dst[len(dst) : len(dst)+outLen]
} else {
buf = make([]byte, len(in)-tagLen)
}
ciphertext := in[:len(in)-tagLen]
tag := in[len(in)-tagLen:]
// https://wiki.openssl.org/index.php/EVP_Authenticated_Encryption_and_Decryption#Authenticated_Encryption_using_GCM_mode
// Create scratch space "context"
ctx := C.EVP_CIPHER_CTX_new()
if ctx == nil {
log.Panic("EVP_CIPHER_CTX_new failed")
}
// Set cipher to AES-256
if C.EVP_DecryptInit_ex(ctx, C.EVP_aes_256_gcm(), nil, nil, nil) != 1 {
log.Panic("EVP_DecryptInit_ex I failed")
}
// Use 16-byte IV
if C.EVP_CIPHER_CTX_ctrl(ctx, C.EVP_CTRL_GCM_SET_IVLEN, ivLen, nil) != 1 {
log.Panic("EVP_CIPHER_CTX_ctrl EVP_CTRL_GCM_SET_IVLEN failed")
}
// Set key and IV
if C.EVP_DecryptInit_ex(ctx, nil, nil, (*C.uchar)(&g.key[0]), (*C.uchar)(&iv[0])) != 1 {
log.Panic("EVP_DecryptInit_ex II failed")
}
// Set expected GMAC tag
if C.EVP_CIPHER_CTX_ctrl(ctx, C.EVP_CTRL_GCM_SET_TAG, tagLen, (unsafe.Pointer)(&tag[0])) != 1 {
log.Panic("EVP_CIPHER_CTX_ctrl failed")
}
// Provide authentication data
var resultLen C.int
if C.EVP_DecryptUpdate(ctx, nil, &resultLen, (*C.uchar)(&authData[0]), C.int(len(authData))) != 1 {
log.Panic("EVP_DecryptUpdate authData failed")
}
if int(resultLen) != len(authData) {
log.Panicf("Unexpected length %d", resultLen)
}
// Decrypt "ciphertext" into "buf"
if C.EVP_DecryptUpdate(ctx, (*C.uchar)(&buf[0]), &resultLen, (*C.uchar)(&ciphertext[0]), C.int(len(ciphertext))) != 1 {
log.Panic("EVP_DecryptUpdate failed")
}
if int(resultLen) != len(ciphertext) {
log.Panicf("Unexpected length %d", resultLen)
}
// Check GMAC
dummy := make([]byte, 16)
res := C.EVP_DecryptFinal_ex(ctx, (*C.uchar)(&dummy[0]), &resultLen)
if resultLen != 0 {
log.Panicf("Unexpected length %d", resultLen)
}
// Free scratch space
C.EVP_CIPHER_CTX_free(ctx)
if res != 1 {
// The error code must always be checked by the calling function, because the decrypted buffer
// may contain corrupted data that we are returning in case the user forced reads
if g.forceDecode {
return append(dst, buf...), ErrAuth
}
return nil, ErrAuth
}
if inplace {
return dst[:len(dst)+outLen], nil
}
return append(dst, buf...), nil
}
// Wipe tries to wipe the AES key from memory by overwriting it with zeros
// and setting the reference to nil.
//
// This is not bulletproof due to possible GC copies, but
// still raises to bar for extracting the key.
func (g *StupidGCM) Wipe() {
for i := range g.key {
g.key[i] = 0
}
g.key = nil
}

View File

@ -1,195 +0,0 @@
// +build !without_openssl
// We compare against Go's built-in GCM implementation. Since stupidgcm only
// supports 128-bit IVs and Go only supports that from 1.5 onward, we cannot
// run these tests on older Go versions.
package stupidgcm
import (
"bytes"
"crypto/aes"
"crypto/cipher"
"crypto/rand"
"encoding/hex"
"log"
"testing"
)
// Get "n" random bytes from /dev/urandom or panic
func randBytes(n int) []byte {
b := make([]byte, n)
_, err := rand.Read(b)
if err != nil {
log.Panic("Failed to read random bytes: " + err.Error())
}
return b
}
// TestEncryptDecrypt encrypts and decrypts using both stupidgcm and Go's built-in
// GCM implementation and verifies that the results are identical.
func TestEncryptDecrypt(t *testing.T) {
key := randBytes(32)
sGCM := New(key, false)
authData := randBytes(24)
iv := randBytes(16)
dst := make([]byte, 71) // 71 = random length
gAES, err := aes.NewCipher(key)
if err != nil {
t.Fatal(err)
}
gGCM, err := cipher.NewGCMWithNonceSize(gAES, 16)
if err != nil {
t.Fatal(err)
}
// Check all block sizes from 1 to 5000
for i := 1; i < 5000; i++ {
in := make([]byte, i)
sOut := sGCM.Seal(dst, iv, in, authData)
gOut := gGCM.Seal(dst, iv, in, authData)
// Ciphertext must be identical to Go GCM
if !bytes.Equal(sOut, gOut) {
t.Fatalf("Compare failed for encryption, size %d", i)
t.Log("sOut:")
t.Log("\n" + hex.Dump(sOut))
t.Log("gOut:")
t.Log("\n" + hex.Dump(gOut))
}
sOut2, sErr := sGCM.Open(dst, iv, sOut[len(dst):], authData)
if sErr != nil {
t.Fatal(sErr)
}
gOut2, gErr := gGCM.Open(dst, iv, gOut[len(dst):], authData)
if gErr != nil {
t.Fatal(gErr)
}
// Plaintext must be identical to Go GCM
if !bytes.Equal(sOut2, gOut2) {
t.Fatalf("Compare failed for decryption, size %d", i)
}
}
}
// Seal re-uses the "dst" buffer it is large enough.
// Check that this works correctly by testing different "dst" capacities from
// 5000 to 16 and "in" lengths from 1 to 5000.
func TestInplaceSeal(t *testing.T) {
key := randBytes(32)
sGCM := New(key, false)
authData := randBytes(24)
iv := randBytes(16)
gAES, err := aes.NewCipher(key)
if err != nil {
t.Fatal(err)
}
gGCM, err := cipher.NewGCMWithNonceSize(gAES, 16)
if err != nil {
t.Fatal(err)
}
max := 5016
// Check all block sizes from 1 to 5000
for i := 1; i < max-16; i++ {
in := make([]byte, i)
dst := make([]byte, max-i)
dst = dst[:16]
sOut := sGCM.Seal(dst, iv, in, authData)
dst2 := make([]byte, 16)
gOut := gGCM.Seal(dst2, iv, in, authData)
// Ciphertext must be identical to Go GCM
if !bytes.Equal(sOut, gOut) {
t.Fatalf("Compare failed for encryption, size %d", i)
t.Log("sOut:")
t.Log("\n" + hex.Dump(sOut))
t.Log("gOut:")
t.Log("\n" + hex.Dump(gOut))
}
}
}
// Open re-uses the "dst" buffer it is large enough.
// Check that this works correctly by testing different "dst" capacities from
// 5000 to 16 and "in" lengths from 1 to 5000.
func TestInplaceOpen(t *testing.T) {
key := randBytes(32)
sGCM := New(key, false)
authData := randBytes(24)
iv := randBytes(16)
gAES, err := aes.NewCipher(key)
if err != nil {
t.Fatal(err)
}
gGCM, err := cipher.NewGCMWithNonceSize(gAES, 16)
if err != nil {
t.Fatal(err)
}
max := 5016
// Check all block sizes from 1 to 5000
for i := 1; i < max-16; i++ {
in := make([]byte, i)
gCiphertext := gGCM.Seal(iv, iv, in, authData)
dst := make([]byte, max-i)
// sPlaintext ... stupidgcm plaintext
sPlaintext, err := sGCM.Open(dst[:0], iv, gCiphertext[16:], authData)
if err != nil {
t.Fatal(err)
}
// Plaintext must be identical to Go GCM
if !bytes.Equal(in, sPlaintext) {
t.Fatalf("Compare failed, i=%d", i)
}
}
}
// TestCorruption verifies that changes in the ciphertext result in a decryption
// error
func TestCorruption(t *testing.T) {
key := randBytes(32)
sGCM := New(key, false)
authData := randBytes(24)
iv := randBytes(16)
in := make([]byte, 354)
sOut := sGCM.Seal(nil, iv, in, authData)
sOut2, sErr := sGCM.Open(nil, iv, sOut, authData)
if sErr != nil {
t.Fatal(sErr)
}
if !bytes.Equal(in, sOut2) {
t.Fatalf("Compare failed")
}
// Corrupt first byte
sOut[0]++
sOut2, sErr = sGCM.Open(nil, iv, sOut, authData)
if sErr == nil || sOut2 != nil {
t.Fatalf("Should have gotten error")
}
sOut[0]--
// Corrupt last byte
sOut[len(sOut)-1]++
sOut2, sErr = sGCM.Open(nil, iv, sOut, authData)
if sErr == nil || sOut2 != nil {
t.Fatalf("Should have gotten error")
}
sOut[len(sOut)-1]--
// Append one byte
sOut = append(sOut, 0)
sOut2, sErr = sGCM.Open(nil, iv, sOut, authData)
if sErr == nil || sOut2 != nil {
t.Fatalf("Should have gotten error")
}
}

View File

@ -6,47 +6,32 @@ import (
"fmt" "fmt"
"os" "os"
"crypto/cipher"
"github.com/rfjakob/gocryptfs/v2/internal/exitcodes" "github.com/rfjakob/gocryptfs/v2/internal/exitcodes"
) )
type StupidGCM struct{}
const ( const (
// BuiltWithoutOpenssl indicates if openssl been disabled at compile-time // BuiltWithoutOpenssl indicates if openssl been disabled at compile-time
BuiltWithoutOpenssl = true BuiltWithoutOpenssl = true
) )
func errExit() { func errExit() {
fmt.Fprintln(os.Stderr, "gocryptfs has been compiled without openssl support but you are still trying to use openssl") fmt.Fprintln(os.Stderr, "I have been compiled without openssl support but you are still trying to use openssl")
os.Exit(exitcodes.OpenSSL) os.Exit(exitcodes.OpenSSL)
} }
func New(_ []byte, _ bool) *StupidGCM { func NewAES256GCM(_ []byte) cipher.AEAD {
errExit()
// Never reached
return &StupidGCM{}
}
func (g *StupidGCM) NonceSize() int {
errExit()
return -1
}
func (g *StupidGCM) Overhead() int {
errExit()
return -1
}
func (g *StupidGCM) Seal(_, _, _, _ []byte) []byte {
errExit() errExit()
return nil return nil
} }
func (g *StupidGCM) Open(_, _, _, _ []byte) ([]byte, error) { func NewChacha20poly1305(_ []byte) cipher.AEAD {
errExit() errExit()
return nil, nil return nil
} }
func (g *StupidGCM) Wipe() { func NewXchacha20poly1305(_ []byte) cipher.AEAD {
errExit() errExit()
return nil
} }

View File

@ -0,0 +1,117 @@
// +build !without_openssl
// Copyright 2018 The Go Authors. All rights reserved.
// Use of this source code is governed by a BSD-style
// license that can be found in the LICENSE file.
//
// Copied from
// https://github.com/golang/crypto/blob/32db794688a5a24a23a43f2a984cecd5b3d8da58/chacha20poly1305/xchacha20poly1305.go
// and adapted for stupidgcm by @rfjakob.
package stupidgcm
import (
"crypto/cipher"
"errors"
"log"
"golang.org/x/crypto/chacha20"
"golang.org/x/crypto/chacha20poly1305"
)
type stupidXchacha20poly1305 struct {
// array instead of byte slice like
// `struct xchacha20poly1305` in x/crypto/chacha20poly1305
key [chacha20poly1305.KeySize]byte
wiped bool
}
// NewXchacha20poly1305 returns a XChaCha20-Poly1305 cipher that satisfied the
// cipher.AEAD interface.
//
// XChaCha20-Poly1305 is a ChaCha20-Poly1305 variant that takes a longer nonce,
// suitable to be generated randomly without risk of collisions. It should be
// preferred when nonce uniqueness cannot be trivially ensured, or whenever
// nonces are randomly generated.
//
// Only 32-bytes keys and 24-byte IVs are supported.
func NewXchacha20poly1305(key []byte) cipher.AEAD {
if len(key) != chacha20poly1305.KeySize {
log.Panic("bad key length")
}
ret := new(stupidXchacha20poly1305)
copy(ret.key[:], key)
return ret
}
func (*stupidXchacha20poly1305) NonceSize() int {
return chacha20poly1305.NonceSizeX
}
func (*stupidXchacha20poly1305) Overhead() int {
return tagLen
}
func (x *stupidXchacha20poly1305) Seal(dst, nonce, plaintext, additionalData []byte) []byte {
if x.wiped {
log.Panic("BUG: tried to use wiped key")
}
if len(nonce) != chacha20poly1305.NonceSizeX {
log.Panic("bad nonce length passed to Seal")
}
// XChaCha20-Poly1305 technically supports a 64-bit counter, so there is no
// size limit. However, since we reuse the ChaCha20-Poly1305 implementation,
// the second half of the counter is not available. This is unlikely to be
// an issue because the cipher.AEAD API requires the entire message to be in
// memory, and the counter overflows at 256 GB.
if uint64(len(plaintext)) > (1<<38)-64 {
log.Panic("plaintext too large")
}
hKey, _ := chacha20.HChaCha20(x.key[:], nonce[0:16])
c := NewChacha20poly1305(hKey).(*stupidChacha20poly1305)
defer c.Wipe()
// The first 4 bytes of the final nonce are unused counter space.
cNonce := make([]byte, chacha20poly1305.NonceSize)
copy(cNonce[4:12], nonce[16:24])
return c.Seal(dst, cNonce[:], plaintext, additionalData)
}
func (x *stupidXchacha20poly1305) Open(dst, nonce, ciphertext, additionalData []byte) ([]byte, error) {
if x.wiped {
log.Panic("BUG: tried to use wiped key")
}
if len(nonce) != chacha20poly1305.NonceSizeX {
log.Panic("bad nonce length passed to Open")
}
if len(ciphertext) < 16 {
return nil, errors.New("message too short")
}
if uint64(len(ciphertext)) > (1<<38)-48 {
log.Panic("ciphertext too large")
}
hKey, _ := chacha20.HChaCha20(x.key[:], nonce[0:16])
c := NewChacha20poly1305(hKey).(*stupidChacha20poly1305)
defer c.Wipe()
// The first 4 bytes of the final nonce are unused counter space.
cNonce := make([]byte, chacha20poly1305.NonceSize)
copy(cNonce[4:12], nonce[16:24])
return c.Open(dst, cNonce[:], ciphertext, additionalData)
}
// Wipe tries to wipe the key from memory by overwriting it with zeros.
//
// This is not bulletproof due to possible GC copies, but
// still raises the bar for extracting the key.
func (g *stupidXchacha20poly1305) Wipe() {
g.wiped = true
for i := range g.key {
g.key[i] = 0
}
}

View File

@ -0,0 +1,20 @@
// +build !without_openssl
package stupidgcm
import (
"testing"
"golang.org/x/crypto/chacha20poly1305"
)
func TestStupidXchacha20poly1305(t *testing.T) {
key := randBytes(32)
c := NewXchacha20poly1305(key)
ref, err := chacha20poly1305.NewX(key)
if err != nil {
t.Fatal(err)
}
testCiphers(t, c, ref)
}

View File

@ -12,7 +12,7 @@ import (
// https://github.com/golang/go/blob/d2a80f3fb5b44450e0b304ac5a718f99c053d82a/src/os/file_posix.go#L243 // https://github.com/golang/go/blob/d2a80f3fb5b44450e0b304ac5a718f99c053d82a/src/os/file_posix.go#L243
// //
// This is needed because CIFS throws lots of EINTR errors: // This is needed because CIFS throws lots of EINTR errors:
// https://github.com/rfjakob/gocryptfs/v2/issues/483 // https://github.com/rfjakob/gocryptfs/issues/483
// //
// Don't use retryEINTR() with syscall.Close()! // Don't use retryEINTR() with syscall.Close()!
// See https://code.google.com/p/chromium/issues/detail?id=269623 . // See https://code.google.com/p/chromium/issues/detail?id=269623 .

View File

@ -23,7 +23,7 @@ const sizeofDirent = int(unsafe.Sizeof(unix.Dirent{}))
// maxReclen sanity check: Reclen should never be larger than this. // maxReclen sanity check: Reclen should never be larger than this.
// Due to padding between entries, it is 280 even on 32-bit architectures. // Due to padding between entries, it is 280 even on 32-bit architectures.
// See https://github.com/rfjakob/gocryptfs/v2/issues/197 for details. // See https://github.com/rfjakob/gocryptfs/issues/197 for details.
const maxReclen = 280 const maxReclen = 280
// getdents wraps unix.Getdents and converts the result to []fuse.DirEntry. // getdents wraps unix.Getdents and converts the result to []fuse.DirEntry.
@ -43,7 +43,7 @@ func getdents(fd int) (entries []fuse.DirEntry, entriesSpecial []fuse.DirEntry,
continue continue
} else if err != nil { } else if err != nil {
if smartBuf.Len() > 0 { if smartBuf.Len() > 0 {
tlog.Warn.Printf("warning: unix.Getdents returned errno %d in the middle of data ( https://github.com/rfjakob/gocryptfs/v2/issues/483 )", err.(syscall.Errno)) tlog.Warn.Printf("warning: unix.Getdents returned errno %d in the middle of data ( https://github.com/rfjakob/gocryptfs/issues/483 )", err.(syscall.Errno))
return nil, nil, syscall.EIO return nil, nil, syscall.EIO
} }
return nil, nil, err return nil, nil, err
@ -145,7 +145,7 @@ func dtUnknownWarn(dirfd int) {
if err == nil && buf.Type == XFS_SUPER_MAGIC { if err == nil && buf.Type == XFS_SUPER_MAGIC {
// Old XFS filesystems always return DT_UNKNOWN. Downgrade the message // Old XFS filesystems always return DT_UNKNOWN. Downgrade the message
// to "info" level if we are on XFS. // to "info" level if we are on XFS.
// https://github.com/rfjakob/gocryptfs/v2/issues/267 // https://github.com/rfjakob/gocryptfs/issues/267
tlog.Info.Printf("Getdents: convertDType: received DT_UNKNOWN, fstype=xfs, falling back to stat") tlog.Info.Printf("Getdents: convertDType: received DT_UNKNOWN, fstype=xfs, falling back to stat")
} else { } else {
tlog.Warn.Printf("Getdents: convertDType: received DT_UNKNOWN, fstype=%#x, falling back to stat", tlog.Warn.Printf("Getdents: convertDType: received DT_UNKNOWN, fstype=%#x, falling back to stat",

View File

@ -27,7 +27,7 @@ func TestGetdents(t *testing.T) {
// skipOnGccGo skips the emulateGetdents test when we are // skipOnGccGo skips the emulateGetdents test when we are
// running linux and were compiled with gccgo. The test is known to fail // running linux and were compiled with gccgo. The test is known to fail
// (https://github.com/rfjakob/gocryptfs/v2/issues/201), but getdents emulation // (https://github.com/rfjakob/gocryptfs/issues/201), but getdents emulation
// is not used on linux, so let's skip the test and ignore the failure. // is not used on linux, so let's skip the test and ignore the failure.
func skipOnGccGo(t *testing.T) { func skipOnGccGo(t *testing.T) {
if !emulate || runtime.GOOS != "linux" { if !emulate || runtime.GOOS != "linux" {

View File

@ -6,12 +6,12 @@ import (
const ( const (
// QuirkBrokenFalloc means the falloc is broken. // QuirkBrokenFalloc means the falloc is broken.
// Preallocation on Btrfs is broken ( https://github.com/rfjakob/gocryptfs/v2/issues/395 ) // Preallocation on Btrfs is broken ( https://github.com/rfjakob/gocryptfs/issues/395 )
// and slow ( https://github.com/rfjakob/gocryptfs/v2/issues/63 ). // and slow ( https://github.com/rfjakob/gocryptfs/issues/63 ).
QuirkBrokenFalloc = uint64(1 << iota) QuirkBrokenFalloc = uint64(1 << iota)
// QuirkDuplicateIno1 means that we have duplicate inode numbers. // QuirkDuplicateIno1 means that we have duplicate inode numbers.
// On MacOS ExFAT, all empty files share inode number 1: // On MacOS ExFAT, all empty files share inode number 1:
// https://github.com/rfjakob/gocryptfs/v2/issues/585 // https://github.com/rfjakob/gocryptfs/issues/585
QuirkDuplicateIno1 QuirkDuplicateIno1
) )

View File

@ -8,7 +8,7 @@ import (
func DetectQuirks(cipherdir string) (q uint64) { func DetectQuirks(cipherdir string) (q uint64) {
const ( const (
// From https://github.com/rfjakob/gocryptfs/v2/issues/585#issuecomment-887370065 // From https://github.com/rfjakob/gocryptfs/issues/585#issuecomment-887370065
FstypenameExfat = "exfat" FstypenameExfat = "exfat"
) )
@ -31,9 +31,9 @@ func DetectQuirks(cipherdir string) (q uint64) {
tlog.Debug.Printf("DetectQuirks: Fstypename=%q\n", fstypename) tlog.Debug.Printf("DetectQuirks: Fstypename=%q\n", fstypename)
// On MacOS ExFAT, all empty files share inode number 1: // On MacOS ExFAT, all empty files share inode number 1:
// https://github.com/rfjakob/gocryptfs/v2/issues/585 // https://github.com/rfjakob/gocryptfs/issues/585
if fstypename == FstypenameExfat { if fstypename == FstypenameExfat {
logQuirk("ExFAT detected, disabling hard links. See https://github.com/rfjakob/gocryptfs/v2/issues/585 for why.") logQuirk("ExFAT detected, disabling hard links. See https://github.com/rfjakob/gocryptfs/issues/585 for why.")
q |= QuirkDuplicateIno1 q |= QuirkDuplicateIno1
} }

View File

@ -18,12 +18,12 @@ func DetectQuirks(cipherdir string) (q uint64) {
return 0 return 0
} }
// Preallocation on Btrfs is broken ( https://github.com/rfjakob/gocryptfs/v2/issues/395 ) // Preallocation on Btrfs is broken ( https://github.com/rfjakob/gocryptfs/issues/395 )
// and slow ( https://github.com/rfjakob/gocryptfs/v2/issues/63 ). // and slow ( https://github.com/rfjakob/gocryptfs/issues/63 ).
// //
// Cast to uint32 avoids compile error on arm: "constant 2435016766 overflows int32" // Cast to uint32 avoids compile error on arm: "constant 2435016766 overflows int32"
if uint32(st.Type) == unix.BTRFS_SUPER_MAGIC { if uint32(st.Type) == unix.BTRFS_SUPER_MAGIC {
logQuirk("Btrfs detected, forcing -noprealloc. See https://github.com/rfjakob/gocryptfs/v2/issues/395 for why.") logQuirk("Btrfs detected, forcing -noprealloc. See https://github.com/rfjakob/gocryptfs/issues/395 for why.")
q |= QuirkBrokenFalloc q |= QuirkBrokenFalloc
} }

View File

@ -59,7 +59,7 @@ func setattrlist(path *byte, list unsafe.Pointer, buf unsafe.Pointer, size uintp
// Sorry, fallocate is not available on OSX at all and // Sorry, fallocate is not available on OSX at all and
// fcntl F_PREALLOCATE is not accessible from Go. // fcntl F_PREALLOCATE is not accessible from Go.
// See https://github.com/rfjakob/gocryptfs/v2/issues/18 if you want to help. // See https://github.com/rfjakob/gocryptfs/issues/18 if you want to help.
func EnospcPrealloc(fd int, off int64, len int64) error { func EnospcPrealloc(fd int, off int64, len int64) error {
return nil return nil
} }

View File

@ -48,7 +48,7 @@ func EnospcPrealloc(fd int, off int64, len int64) (err error) {
} }
if err == syscall.EOPNOTSUPP { if err == syscall.EOPNOTSUPP {
// ZFS and ext3 do not support fallocate. Warn but continue anyway. // ZFS and ext3 do not support fallocate. Warn but continue anyway.
// https://github.com/rfjakob/gocryptfs/v2/issues/22 // https://github.com/rfjakob/gocryptfs/issues/22
preallocWarn.Do(func() { preallocWarn.Do(func() {
tlog.Warn.Printf("Warning: The underlying filesystem " + tlog.Warn.Printf("Warning: The underlying filesystem " +
"does not support fallocate(2). gocryptfs will continue working " + "does not support fallocate(2). gocryptfs will continue working " +

View File

@ -1,5 +1,5 @@
// gocryptfs is an encrypted overlay filesystem written in Go. // gocryptfs is an encrypted overlay filesystem written in Go.
// See README.md ( https://github.com/rfjakob/gocryptfs/v2/blob/master/README.md ) // See README.md ( https://github.com/rfjakob/gocryptfs/blob/master/README.md )
// and the official website ( https://nuetzlich.net/gocryptfs/ ) for details. // and the official website ( https://nuetzlich.net/gocryptfs/ ) for details.
package main package main

View File

@ -75,7 +75,7 @@ func doMount(args *argContainer) {
// and `drop_privileges` in `man mount.fuse3` for background. // and `drop_privileges` in `man mount.fuse3` for background.
} else { } else {
err = isEmptyDir(args.mountpoint) err = isEmptyDir(args.mountpoint)
// OSXFuse will create the mountpoint for us ( https://github.com/rfjakob/gocryptfs/v2/issues/194 ) // OSXFuse will create the mountpoint for us ( https://github.com/rfjakob/gocryptfs/issues/194 )
if runtime.GOOS == "darwin" && os.IsNotExist(err) { if runtime.GOOS == "darwin" && os.IsNotExist(err) {
tlog.Info.Printf("Mountpoint %q does not exist, but should be created by OSXFuse", tlog.Info.Printf("Mountpoint %q does not exist, but should be created by OSXFuse",
args.mountpoint) args.mountpoint)
@ -259,7 +259,11 @@ func initFuseFrontend(args *argContainer) (rootNode fs.InodeEmbedder, wipeKeys f
cryptoBackend = cryptocore.BackendAESSIV cryptoBackend = cryptocore.BackendAESSIV
} }
if args.xchacha { if args.xchacha {
if args.openssl {
cryptoBackend = cryptocore.BackendXChaCha20Poly1305OpenSSL
} else {
cryptoBackend = cryptocore.BackendXChaCha20Poly1305 cryptoBackend = cryptocore.BackendXChaCha20Poly1305
}
IVBits = chacha20poly1305.NonceSizeX * 8 IVBits = chacha20poly1305.NonceSizeX * 8
} }
// forceOwner implies allow_other, as documented. // forceOwner implies allow_other, as documented.
@ -273,8 +277,6 @@ func initFuseFrontend(args *argContainer) (rootNode fs.InodeEmbedder, wipeKeys f
LongNames: args.longnames, LongNames: args.longnames,
ConfigCustom: args._configCustom, ConfigCustom: args._configCustom,
NoPrealloc: args.noprealloc, NoPrealloc: args.noprealloc,
SerializeReads: args.serialize_reads,
ForceDecode: args.forcedecode,
ForceOwner: args._forceOwner, ForceOwner: args._forceOwner,
Exclude: args.exclude, Exclude: args.exclude,
ExcludeWildcard: args.excludeWildcard, ExcludeWildcard: args.excludeWildcard,
@ -292,6 +294,7 @@ func initFuseFrontend(args *argContainer) (rootNode fs.InodeEmbedder, wipeKeys f
frontendArgs.DeterministicNames = !confFile.IsFeatureFlagSet(configfile.FlagDirIV) frontendArgs.DeterministicNames = !confFile.IsFeatureFlagSet(configfile.FlagDirIV)
args.raw64 = confFile.IsFeatureFlagSet(configfile.FlagRaw64) args.raw64 = confFile.IsFeatureFlagSet(configfile.FlagRaw64)
args.hkdf = confFile.IsFeatureFlagSet(configfile.FlagHKDF) args.hkdf = confFile.IsFeatureFlagSet(configfile.FlagHKDF)
// Note: this will always return the non-openssl variant
cryptoBackend, err = confFile.ContentEncryption() cryptoBackend, err = confFile.ContentEncryption()
if err != nil { if err != nil {
tlog.Fatal.Printf("%v", err) tlog.Fatal.Printf("%v", err)
@ -302,8 +305,14 @@ func initFuseFrontend(args *argContainer) (rootNode fs.InodeEmbedder, wipeKeys f
tlog.Fatal.Printf("AES-SIV is required by reverse mode, but not enabled in the config file") tlog.Fatal.Printf("AES-SIV is required by reverse mode, but not enabled in the config file")
os.Exit(exitcodes.Usage) os.Exit(exitcodes.Usage)
} }
if cryptoBackend == cryptocore.BackendGoGCM && args.openssl { // Upgrade to OpenSSL variant if requested
if args.openssl {
switch cryptoBackend {
case cryptocore.BackendGoGCM:
cryptoBackend = cryptocore.BackendOpenSSL cryptoBackend = cryptocore.BackendOpenSSL
case cryptocore.BackendXChaCha20Poly1305:
cryptoBackend = cryptocore.BackendXChaCha20Poly1305OpenSSL
}
} }
} }
// If allow_other is set and we run as root, try to give newly created files to // If allow_other is set and we run as root, try to give newly created files to
@ -313,8 +322,8 @@ func initFuseFrontend(args *argContainer) (rootNode fs.InodeEmbedder, wipeKeys f
} }
// Init crypto backend // Init crypto backend
cCore := cryptocore.New(masterkey, cryptoBackend, IVBits, args.hkdf, args.forcedecode) cCore := cryptocore.New(masterkey, cryptoBackend, IVBits, args.hkdf)
cEnc := contentenc.New(cCore, contentenc.DefaultBS, args.forcedecode) cEnc := contentenc.New(cCore, contentenc.DefaultBS)
nameTransform := nametransform.New(cCore.EMECipher, frontendArgs.LongNames, nameTransform := nametransform.New(cCore.EMECipher, frontendArgs.LongNames,
args.raw64, []string(args.badname), frontendArgs.DeterministicNames) args.raw64, []string(args.badname), frontendArgs.DeterministicNames)
// After the crypto backend is initialized, // After the crypto backend is initialized,
@ -377,6 +386,14 @@ func initGoFuse(rootNode fs.InodeEmbedder, args *argContainer) *fuse.Server {
MaxWrite: fuse.MAX_KERNEL_WRITE, MaxWrite: fuse.MAX_KERNEL_WRITE,
Options: []string{fmt.Sprintf("max_read=%d", fuse.MAX_KERNEL_WRITE)}, Options: []string{fmt.Sprintf("max_read=%d", fuse.MAX_KERNEL_WRITE)},
Debug: args.fusedebug, Debug: args.fusedebug,
// The kernel usually submits multiple read requests in parallel,
// which means we serve them in any order. Out-of-order reads are
// expensive on some backing network filesystems
// ( https://github.com/rfjakob/gocryptfs/issues/92 ).
//
// Setting SyncRead disables FUSE_CAP_ASYNC_READ. This makes the kernel
// do everything in-order without parallelism.
SyncRead: args.serialize_reads,
} }
mOpts := &fuseOpts.MountOptions mOpts := &fuseOpts.MountOptions
@ -390,10 +407,6 @@ func initGoFuse(rootNode fs.InodeEmbedder, args *argContainer) *fuse.Server {
if args.acl { if args.acl {
mOpts.EnableAcl = true mOpts.EnableAcl = true
} }
if args.forcedecode {
tlog.Info.Printf(tlog.ColorYellow + "THE OPTION \"-forcedecode\" IS ACTIVE. GOCRYPTFS WILL RETURN CORRUPT DATA!" +
tlog.ColorReset)
}
// fusermount from libfuse 3.x removed the "nonempty" option and exits // fusermount from libfuse 3.x removed the "nonempty" option and exits
// with an error if it sees it. Only add it to the options on libfuse 2.x. // with an error if it sees it. Only add it to the options on libfuse 2.x.
if args.nonempty && haveFusermount2() { if args.nonempty && haveFusermount2() {

View File

@ -6,10 +6,12 @@ cd "$(dirname "$0")"
../tests/dl-linux-tarball.bash ../tests/dl-linux-tarball.bash
T=$(mktemp -d) T=$(mktemp -d)
mkdir $T/a $T/b mkdir "$T/a" "$T/b"
../gocryptfs -init -quiet -scryptn 10 -extpass "echo test" $T/a set -x
../gocryptfs -quiet -nosyslog -extpass "echo test" $T/a $T/b ../gocryptfs -init -quiet -scryptn 10 -extpass "echo test" "$@" "$T/a"
{ set +x ; } 2> /dev/null
../gocryptfs -quiet -nosyslog -extpass "echo test" "$@" "$T/a" "$T/b"
# Cleanup trap # Cleanup trap
trap "cd /; fusermount -u -z $T/b; rm -Rf $T/a" EXIT trap "cd /; fusermount -u -z $T/b; rm -Rf $T/a" EXIT
@ -17,20 +19,22 @@ trap "cd /; fusermount -u -z $T/b; rm -Rf $T/a" EXIT
echo "Creating 40000 empty files (linux-3.0.tar.gz contains 36782 files)..." echo "Creating 40000 empty files (linux-3.0.tar.gz contains 36782 files)..."
SECONDS=0 SECONDS=0
for dir in $(seq -w 1 200); do for dir in $(seq -w 1 200); do
mkdir $T/b/$dir mkdir "$T/b/$dir"
( cd $T/b/$dir ; touch $(seq -w 1 200) ) ( cd "$T/b/$dir" ; touch $(seq -w 1 200) )
done done
echo "done, $SECONDS seconds" echo "done, $SECONDS seconds"
echo "Remount..." echo "Remount..."
fusermount -u $T/b fusermount -u "$T/b"
../gocryptfs -quiet -nosyslog -extpass "echo test" -cpuprofile $T/cprof -memprofile $T/mprof \ set -x
$T/a $T/b ../gocryptfs -quiet -nosyslog -extpass "echo test" -cpuprofile "$T/cprof" -memprofile "$T/mprof" \
"$@" "$T/a" "$T/b"
{ set +x ; } 2> /dev/null
echo "Running ls under profiler (3x)..." echo "Running ls under profiler (3x)..."
for i in 1 2 3; do for i in 1 2 3; do
SECONDS=0 SECONDS=0
ls -lR $T/b > /dev/null ls -lR "$T/b" > /dev/null
echo "$i done, $SECONDS seconds" echo "$i done, $SECONDS seconds"
done done

View File

@ -3,25 +3,29 @@
cd "$(dirname "$0")" cd "$(dirname "$0")"
T=$(mktemp -d) T=$(mktemp -d)
mkdir $T/a $T/b mkdir "$T/a" "$T/b"
../gocryptfs -init -quiet -scryptn 10 -extpass "echo test" $T/a set -x
../gocryptfs -quiet -extpass "echo test" $T/a $T/b ../gocryptfs -init -quiet -scryptn 10 -extpass "echo test" "$@" "$T/a"
{ set +x ; } 2> /dev/null
../gocryptfs -quiet -extpass "echo test" "$@" "$T/a" "$T/b"
# Cleanup trap # Cleanup trap
trap "cd /; fusermount -u -z $T/b; rm -Rf $T/a" EXIT trap "cd /; fusermount -u -z $T/b; rm -Rf $T/a" EXIT
# Write 100MB test file # Write 100MB test file
dd if=/dev/zero of=$T/b/zero bs=1M count=100 status=none dd if=/dev/zero of="$T/b/zero" bs=1M count=100 status=none
# Remount with profiling # Remount with profiling
fusermount -u $T/b fusermount -u "$T/b"
../gocryptfs -quiet -extpass "echo test" -cpuprofile $T/cprof -memprofile $T/mprof \ set -x
$T/a $T/b ../gocryptfs -quiet -extpass "echo test" -cpuprofile "$T/cprof" -memprofile "$T/mprof" \
"$@" "$T/a" "$T/b"
{ set +x ; } 2> /dev/null
# Read 10 x 100MB instead of 1 x 1GB to keep the used disk space low # Read 10 x 100MB instead of 1 x 1GB to keep the used disk space low
for i in $(seq 1 10); do for i in $(seq 1 10); do
dd if=$T/b/zero of=/dev/null bs=1M count=100 dd if="$T/b/zero" of=/dev/null bs=1M count=100
done done
echo echo

View File

@ -3,18 +3,20 @@
cd "$(dirname "$0")" cd "$(dirname "$0")"
T=$(mktemp -d) T=$(mktemp -d)
mkdir $T/a $T/b mkdir "$T/a" "$T/b"
../gocryptfs -init -quiet -scryptn 10 -extpass "echo test" $T/a set -x
../gocryptfs -quiet -extpass "echo test" -cpuprofile $T/cprof -memprofile $T/mprof \ ../gocryptfs -init -quiet -scryptn 10 -extpass "echo test" "$@" "$T/a"
$T/a $T/b ../gocryptfs -quiet -extpass "echo test" -cpuprofile "$T/cprof" -memprofile "$T/mprof" \
"$@" "$T/a" "$T/b"
{ set +x ; } 2> /dev/null
# Cleanup trap # Cleanup trap
trap "cd /; fusermount -u -z $T/b; rm -Rf $T/a" EXIT trap "cd /; fusermount -u -z $T/b; rm -Rf $T/a" EXIT
# Write 10 x 100MB instead of 1 x 1GB to keep the used disk space low # Write 10 x 100MB instead of 1 x 1GB to keep the used disk space low
for i in $(seq 1 10); do for i in $(seq 1 10); do
dd if=/dev/zero of=$T/b/zero bs=1M count=100 dd if=/dev/zero of="$T/b/zero" bs=1M count=100
done done
echo echo

View File

@ -6,17 +6,19 @@ cd "$(dirname "$0")"
../tests/dl-linux-tarball.bash ../tests/dl-linux-tarball.bash
T=$(mktemp -d) T=$(mktemp -d)
mkdir $T/a $T/b mkdir "$T/a" "$T/b"
../gocryptfs -init -quiet -scryptn 10 -extpass "echo test" $T/a set -x
../gocryptfs -quiet -extpass "echo test" -cpuprofile $T/cprof -memprofile $T/mprof \ ../gocryptfs -init -quiet -scryptn 10 -extpass "echo test" "$@" "$T/a"
$T/a $T/b ../gocryptfs -quiet -extpass "echo test" -cpuprofile "$T/cprof" -memprofile "$T/mprof" \
"$@" "$T/a" "$T/b"
{ set +x ; } 2> /dev/null
# Cleanup trap # Cleanup trap
trap "cd /; fusermount -u -z $T/b; rm -Rf $T/a" EXIT trap "cd /; fusermount -u -z $T/b; rm -Rf $T/a" EXIT
echo "Extracting..." echo "Extracting..."
time tar xzf /tmp/linux-3.0.tar.gz -C $T/b time tar xzf /tmp/linux-3.0.tar.gz -C "$T/b"
echo echo
echo "Hint: go tool pprof ../gocryptfs $T/cprof" echo "Hint: go tool pprof ../gocryptfs $T/cprof"

View File

@ -6,18 +6,19 @@
cd "$(dirname "$0")" cd "$(dirname "$0")"
T=$(mktemp -d) T=$(mktemp -d)
mkdir $T/a $T/b mkdir "$T/a" "$T/b"
../gocryptfs -init -quiet -scryptn 10 -extpass "echo test" $T/a set -x
../gocryptfs -quiet -extpass "echo test" -trace $T/trace \ ../gocryptfs -init -quiet -scryptn 10 -extpass "echo test" "$@" "$T/a"
$T/a $T/b ../gocryptfs -quiet -extpass "echo test" -trace "$T/trace" \
"$@" "$T/a" "$T/b"
{ set +x ; } 2> /dev/null
# Cleanup trap # Cleanup trap
trap "cd /; fusermount -u -z $T/b; rm -Rf $T/a" EXIT trap "cd /; fusermount -u -z $T/b; rm -Rf $T/a" EXIT
# Write only 1x100MB, otherwise the trace gets too big. # Write only 1x100MB, otherwise the trace gets too big.
dd if=/dev/zero of=$T/b/zero bs=1M count=100 dd if=/dev/zero of="$T/b/zero" bs=1M count=100
echo echo
echo "Hint: go tool trace $T/trace" echo "Hint: go tool trace $T/trace"

View File

@ -16,7 +16,7 @@ TESTDIR=$TMPDIR/gocryptfs-test-parent-$UID
mkdir -p "$TESTDIR" mkdir -p "$TESTDIR"
LOCKFILE=$TESTDIR/$MYNAME.lock LOCKFILE=$TESTDIR/$MYNAME.lock
function unmount_leftovers { unmount_leftovers() {
RET=0 RET=0
for i in $(mount | grep "$TESTDIR" | cut -f3 -d" "); do for i in $(mount | grep "$TESTDIR" | cut -f3 -d" "); do
echo "Warning: unmounting leftover filesystem: $i" echo "Warning: unmounting leftover filesystem: $i"
@ -28,7 +28,7 @@ function unmount_leftovers {
( (
# Prevent multiple parallel test.bash instances as this causes # Prevent multiple parallel test.bash instances as this causes
# all kinds of mayham # all kinds of mayhem
if ! command -v flock > /dev/null ; then if ! command -v flock > /dev/null ; then
echo "flock is not available, skipping" echo "flock is not available, skipping"
elif ! flock -n 200 ; then elif ! flock -n 200 ; then
@ -39,7 +39,10 @@ fi
# Clean up dangling filesystems and don't exit if we found some # Clean up dangling filesystems and don't exit if we found some
unmount_leftovers || true unmount_leftovers || true
./build-without-openssl.bash ./build-without-openssl.bash || {
echo "$MYNAME: build-without-openssl.bash failed"
exit 1
}
# Don't build with openssl if we were passed "-tags without_openssl" # Don't build with openssl if we were passed "-tags without_openssl"
if [[ "$*" != *without_openssl* ]] ; then if [[ "$*" != *without_openssl* ]] ; then
./build.bash ./build.bash
@ -79,7 +82,7 @@ else
rm -Rf "$TESTDIR" rm -Rf "$TESTDIR"
fi fi
if grep -R "panic(" ./*.go internal ; then if find internal -type f -name \*.go -print0 | xargs -0 grep "panic("; then
echo "$MYNAME: Please use log.Panic instead of naked panic!" echo "$MYNAME: Please use log.Panic instead of naked panic!"
exit 1 exit 1
fi fi

View File

@ -45,7 +45,7 @@ echo -n "UNTAR: "
etime tar xzf /tmp/linux-3.0.tar.gz etime tar xzf /tmp/linux-3.0.tar.gz
sleep 0.1 sleep 0.1
echo -n "MD5: " echo -n "MD5: "
etime md5sum --quiet -c $MD5 etime md5sum --quiet -c "$MD5"
sleep 0.1 sleep 0.1
echo -n "LS: " echo -n "LS: "
etime ls -lR linux-3.0 etime ls -lR linux-3.0

View File

@ -60,9 +60,9 @@ func TestInitFilePerms(t *testing.T) {
syscall.Stat(dir+"/gocryptfs.diriv", &st) syscall.Stat(dir+"/gocryptfs.diriv", &st)
perms = st.Mode & 0777 perms = st.Mode & 0777
// From v1.7.1, these are created with 0440 permissions, see // From v1.7.1, these are created with 0440 permissions, see
// https://github.com/rfjakob/gocryptfs/v2/issues/387 . // https://github.com/rfjakob/gocryptfs/issues/387 .
// From v2.0, created with 0444 perms, see // From v2.0, created with 0444 perms, see
// https://github.com/rfjakob/gocryptfs/v2/issues/539 . // https://github.com/rfjakob/gocryptfs/issues/539 .
if perms != 0444 { if perms != 0444 {
t.Errorf("Wrong permissions for gocryptfs.diriv: %#o", perms) t.Errorf("Wrong permissions for gocryptfs.diriv: %#o", perms)
} }
@ -441,7 +441,7 @@ func TestPasswdPasswordIncorrect(t *testing.T) {
// Check that we correctly background on mount and close stderr and stdout. // Check that we correctly background on mount and close stderr and stdout.
// Something like // Something like
// gocryptfs a b | cat // gocryptfs a b | cat
// must not hang ( https://github.com/rfjakob/gocryptfs/v2/issues/130 ). // must not hang ( https://github.com/rfjakob/gocryptfs/issues/130 ).
func TestMountBackground(t *testing.T) { func TestMountBackground(t *testing.T) {
dir := test_helpers.InitFS(t) dir := test_helpers.InitFS(t)
mnt := dir + ".mnt" mnt := dir + ".mnt"
@ -557,7 +557,7 @@ func TestExcludeForward(t *testing.T) {
} }
// Check that the config file can be read from a named pipe. // Check that the config file can be read from a named pipe.
// Make sure bug https://github.com/rfjakob/gocryptfs/v2/issues/258 does not come // Make sure bug https://github.com/rfjakob/gocryptfs/issues/258 does not come
// back. // back.
func TestConfigPipe(t *testing.T) { func TestConfigPipe(t *testing.T) {
dir := test_helpers.InitFS(t) dir := test_helpers.InitFS(t)
@ -580,7 +580,7 @@ func TestConfigPipe(t *testing.T) {
} }
// Ciphertext dir and mountpoint contains a comma // Ciphertext dir and mountpoint contains a comma
// https://github.com/rfjakob/gocryptfs/v2/issues/262 // https://github.com/rfjakob/gocryptfs/issues/262
func TestComma(t *testing.T) { func TestComma(t *testing.T) {
dir0 := test_helpers.InitFS(t) dir0 := test_helpers.InitFS(t)
dir := dir0 + ",foo,bar" dir := dir0 + ",foo,bar"
@ -625,7 +625,7 @@ func TestIdle(t *testing.T) {
} }
// Mount with idle timeout of 100ms read something every 10ms. The fs should // Mount with idle timeout of 100ms read something every 10ms. The fs should
// NOT get unmounted. Regression test for https://github.com/rfjakob/gocryptfs/v2/issues/421 // NOT get unmounted. Regression test for https://github.com/rfjakob/gocryptfs/issues/421
func TestNotIdle(t *testing.T) { func TestNotIdle(t *testing.T) {
dir := test_helpers.InitFS(t) dir := test_helpers.InitFS(t)
mnt := dir + ".mnt" mnt := dir + ".mnt"
@ -663,7 +663,7 @@ func TestNotIdle(t *testing.T) {
// TestSymlinkedCipherdir checks that if CIPHERDIR itself is a symlink, it is // TestSymlinkedCipherdir checks that if CIPHERDIR itself is a symlink, it is
// followed. // followed.
// https://github.com/rfjakob/gocryptfs/v2/issues/450 // https://github.com/rfjakob/gocryptfs/issues/450
func TestSymlinkedCipherdir(t *testing.T) { func TestSymlinkedCipherdir(t *testing.T) {
dir := test_helpers.InitFS(t) dir := test_helpers.InitFS(t)
dirSymlink := dir + ".symlink" dirSymlink := dir + ".symlink"
@ -909,7 +909,7 @@ func TestPassfileX2(t *testing.T) {
} }
// TestInitNotEmpty checks that `gocryptfs -init` returns the right error code // TestInitNotEmpty checks that `gocryptfs -init` returns the right error code
// if CIPHERDIR is not empty. See https://github.com/rfjakob/gocryptfs/v2/pull/503 // if CIPHERDIR is not empty. See https://github.com/rfjakob/gocryptfs/pull/503
func TestInitNotEmpty(t *testing.T) { func TestInitNotEmpty(t *testing.T) {
dir := test_helpers.TmpDir + "/" + t.Name() dir := test_helpers.TmpDir + "/" + t.Name()
if err := os.Mkdir(dir, 0700); err != nil { if err := os.Mkdir(dir, 0700); err != nil {

View File

@ -17,7 +17,7 @@ import (
"github.com/rfjakob/gocryptfs/v2/tests/test_helpers" "github.com/rfjakob/gocryptfs/v2/tests/test_helpers"
) )
// https://github.com/rfjakob/gocryptfs/v2/issues/543 // https://github.com/rfjakob/gocryptfs/issues/543
func TestCpA(t *testing.T) { func TestCpA(t *testing.T) {
fn1 := filepath.Join(test_helpers.TmpDir, t.Name()) fn1 := filepath.Join(test_helpers.TmpDir, t.Name())
fn2 := filepath.Join(test_helpers.DefaultPlainDir, t.Name()) fn2 := filepath.Join(test_helpers.DefaultPlainDir, t.Name())
@ -77,7 +77,7 @@ func getfacl(fn string) (string, error) {
return string(out), err return string(out), err
} }
// https://github.com/rfjakob/gocryptfs/v2/issues/543 // https://github.com/rfjakob/gocryptfs/issues/543
func TestAcl543(t *testing.T) { func TestAcl543(t *testing.T) {
fn1 := test_helpers.TmpDir + "/TestAcl543" fn1 := test_helpers.TmpDir + "/TestAcl543"
fn2 := test_helpers.DefaultPlainDir + "/TestAcl543" fn2 := test_helpers.DefaultPlainDir + "/TestAcl543"

View File

@ -239,7 +239,7 @@ func TestMvWarningSymlink(t *testing.T) {
if err != nil { if err != nil {
t.Log(string(out)) t.Log(string(out))
if runtime.GOOS == "darwin" { if runtime.GOOS == "darwin" {
t.Skip("mv on darwin chokes on broken symlinks, see https://github.com/rfjakob/gocryptfs/v2/issues/349") t.Skip("mv on darwin chokes on broken symlinks, see https://github.com/rfjakob/gocryptfs/issues/349")
} }
t.Fatal(err) t.Fatal(err)
} }

View File

@ -10,18 +10,18 @@ SIZE_WANT=96675825
SIZE_ACTUAL=0 SIZE_ACTUAL=0
if [[ -e $TGZ ]]; then if [[ -e $TGZ ]]; then
if [[ $OSTYPE == linux* ]] ; then if [[ $OSTYPE == linux* ]] ; then
SIZE_ACTUAL=$(stat -c %s $TGZ) SIZE_ACTUAL=$(stat -c %s "$TGZ")
else else
# Mac OS X # Mac OS X
SIZE_ACTUAL=$(stat -f %z $TGZ) SIZE_ACTUAL=$(stat -f %z "$TGZ")
fi fi
fi fi
if [[ $SIZE_ACTUAL -ne $SIZE_WANT ]]; then if [[ $SIZE_ACTUAL -ne $SIZE_WANT ]]; then
echo "Downloading linux-3.0.tar.gz" echo "Downloading linux-3.0.tar.gz"
if command -v wget > /dev/null ; then if command -v wget > /dev/null ; then
wget -nv --show-progress -c -O $TGZ $URL wget -nv --show-progress -c -O "$TGZ" "$URL"
else else
curl -o $TGZ $URL curl -o "$TGZ" "$URL"
fi fi
fi fi

View File

@ -5,7 +5,7 @@
# #
# This script can be sourced or executed directly. # This script can be sourced or executed directly.
# #
function fuse-unmount { fuse-unmount() {
local MYNAME=$(basename "$BASH_SOURCE") local MYNAME=$(basename "$BASH_SOURCE")
if [[ $# -eq 0 ]] ; then if [[ $# -eq 0 ]] ; then
echo "$MYNAME: missing argument" echo "$MYNAME: missing argument"

View File

@ -20,10 +20,10 @@ fi
rm -f b/* rm -f b/*
while [[ $LEN -le 255 ]]; do while [[ $LEN -le 255 ]]; do
touch b/$NAME || break touch "b/$NAME" || break
ELEN=$(ls a | wc -L) ELEN=$(ls a | wc -L)
echo $LEN $ELEN echo "$LEN $ELEN"
rm b/$NAME rm "b/$NAME"
NAME="${NAME}x" NAME="${NAME}x"
LEN=${#NAME} LEN=${#NAME}
done done

View File

@ -12,7 +12,7 @@ import (
"github.com/rfjakob/gocryptfs/v2/tests/test_helpers" "github.com/rfjakob/gocryptfs/v2/tests/test_helpers"
) )
// https://github.com/rfjakob/gocryptfs/v2/issues/363 // https://github.com/rfjakob/gocryptfs/issues/363
// //
// Note: this test calls log.Fatal() instead of t.Fatal() because apparently, // Note: this test calls log.Fatal() instead of t.Fatal() because apparently,
// calling t.Fatal() from a goroutine hangs the test. // calling t.Fatal() from a goroutine hangs the test.
@ -73,7 +73,7 @@ func TestConcurrentReadWrite(t *testing.T) {
wg.Wait() wg.Wait()
} }
// https://github.com/rfjakob/gocryptfs/v2/issues/363 // https://github.com/rfjakob/gocryptfs/issues/363
// //
// Note: this test calls log.Fatal() instead of t.Fatal() because apparently, // Note: this test calls log.Fatal() instead of t.Fatal() because apparently,
// calling t.Fatal() from a goroutine hangs the test. // calling t.Fatal() from a goroutine hangs the test.
@ -110,6 +110,7 @@ func TestConcurrentReadCreate(t *testing.T) {
continue continue
} }
n, err := f.Read(buf0) n, err := f.Read(buf0)
f.Close()
if err == io.EOF { if err == io.EOF {
i++ i++
continue continue
@ -122,7 +123,6 @@ func TestConcurrentReadCreate(t *testing.T) {
// Calling t.Fatal() from a goroutine hangs the test so we use log.Fatal // Calling t.Fatal() from a goroutine hangs the test so we use log.Fatal
log.Fatalf("%s: content mismatch: have=%q want=%q", t.Name(), string(buf), string(content)) log.Fatalf("%s: content mismatch: have=%q want=%q", t.Name(), string(buf), string(content))
} }
f.Close()
} }
wg.Done() wg.Done()
}() }()

View File

@ -34,7 +34,7 @@ func TestDirOverwrite(t *testing.T) {
} }
// Test that we can create and remove a directory regardless of the permission it has // Test that we can create and remove a directory regardless of the permission it has
// https://github.com/rfjakob/gocryptfs/v2/issues/354 // https://github.com/rfjakob/gocryptfs/issues/354
func TestRmdirPerms(t *testing.T) { func TestRmdirPerms(t *testing.T) {
for _, perm := range []uint32{0000, 0100, 0200, 0300, 0400, 0500, 0600, 0700} { for _, perm := range []uint32{0000, 0100, 0200, 0300, 0400, 0500, 0600, 0700} {
dir := fmt.Sprintf("TestRmdir%#o", perm) dir := fmt.Sprintf("TestRmdir%#o", perm)

View File

@ -149,7 +149,7 @@ func TestFallocate(t *testing.T) {
} }
} }
// We used to allocate 18 bytes too much: // We used to allocate 18 bytes too much:
// https://github.com/rfjakob/gocryptfs/v2/issues/311 // https://github.com/rfjakob/gocryptfs/issues/311
// //
// 8110 bytes of plaintext should get us exactly 8192 bytes of ciphertext. // 8110 bytes of plaintext should get us exactly 8192 bytes of ciphertext.
err = file.Truncate(0) err = file.Truncate(0)

View File

@ -20,6 +20,7 @@ import (
"sync" "sync"
"syscall" "syscall"
"testing" "testing"
"time"
"golang.org/x/sys/unix" "golang.org/x/sys/unix"
@ -66,7 +67,9 @@ var matrix = []testcaseMatrix{
{false, "auto", false, false, []string{"-serialize_reads"}}, {false, "auto", false, false, []string{"-serialize_reads"}},
{false, "auto", false, false, []string{"-sharedstorage"}}, {false, "auto", false, false, []string{"-sharedstorage"}},
{false, "auto", false, false, []string{"-deterministic-names"}}, {false, "auto", false, false, []string{"-deterministic-names"}},
{false, "auto", false, true, []string{"-xchacha"}}, // Test xchacha with and without openssl
{false, "true", false, true, []string{"-xchacha"}},
{false, "false", false, true, []string{"-xchacha"}},
} }
// This is the entry point for the tests // This is the entry point for the tests
@ -97,7 +100,11 @@ func TestMain(m *testing.M) {
opts = append(opts, testcase.extraArgs...) opts = append(opts, testcase.extraArgs...)
test_helpers.MountOrExit(test_helpers.DefaultCipherDir, test_helpers.DefaultPlainDir, opts...) test_helpers.MountOrExit(test_helpers.DefaultCipherDir, test_helpers.DefaultPlainDir, opts...)
before := test_helpers.ListFds(0, test_helpers.TmpDir) before := test_helpers.ListFds(0, test_helpers.TmpDir)
t0 := time.Now()
r := m.Run() r := m.Run()
if testing.Verbose() {
fmt.Printf("matrix: run took %v\n", time.Since(t0))
}
// Catch fd leaks in the tests. NOTE: this does NOT catch leaks in // Catch fd leaks in the tests. NOTE: this does NOT catch leaks in
// the gocryptfs FUSE process, but only in the tests that access it! // the gocryptfs FUSE process, but only in the tests that access it!
// All fds that point outside TmpDir are not interesting (the Go test // All fds that point outside TmpDir are not interesting (the Go test
@ -774,7 +781,7 @@ func TestMkfifo(t *testing.T) {
} }
// TestMagicNames verifies that "magic" names are handled correctly // TestMagicNames verifies that "magic" names are handled correctly
// https://github.com/rfjakob/gocryptfs/v2/issues/174 // https://github.com/rfjakob/gocryptfs/issues/174
func TestMagicNames(t *testing.T) { func TestMagicNames(t *testing.T) {
names := []string{"warmup1", "warmup2", "gocryptfs.longname.QhUr5d9FHerwEs--muUs6_80cy6JRp89c1otLwp92Cs", "gocryptfs.diriv"} names := []string{"warmup1", "warmup2", "gocryptfs.longname.QhUr5d9FHerwEs--muUs6_80cy6JRp89c1otLwp92Cs", "gocryptfs.diriv"}
for _, n := range names { for _, n := range names {
@ -891,7 +898,7 @@ func TestStatfs(t *testing.T) {
} }
// gocryptfs 2.0 reported the ciphertext size on symlink creation, causing // gocryptfs 2.0 reported the ciphertext size on symlink creation, causing
// confusion: https://github.com/rfjakob/gocryptfs/v2/issues/574 // confusion: https://github.com/rfjakob/gocryptfs/issues/574
func TestSymlinkSize(t *testing.T) { func TestSymlinkSize(t *testing.T) {
p := filepath.Join(test_helpers.DefaultPlainDir, t.Name()) p := filepath.Join(test_helpers.DefaultPlainDir, t.Name())
// SYMLINK reports the size to the kernel // SYMLINK reports the size to the kernel
@ -911,7 +918,7 @@ func TestSymlinkSize(t *testing.T) {
// TestPwd check that /usr/bin/pwd works inside gocryptfs. // TestPwd check that /usr/bin/pwd works inside gocryptfs.
// //
// This was broken in gocryptfs v2.0 with -sharedstorage: // This was broken in gocryptfs v2.0 with -sharedstorage:
// https://github.com/rfjakob/gocryptfs/v2/issues/584 // https://github.com/rfjakob/gocryptfs/issues/584
func TestPwd(t *testing.T) { func TestPwd(t *testing.T) {
dir := test_helpers.DefaultPlainDir dir := test_helpers.DefaultPlainDir
for i := 0; i < 3; i++ { for i := 0; i < 3; i++ {

View File

@ -185,7 +185,7 @@ func TestEnoent(t *testing.T) {
// If the symlink target gets too long due to base64 encoding, we should // If the symlink target gets too long due to base64 encoding, we should
// return ENAMETOOLONG instead of having the kernel reject the data and // return ENAMETOOLONG instead of having the kernel reject the data and
// returning an I/O error to the user. // returning an I/O error to the user.
// https://github.com/rfjakob/gocryptfs/v2/issues/167 // https://github.com/rfjakob/gocryptfs/issues/167
func TestTooLongSymlink(t *testing.T) { func TestTooLongSymlink(t *testing.T) {
var err error var err error
var l int var l int

View File

@ -128,7 +128,7 @@ func TestExcludeTestFs(t *testing.T) {
} }
// Exclude everything using "/*", then selectively include only dir1 using "!/dir1" // Exclude everything using "/*", then selectively include only dir1 using "!/dir1"
// https://github.com/rfjakob/gocryptfs/v2/issues/588 // https://github.com/rfjakob/gocryptfs/issues/588
func TestExcludeAllOnlyDir1(t *testing.T) { func TestExcludeAllOnlyDir1(t *testing.T) {
// --exclude-wildcard patterns, gitignore syntax // --exclude-wildcard patterns, gitignore syntax
patterns := []string{ patterns := []string{

View File

@ -10,12 +10,12 @@ source ../fuse-unmount.bash
# Setup dirs # Setup dirs
../dl-linux-tarball.bash ../dl-linux-tarball.bash
cd /tmp cd /tmp
WD=$(mktemp -d /tmp/$MYNAME.XXX) WD=$(mktemp -d "/tmp/$MYNAME.XXX")
# Cleanup trap # Cleanup trap
trap "set +u; cd /; fuse-unmount -z $WD/c; fuse-unmount -z $WD/b; rm -rf $WD" EXIT trap "set +u; cd /; fuse-unmount -z $WD/c; fuse-unmount -z $WD/b; rm -rf $WD" EXIT
cd $WD cd "$WD"
mkdir a b c mkdir a b c
echo "Extracting tarball" echo "Extracting tarball"
tar -x -f /tmp/linux-3.0.tar.gz -C a tar -x -f /tmp/linux-3.0.tar.gz -C a
@ -30,4 +30,4 @@ gocryptfs -q -extpass="echo test" b c
cd c cd c
echo "Checking md5 sums" echo "Checking md5 sums"
set -o pipefail set -o pipefail
md5sum -c $MD5 | pv -l -s 36782 -N "files checked" | (grep -v ": OK" || true) md5sum -c "$MD5" | pv -l -s 36782 -N "files checked" | (grep -v ": OK" || true)

View File

@ -2,7 +2,7 @@
set -eu set -eu
function cleanup { cleanup() {
cd "$LOCAL_TMP" cd "$LOCAL_TMP"
fusermount -u gocryptfs.mnt fusermount -u gocryptfs.mnt
rm -Rf "$SSHFS_TMP" rm -Rf "$SSHFS_TMP"
@ -11,22 +11,22 @@ function cleanup {
rm -Rf "$LOCAL_TMP" rm -Rf "$LOCAL_TMP"
} }
function prepare_mounts { prepare_mounts() {
LOCAL_TMP=$(mktemp -d -t "$MYNAME.XXX") LOCAL_TMP=$(mktemp -d -t "$MYNAME.XXX")
cd $LOCAL_TMP cd "$LOCAL_TMP"
echo "working directory: $PWD" echo "working directory: $PWD"
mkdir sshfs.mnt gocryptfs.mnt mkdir sshfs.mnt gocryptfs.mnt
sshfs $HOST:/tmp sshfs.mnt sshfs "$HOST:/tmp" sshfs.mnt
echo "sshfs mounted: $HOST:/tmp -> sshfs.mnt" echo "sshfs mounted: $HOST:/tmp -> sshfs.mnt"
trap cleanup EXIT trap cleanup EXIT
SSHFS_TMP=$(mktemp -d "sshfs.mnt/$MYNAME.XXX") SSHFS_TMP=$(mktemp -d "sshfs.mnt/$MYNAME.XXX")
mkdir $SSHFS_TMP/gocryptfs.crypt mkdir "$SSHFS_TMP/gocryptfs.crypt"
gocryptfs -q -init -extpass "echo test" -scryptn=10 $SSHFS_TMP/gocryptfs.crypt gocryptfs -q -init -extpass "echo test" -scryptn=10 "$SSHFS_TMP/gocryptfs.crypt"
gocryptfs -q -extpass "echo test" $SSHFS_TMP/gocryptfs.crypt gocryptfs.mnt gocryptfs -q -extpass "echo test" "$SSHFS_TMP/gocryptfs.crypt" gocryptfs.mnt
echo "gocryptfs mounted: $SSHFS_TMP/gocryptfs.crypt -> gocryptfs.mnt" echo "gocryptfs mounted: $SSHFS_TMP/gocryptfs.crypt -> gocryptfs.mnt"
} }
function etime { etime() {
T=$(/usr/bin/time -f %e -o /dev/stdout "$@") T=$(/usr/bin/time -f %e -o /dev/stdout "$@")
LC_ALL=C printf %20.2f "$T" LC_ALL=C printf %20.2f "$T"
} }

View File

@ -28,17 +28,17 @@ source ../fuse-unmount.bash
# Setup dirs # Setup dirs
../dl-linux-tarball.bash ../dl-linux-tarball.bash
cd $TMPDIR cd "$TMPDIR"
EXTRACTLOOP_TMPDIR=$TMPDIR/extractloop_tmpdir EXTRACTLOOP_TMPDIR=$TMPDIR/extractloop_tmpdir
mkdir -p $EXTRACTLOOP_TMPDIR mkdir -p "$EXTRACTLOOP_TMPDIR"
CRYPT=$(mktemp -d $EXTRACTLOOP_TMPDIR/XXX) CRYPT=$(mktemp -d "$EXTRACTLOOP_TMPDIR/XXX")
CSV=$CRYPT.csv CSV=$CRYPT.csv
MNT=$CRYPT.mnt MNT=$CRYPT.mnt
mkdir $MNT mkdir "$MNT"
function check_md5sums { check_md5sums() {
if command -v md5sum > /dev/null ; then if command -v md5sum > /dev/null ; then
md5sum --status -c $1 md5sum --status -c "$1"
else else
# MacOS / darwin which do not have the md5sum utility # MacOS / darwin which do not have the md5sum utility
# installed by default # installed by default
@ -52,50 +52,50 @@ FS=""
if [ $# -eq 1 ] && [ "$1" == "-encfs" ]; then if [ $# -eq 1 ] && [ "$1" == "-encfs" ]; then
FS=encfs FS=encfs
echo "Testing EncFS" echo "Testing EncFS"
encfs --extpass="echo test" --standard $CRYPT $MNT > /dev/null encfs --extpass="echo test" --standard "$CRYPT" "$MNT" > /dev/null
elif [ $# -eq 1 ] && [ "$1" == "-loopback" ]; then elif [ $# -eq 1 ] && [ "$1" == "-loopback" ]; then
FS=loopback FS=loopback
echo "Testing go-fuse loopback" echo "Testing go-fuse loopback"
rm -f /tmp/loopback*.memprof rm -f /tmp/loopback*.memprof
loopback -memprofile=/tmp/loopback $MNT $CRYPT & loopback -memprofile=/tmp/loopback "$MNT" "$CRYPT" &
FSPID=$(jobs -p) FSPID=$(jobs -p)
disown disown
else else
FS=gocryptfs FS=gocryptfs
echo "Testing gocryptfs" echo "Testing gocryptfs"
gocryptfs -q -init -extpass="echo test" -scryptn=10 $CRYPT gocryptfs -q -init -extpass="echo test" -scryptn=10 "$CRYPT"
gocryptfs -q -extpass="echo test" -nosyslog -fg $CRYPT $MNT & gocryptfs -q -extpass="echo test" -nosyslog -fg "$CRYPT" "$MNT" &
FSPID=$(jobs -p) FSPID=$(jobs -p)
disown disown
#gocryptfs -q -extpass="echo test" -nosyslog -memprofile /tmp/extractloop-mem $CRYPT $MNT #gocryptfs -q -extpass="echo test" -nosyslog -memprofile /tmp/extractloop-mem "$CRYPT" "$MNT"
fi fi
echo "Test dir: $CRYPT" echo "Test dir: $CRYPT"
# Sleep to make sure the FS is already mounted on MNT # Sleep to make sure the FS is already mounted on MNT
sleep 1 sleep 1
cd $MNT cd "$MNT"
ln -v -sTf $CSV /tmp/extractloop.csv 2> /dev/null || true # fails on MacOS, ignore ln -v -sTf "$CSV" /tmp/extractloop.csv 2> /dev/null || true # fails on MacOS, ignore
# Cleanup trap # Cleanup trap
# Note: gocryptfs may have already umounted itself because bash relays SIGINT # Note: gocryptfs may have already umounted itself because bash relays SIGINT
# Just ignore unmount errors. # Just ignore unmount errors.
trap cleanup_exit EXIT trap cleanup_exit EXIT
function cleanup_exit { cleanup_exit() {
if [[ $FS == loopback ]]; then if [[ $FS == loopback ]]; then
# SIGUSR1 causes loopback to write the memory profile to disk # SIGUSR1 causes loopback to write the memory profile to disk
kill -USR1 $FSPID kill -USR1 $FSPID
fi fi
cd / cd /
rm -Rf $CRYPT rm -Rf "$CRYPT"
fuse-unmount -z $MNT || true fuse-unmount -z "$MNT" || true
rmdir $MNT rmdir "$MNT"
} }
function loop { loop() {
ID=$1 ID=$1
mkdir $ID mkdir "$ID"
cd $ID cd "$ID"
echo "[looper $ID] Starting" echo "[looper $ID] Starting"
@ -107,20 +107,20 @@ function loop {
tar xf /tmp/linux-3.0.tar.gz --exclude linux-3.0/arch/microblaze/boot/dts/system.dts tar xf /tmp/linux-3.0.tar.gz --exclude linux-3.0/arch/microblaze/boot/dts/system.dts
# ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ # ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
# Exclude the one symlink in the tarball - causes problems on MacOS: "Can't set permissions to 0755" # Exclude the one symlink in the tarball - causes problems on MacOS: "Can't set permissions to 0755"
check_md5sums $MD5 check_md5sums "$MD5"
rm -R linux-3.0 rm -R linux-3.0
t2=$SECONDS t2=$SECONDS
delta=$((t2-t1)) delta=$((t2-t1))
if [[ $FSPID -gt 0 && -d /proc ]]; then if [[ $FSPID -gt 0 && -d /proc ]]; then
RSS=$(grep VmRSS /proc/$FSPID/status | tr -s ' ' | cut -f2 -d ' ') RSS=$(grep VmRSS /proc/$FSPID/status | tr -s ' ' | cut -f2 -d ' ')
echo "$N,$SECONDS,$RSS,$delta" >> $CSV echo "$N,$SECONDS,$RSS,$delta" >> "$CSV"
fi fi
echo "[looper $ID] Iteration $N done, $delta seconds, RSS $RSS kiB" echo "[looper $ID] Iteration $N done, $delta seconds, RSS $RSS kiB"
let N=$N+1 N=$((N+1))
done done
} }
function memprof { memprof() {
while true; do while true; do
kill -USR1 $FSPID kill -USR1 $FSPID
sleep 60 sleep 60

View File

@ -19,7 +19,7 @@ export TMPDIR=${TMPDIR:-/var/tmp}
DEBUG=${DEBUG:-0} DEBUG=${DEBUG:-0}
cd "$(dirname "$0")" cd "$(dirname "$0")"
MYNAME=$(basename $0) MYNAME=$(basename "$0")
source ../fuse-unmount.bash source ../fuse-unmount.bash
# fsstress binary # fsstress binary
@ -32,23 +32,23 @@ then
fi fi
# Backing directory # Backing directory
DIR=$(mktemp -d $TMPDIR/$MYNAME.XXX) DIR=$(mktemp -d "$TMPDIR/$MYNAME.XXX")
# Mountpoint # Mountpoint
MNT="$DIR.mnt" MNT="$DIR.mnt"
mkdir $MNT mkdir "$MNT"
# Set the GOPATH variable to the default if it is empty # Set the GOPATH variable to the default if it is empty
GOPATH=$(go env GOPATH) GOPATH=$(go env GOPATH)
# Clean up old mounts # Clean up old mounts
for i in $(mount | cut -d" " -f3 | grep $TMPDIR/$MYNAME) ; do for i in $(mount | cut -d" " -f3 | grep "$TMPDIR/$MYNAME") ; do
fusermount -u $i fusermount -u "$i"
done done
# FS-specific compile and mount # FS-specific compile and mount
if [[ $MYNAME = fsstress-loopback.bash ]]; then if [[ $MYNAME = fsstress-loopback.bash ]]; then
echo -n "Recompile go-fuse loopback: " echo -n "Recompile go-fuse loopback: "
cd $GOPATH/src/github.com/hanwen/go-fuse/example/loopback cd "$GOPATH/src/github.com/hanwen/go-fuse/example/loopback"
git describe git describe
go build && go install go build && go install
OPTS="-q" OPTS="-q"
@ -59,20 +59,20 @@ if [[ $MYNAME = fsstress-loopback.bash ]]; then
disown disown
elif [[ $MYNAME = fsstress-gocryptfs.bash ]]; then elif [[ $MYNAME = fsstress-gocryptfs.bash ]]; then
echo "Recompile gocryptfs" echo "Recompile gocryptfs"
cd $GOPATH/src/github.com/rfjakob/gocryptfs cd "$GOPATH/src/github.com/rfjakob/gocryptfs"
./build.bash # also prints the version ./build.bash # also prints the version
$GOPATH/bin/gocryptfs -q -init -extpass "echo test" -scryptn=10 $DIR $GOPATH/bin/gocryptfs -q -init -extpass "echo test" -scryptn=10 "$DIR"
$GOPATH/bin/gocryptfs -q -extpass "echo test" -nosyslog -fusedebug=$DEBUG $DIR $MNT $GOPATH/bin/gocryptfs -q -extpass "echo test" -nosyslog -fusedebug="$DEBUG" "$DIR" "$MNT"
elif [[ $MYNAME = fsstress-encfs.bash ]]; then elif [[ $MYNAME = fsstress-encfs.bash ]]; then
encfs --extpass "echo test" --standard $DIR $MNT encfs --extpass "echo test" --standard "$DIR" "$MNT"
else else
echo Unknown mode: $MYNAME echo "Unknown mode: $MYNAME"
exit 1 exit 1
fi fi
sleep 0.5 sleep 0.5
echo -n "Waiting for mount: " echo -n "Waiting for mount: "
while ! grep "$(basename $MNT) fuse" /proc/self/mounts > /dev/null while ! grep "$(basename "$MNT") fuse" /proc/self/mounts > /dev/null
do do
sleep 1 sleep 1
echo -n x echo -n x
@ -87,26 +87,25 @@ N=1
while true while true
do do
echo "$N ................................. $(date)" echo "$N ................................. $(date)"
mkdir $MNT/fsstress.1 mkdir "$MNT/fsstress.1"
echo -n " fsstress.1 " echo -n " fsstress.1 "
$FSSTRESS -r -m 8 -n 1000 -d $MNT/fsstress.1 & $FSSTRESS -r -m 8 -n 1000 -d "$MNT/fsstress.1" &
wait wait
mkdir $MNT/fsstress.2 mkdir "$MNT/fsstress.2"
echo -n " fsstress.2 " echo -n " fsstress.2 "
$FSSTRESS -p 20 -r -m 8 -n 1000 -d $MNT/fsstress.2 & $FSSTRESS -p 20 -r -m 8 -n 1000 -d "$MNT/fsstress.2" &
wait wait
mkdir $MNT/fsstress.3 mkdir "$MNT/fsstress.3"
echo -n " fsstress.3 " echo -n " fsstress.3 "
$FSSTRESS -p 4 -z -f rmdir=10 -f link=10 -f creat=10 -f mkdir=10 \ $FSSTRESS -p 4 -z -f rmdir=10 -f link=10 -f creat=10 -f mkdir=10 \
-f rename=30 -f stat=30 -f unlink=30 -f truncate=20 -m 8 \ -f rename=30 -f stat=30 -f unlink=30 -f truncate=20 -m 8 \
-n 1000 -d $MNT/fsstress.3 & -n 1000 -d "$MNT/fsstress.3" &
wait wait
echo " rm" echo " rm"
rm -Rf $MNT/* rm -Rf "$MNT"/*
let N=$N+1 N=$((N+1))
done done

View File

@ -18,32 +18,32 @@ if [[ -z $TMPDIR ]]; then
fi fi
cd "$(dirname "$0")" cd "$(dirname "$0")"
MYNAME=$(basename $0) MYNAME=$(basename "$0")
source ../fuse-unmount.bash source ../fuse-unmount.bash
# Set the GOPATH variable to the default if it is empty # Set the GOPATH variable to the default if it is empty
GOPATH=$(go env GOPATH) GOPATH=$(go env GOPATH)
# Backing directory # Backing directory
DIR=$(mktemp -d $TMPDIR/$MYNAME.XXX) DIR=$(mktemp -d "$TMPDIR/$MYNAME.XXX")
$GOPATH/bin/gocryptfs -q -init -extpass "echo test" -scryptn=10 $DIR $GOPATH/bin/gocryptfs -q -init -extpass "echo test" -scryptn=10 "$DIR"
# Mountpoint # Mountpoint
MNT="$DIR.mnt" MNT="$DIR.mnt"
mkdir $MNT mkdir "$MNT"
$GOPATH/bin/gocryptfs -q -extpass "echo test" -nosyslog $DIR $MNT $GOPATH/bin/gocryptfs -q -extpass "echo test" -nosyslog "$DIR" "$MNT"
echo "Mounted gocryptfs $DIR at $MNT" echo "Mounted gocryptfs $DIR at $MNT"
# Cleanup trap # Cleanup trap
trap "cd / ; fuse-unmount -z $MNT ; rm -rf $DIR $MNT" EXIT trap "cd / ; fuse-unmount -z $MNT ; rm -rf $DIR $MNT" EXIT
cd $MNT cd "$MNT"
SECONDS=0 SECONDS=0
echo "creating files with dd" echo "creating files with dd"
mkdir -p origin mkdir -p origin
for i in $(seq 1 778) ; do for i in $(seq 1 778) ; do
dd if=/dev/zero of=origin/file_$i bs=8192 count=1 status=none dd if=/dev/zero of="origin/file_$i" bs=8192 count=1 status=none
done done
# Perform the shell expansion only once and store the list # Perform the shell expansion only once and store the list
ORIGIN_FILES=origin/* ORIGIN_FILES=origin/*
@ -51,7 +51,7 @@ ORIGIN_FILES=origin/*
echo -n "cp starting: " echo -n "cp starting: "
for i in $(seq 1 100) ; do for i in $(seq 1 100) ; do
echo -n "$i " echo -n "$i "
(mkdir sub_$i && cp $ORIGIN_FILES sub_$i ; echo -n "$i ") & (mkdir "sub_$i" && cp $ORIGIN_FILES "sub_$i" ; echo -n "$i ") &
done done
echo echo

View File

@ -13,7 +13,7 @@ renice 19 $$
cd "$(dirname "$0")" cd "$(dirname "$0")"
MD5="$PWD/linux-3.0.md5sums" MD5="$PWD/linux-3.0.md5sums"
MYNAME=$(basename $0) MYNAME=$(basename "$0")
source ../fuse-unmount.bash source ../fuse-unmount.bash
# Setup # Setup
@ -22,48 +22,48 @@ cd /tmp
PING=$(mktemp -d ping.XXX) PING=$(mktemp -d ping.XXX)
PONG=$(mktemp -d pong.XXX) PONG=$(mktemp -d pong.XXX)
mkdir $PING.mnt $PONG.mnt mkdir "$PING.mnt" "$PONG.mnt"
# Cleanup trap # Cleanup trap
# Note: gocryptfs may have already umounted itself because bash relays SIGINT # Note: gocryptfs may have already umounted itself because bash relays SIGINT
# Just ignore unmount errors. # Just ignore unmount errors.
trap "set +e ; cd /tmp; fuse-unmount -z $PING.mnt ; fuse-unmount -z $PONG.mnt ; rm -rf $PING $PONG $PING.mnt $PONG.mnt" EXIT trap "set +e ; cd /tmp; fuse-unmount -z $PING.mnt ; fuse-unmount -z $PONG.mnt ; rm -rf $PING $PONG $PING.mnt $PONG.mnt" EXIT
gocryptfs -q -init -extpass="echo test" -scryptn=10 $PING gocryptfs -q -init -extpass="echo test" -scryptn=10 "$PING"
gocryptfs -q -init -extpass="echo test" -scryptn=10 $PONG gocryptfs -q -init -extpass="echo test" -scryptn=10 "$PONG"
gocryptfs -q -extpass="echo test" -nosyslog $PING $PING.mnt gocryptfs -q -extpass="echo test" -nosyslog "$PING" "$PING.mnt"
gocryptfs -q -extpass="echo test" -nosyslog $PONG $PONG.mnt gocryptfs -q -extpass="echo test" -nosyslog "$PONG" "$PONG.mnt"
echo "Initial extract" echo "Initial extract"
tar xf /tmp/linux-3.0.tar.gz -C $PING.mnt tar xf /tmp/linux-3.0.tar.gz -C "$PING.mnt"
function move_and_md5 { move_and_md5() {
if [ $MYNAME = pingpong-rsync.bash ]; then if [ "$MYNAME" = "pingpong-rsync.bash" ]; then
echo -n "rsync " echo -n "rsync "
rsync -a --remove-source-files $1 $2 rsync -a --remove-source-files "$1" "$2"
find $1 -type d -delete find "$1" -type d -delete
else else
echo -n "mv " echo -n "mv "
mv $1 $2 mv "$1" "$2"
fi fi
if [ -e $1 ]; then if [ -e "$1" ]; then
echo "error: source directory $1 was not removed" echo "error: source directory $1 was not removed"
exit 1 exit 1
fi fi
cd $2 cd "$2"
echo -n "md5 " echo -n "md5 "
md5sum --status -c $MD5 md5sum --status -c "$MD5"
cd .. cd ..
} }
N=1 N=1
while true; do while true; do
echo -n "$N: " echo -n "$N: "
move_and_md5 $PING.mnt/linux-3.0 $PONG.mnt move_and_md5 "$PING.mnt/linux-3.0" "$PONG.mnt"
move_and_md5 $PONG.mnt/linux-3.0 $PING.mnt move_and_md5 "$PONG.mnt/linux-3.0" "$PING.mnt"
date +%H:%M:%S date +%H:%M:%S
let N=$N+1 N=$((N+1))
done done
wait wait