Sebastian Messmer
ade23f9b86
Merge branch 'develop' into next
2017-08-19 11:53:22 +01:00
Mouse
7abed14d63
Make compatible with the current Crypto++ master ( #163 )
...
* Make compatible with the current Crypto++ master
* Add auto-config and build script. Not important, just a time-saver.
* Address compatibility with Crypto++ 6.0 release, while maintaining compatibility
with the older Crypto++ releases.
* Polish comments in cryptopp_byte.h. Forgot to include it to RandomGeneratorThread - fixed.
* Late at night - forgot to fix the .cpp files that used ::byte...
* Renamed auto-config-and-run script
* Added comments/description, and commented out "make check" that fails anyway
* Changed the include guard to match the rest of the .h files
* Delete build script
* Update ChangeLog.txt
* Update ChangeLog.txt
2017-08-16 03:00:46 +01:00
Sebastian Messmer
4a602ce7a5
Remove old blockstore implementations
2017-07-20 19:32:42 -07:00
Sebastian Messmer
36c29f19cf
Fix LowToHighLevelBlock
2017-07-18 16:59:48 -07:00
Sebastian Messmer
acf01ec4ff
Actually wire up CryDevice to use new block stores
2017-07-18 14:49:51 -07:00
Sebastian Messmer
872c94865c
Use using for shorter code
2017-07-13 02:13:13 -05:00
Sebastian Messmer
534968484c
Use using for shorter code
2017-07-13 01:33:35 -05:00
Sebastian Messmer
fbcab0abcc
Move stuff to .cpp files
2017-07-13 01:07:07 -05:00
Sebastian Messmer
428290fda5
Extend BlockStore2 and write a LowToHighLevelBlockStore as wrapper from BlockStore to BlockStore2
2017-07-07 16:18:09 -05:00
Sebastian Messmer
0942275d8c
TODOs
2017-07-07 18:09:29 +02:00
Sebastian Messmer
d6c27e92c5
Add VersionCountingBlockStore2 to build
2017-07-07 17:35:38 +02:00
Sebastian Messmer
6bdefc56a4
Simplify VersionCountingBlockStore::store()
2017-07-07 17:21:52 +02:00
Sebastian Messmer
f22a9b2f0a
Merge branch 'next' into newblockstore
2017-07-07 15:48:24 +02:00
Sebastian Messmer
317088e449
Rename unique_ref::isValid to is_valid to comply with naming scheme of other functions
2017-04-10 14:16:30 -07:00
Sebastian Messmer
dbf54b9563
Add test cases for BlockStore2 and fix existing BlockStore2 implementations
2017-02-21 22:27:46 +00:00
Sebastian Messmer
70bfc47a2f
Start implementing new blockstore
2017-02-14 08:16:38 +00:00
Sebastian Messmer
cab1c71cac
Merge from develop
2017-02-04 19:18:00 +00:00
Sebastian Messmer
9a304fc52b
Upgrade spdlog to 0.11.0
2017-02-04 19:03:20 +00:00
Sebastian Messmer
42765c6be6
Merge from develop
2017-01-21 20:24:09 +00:00
Sebastian Messmer
b027f8c007
Merge from develop
2016-09-24 20:43:54 +02:00
Sebastian Messmer
46a2251e9d
Fix potential (although improbable) deadlock
2016-09-24 11:01:22 +02:00
Sebastian Messmer
eab7cb1df4
Implement BlockStore::overwrite(). This is the last step in ensuring that the write() call doen't have to load leaves if they're only overwritten anyhow.
2016-07-16 11:42:06 +02:00
Sebastian Messmer
c428d5642a
Minimize number of Block::write() calls when creating new nodes in a traversal. Also add test cases for it.
2016-07-15 12:48:20 +02:00
Sebastian Messmer
d626349802
Finish MockBlockStore to also collect other performance metrics, and implement the standard block store tests for it.
2016-07-15 10:23:09 +02:00
Sebastian Messmer
457ca52eb3
Add assertions to unique_ref
2016-07-14 16:29:02 +02:00
Sebastian Messmer
9e9369b9ed
Performance optimization: When removing or shrinking a tree, we don't load/read leaves anymore. Loading inner nodes is enough to get all block IDs and then we can delete the leaves from their IDs without loading them.
2016-07-13 11:27:31 +02:00
Sebastian Messmer
0d860fa4f0
Speed up block loading/storing by issuing only one read/write syscall to the base file system
2016-07-06 15:35:51 -07:00
Sebastian Messmer
ff0ba06846
Add mutex to fix race condition
2016-06-29 16:42:43 -07:00
Sebastian Messmer
bfb397f44e
Tell user to not interrupt migration process
2016-06-27 20:01:02 -07:00
Sebastian Messmer
514dbcb6c7
ClientId is not loaded/generated inside of KnownBlockVersions, but in CryDevice. This way, CryDevice also has access to the ClientID. This is later needed to recognize whether we're in a single-client or in a multi-client setting.
2016-06-26 22:15:52 -07:00
Sebastian Messmer
1c654305a8
In integrity violations, use individual messages saying which check exactly failed.
2016-06-26 22:15:52 -07:00
Sebastian Messmer
f066b45954
VersionCountingBlockStore::forEachBlock() has an integrity check checking that all expected blocks are still existing.
2016-06-26 22:15:52 -07:00
Sebastian Messmer
491b277cee
In case of an integrity violation, tell the user how they can reset integrity data.
2016-06-26 22:15:52 -07:00
Sebastian Messmer
839a511c4d
When an integrity violation is detected, any further access to the file system is blocked. This ensures that the user notices the integrity violation.
2016-06-26 22:15:52 -07:00
Sebastian Messmer
c84a230afd
VersionCountingBlockStore is initialized with a flag, saying whether missing blocks are integrity errors. This flag is meant to be activated in a single-client scenario to have this additional integrity check.
2016-06-26 22:15:52 -07:00
Sebastian Messmer
e02021ecdc
Throw an IntegrityViolationError exception instead of just logging integrity violations. This makes sure the user notices.
2016-06-26 22:15:52 -07:00
Sebastian Messmer
d306f4872e
Fix merge error
2016-06-26 22:15:52 -07:00
Sebastian Messmer
fd5dd2c6e1
New way of handling block deletions: Set last-update-client-id to zero instead of increasing version number. This has the same effect (i.e. invalidates all known block versions) and it additionally allows for checking in a local list whether a block is deleted or not.
2016-06-26 22:15:52 -07:00
Sebastian Messmer
ee1f0f4d3a
Warn in migration that it might take a while
2016-06-26 22:15:52 -07:00
Sebastian Messmer
86c2144a37
Implement migration from file systems without version numbers
2016-06-26 22:15:52 -07:00
Sebastian Messmer
f9a10eea4d
Fix VersionCountingBlock::resize()
2016-06-26 22:15:52 -07:00
Sebastian Messmer
473e9cc8bb
Fix compiler errors
2016-06-26 22:15:52 -07:00
Sebastian Messmer
1a72d3c226
Deleting a block doesn't set the version number to MAX_UINT64, but just increases it by one. This makes sure, that if (in case of a synchronization conflict) the block is reintroduced by another client, this is accepted.
2016-06-26 22:15:52 -07:00
Sebastian Messmer
5534b56ce3
When increasing the version number, don't only look at the version number in the block, but also at the last version number given to it by the current client. Ensure, the new version number is higher than both.
2016-06-26 22:15:52 -07:00
Sebastian Messmer
577c697856
When a client deletes a block, it will never again accept it as a valid block.
2016-06-26 22:15:52 -07:00
Sebastian Messmer
9c1a12c063
Fix merge
2016-06-26 22:15:52 -07:00
Sebastian Messmer
e7ac9bec57
Implement BlockStore::forEachBlock
2016-06-26 22:15:52 -07:00
Sebastian Messmer
4d1f7a46b9
* Prevent rollback to the "newest" version of a client when this version was superseded by a version from a different client.
...
* Use mutex/locks to secure access to KnownBlockVersions
2016-06-26 22:15:52 -07:00
Sebastian Messmer
263c540cd0
Make sure, ClientIdAndBlockKey.h includes all its dependencies
2016-06-26 22:15:52 -07:00
Sebastian Messmer
ea3de7360c
* Block versions don't have to be globally nondecreasing, but only per client id. This solves potential synchronization conflicts in a multi-client setting.
...
* Use cpputils::Serializer and cpputils::Deserializer instead of std::ifstream for storing/loading the block version list
2016-06-26 22:15:52 -07:00
Sebastian Messmer
de692c1ee4
Store number of entries at beginning of integrity file, so loading can be faster using unordered_map::reserve()
2016-06-26 22:15:52 -07:00
Sebastian Messmer
adb10343d2
Only hash key once if key not found, i.e. use [] instead of find() and then emplace()
2016-06-26 22:15:52 -07:00
Sebastian Messmer
9da30e3c17
Remove redundant nullbyte
2016-06-26 22:15:52 -07:00
Sebastian Messmer
57af168cfd
Store own client id into the block next to the version number. This will be used to fix synchronization conflicts where the version number on one clients progresses slower than on another client, but synchronizes later.
2016-06-26 22:15:52 -07:00
Sebastian Messmer
13411c4e59
Integrity data from KnownBlockVersions is persisted to a file
2016-06-26 22:15:52 -07:00
Sebastian Messmer
d7f547dd47
Implement a VersionCountingBlockStore that checks that version numbers are nondecreasing. Currently, the block store is not used yet and the state is not stored over different runs of CryFS. This comes with future commits.
2016-06-26 22:15:52 -07:00
Sebastian Messmer
f4a7878e37
Compatibility with GCC 6
2016-06-26 21:26:08 -07:00
Sebastian Messmer
acedbbd0c4
Fix EncryptedBlock::resize() and add test cases for it
2016-06-23 21:20:36 -07:00
Sebastian Messmer
fcbca9ddea
Improve performance of InMemoryBlockStore and FakeBlockStore by using std::unordered_map instead of std::map and the direct representation of a block Key as Key instead of a string representation.
2016-06-23 12:26:47 -07:00
Sebastian Messmer
285dd05072
When counting blocks, ignore cryfs.config. This already worked on Linux, but not on Mac. After this commit, it is also fixed for Mac.
2016-05-03 22:21:28 -07:00
Sebastian Messmer
5aaba511bd
Fix migration step for new ciphertext layout
2016-04-29 12:35:17 -07:00
Sebastian Messmer
9ed7bd0b41
Ciphertext blocks are split into subdirectories (before, all were on top level) to reduce number of files per directory. Some unix tools don't work well with directories with too many entries.
2016-04-29 12:21:02 -07:00
Sebastian Messmer
c403ec6b48
Fix a potential deadlock in the cache
2016-04-26 18:09:42 -07:00
Sebastian Messmer
037b59634e
Implemented BlockStore::blockSizeFromPhysicalBlockSize. This will be used to let the user configure physical block size instead of virtual block size.
2016-03-16 18:56:06 +00:00
Sebastian Messmer
80f63969c7
Add TODOs
2016-02-17 13:09:03 +01:00
Sebastian Messmer
25b93ebe41
When deleting a block fails, log the error but don't crash (i.e. continue to delete other blocks of the same blob on a best-effort basis)
2016-02-17 13:01:12 +01:00
Sebastian Messmer
bb54c2f879
When removing a block, check whether it was successful
2016-02-17 11:03:06 +01:00
Sebastian Messmer
35f4723b65
Fix compiler error
2016-02-17 00:24:40 +01:00
Sebastian Messmer
74288c81e4
Replace sysctl on Mac with a syscall that is supported in all kernels
2016-02-16 23:24:11 +01:00
Sebastian Messmer
a05f51ee08
Replace sysctl on Mac with a syscall that is supported in all kernels
2016-02-16 22:27:07 +01:00
Sebastian Messmer
68acc27e88
Report file system statistics (like free space) to operating system
2016-02-15 14:20:21 +01:00
Sebastian Messmer
456b240e21
Add TODOs
2016-02-15 03:45:09 +01:00
Sebastian Messmer
e44c44334f
Fix compile on Debian 8 (link it to a library it wants)
2016-02-14 16:54:29 +01:00
Sebastian Messmer
56ccf781eb
Fix compiler warning on GCC 4.8
2016-02-13 15:26:16 +01:00
Sebastian Messmer
636445cd82
Introduce version flags for file system entities to allow future CryFS versions to be backwards-compatible even if the format changes.
2016-02-13 15:06:28 +01:00
Sebastian Messmer
31672e086c
Fix includes after merging repositories
2016-02-11 15:19:58 +01:00
Sebastian Messmer
31d52d3c8d
Merged blobstore repository
2016-02-11 14:50:18 +01:00
Sebastian Messmer
f3d614c633
Merged blockstore
2016-02-11 14:44:28 +01:00
Sebastian Messmer
0c851fa452
Switch to biicode
2015-02-17 00:23:33 +01:00
Sebastian Messmer
76b3724da4
Since blocks now store their keys, we don't need to store it somewhere else.
2015-01-24 22:27:14 +01:00
Sebastian Messmer
196b543cbb
A block stores its key
2015-01-24 22:08:41 +01:00
Sebastian Messmer
f4398dfeec
Implemented copying of blocks
2015-01-24 01:24:37 +01:00
Sebastian Messmer
1bbe0532dc
Introduce DataNodeStore and refactor DataNode
2014-12-13 17:43:02 +01:00
Sebastian Messmer
c5d0e2c245
Add Key::ToBinary, Key::FromBinary and more test cases for Key
2014-12-13 11:59:48 +01:00
Sebastian Messmer
0d6dcd5b8b
Introduce Data::copy() and test case for it, refactor FakeBlobStore using Data::copy()
2014-12-11 01:41:08 +01:00
Sebastian Messmer
15b10feeaf
Create a FakeBlockStore and use it instead of InMemoryBlockStore in tests
2014-12-11 01:31:21 +01:00
Sebastian Messmer
1924c936a4
Remove Key::CreateDummyKey()
2014-12-09 20:57:10 +01:00
Sebastian Messmer
e7aea38f83
Create dedicated Key class for addressing blocks
2014-12-09 20:36:32 +01:00
Sebastian Messmer
868c06a5f1
Rename blobstore to blockstore
2014-12-09 17:19:59 +01:00