Sebastian Messmer
|
f22a9b2f0a
|
Merge branch 'next' into newblockstore
|
2017-07-07 15:48:24 +02:00 |
|
Sebastian Messmer
|
317088e449
|
Rename unique_ref::isValid to is_valid to comply with naming scheme of other functions
|
2017-04-10 14:16:30 -07:00 |
|
Sebastian Messmer
|
dbf54b9563
|
Add test cases for BlockStore2 and fix existing BlockStore2 implementations
|
2017-02-21 22:27:46 +00:00 |
|
Sebastian Messmer
|
70bfc47a2f
|
Start implementing new blockstore
|
2017-02-14 08:16:38 +00:00 |
|
Sebastian Messmer
|
cab1c71cac
|
Merge from develop
|
2017-02-04 19:18:00 +00:00 |
|
Sebastian Messmer
|
9a304fc52b
|
Upgrade spdlog to 0.11.0
|
2017-02-04 19:03:20 +00:00 |
|
Sebastian Messmer
|
42765c6be6
|
Merge from develop
|
2017-01-21 20:24:09 +00:00 |
|
Sebastian Messmer
|
b027f8c007
|
Merge from develop
|
2016-09-24 20:43:54 +02:00 |
|
Sebastian Messmer
|
46a2251e9d
|
Fix potential (although improbable) deadlock
|
2016-09-24 11:01:22 +02:00 |
|
Sebastian Messmer
|
eab7cb1df4
|
Implement BlockStore::overwrite(). This is the last step in ensuring that the write() call doen't have to load leaves if they're only overwritten anyhow.
|
2016-07-16 11:42:06 +02:00 |
|
Sebastian Messmer
|
c428d5642a
|
Minimize number of Block::write() calls when creating new nodes in a traversal. Also add test cases for it.
|
2016-07-15 12:48:20 +02:00 |
|
Sebastian Messmer
|
d626349802
|
Finish MockBlockStore to also collect other performance metrics, and implement the standard block store tests for it.
|
2016-07-15 10:23:09 +02:00 |
|
Sebastian Messmer
|
457ca52eb3
|
Add assertions to unique_ref
|
2016-07-14 16:29:02 +02:00 |
|
Sebastian Messmer
|
9e9369b9ed
|
Performance optimization: When removing or shrinking a tree, we don't load/read leaves anymore. Loading inner nodes is enough to get all block IDs and then we can delete the leaves from their IDs without loading them.
|
2016-07-13 11:27:31 +02:00 |
|
Sebastian Messmer
|
0d860fa4f0
|
Speed up block loading/storing by issuing only one read/write syscall to the base file system
|
2016-07-06 15:35:51 -07:00 |
|
Sebastian Messmer
|
ff0ba06846
|
Add mutex to fix race condition
|
2016-06-29 16:42:43 -07:00 |
|
Sebastian Messmer
|
bfb397f44e
|
Tell user to not interrupt migration process
|
2016-06-27 20:01:02 -07:00 |
|
Sebastian Messmer
|
514dbcb6c7
|
ClientId is not loaded/generated inside of KnownBlockVersions, but in CryDevice. This way, CryDevice also has access to the ClientID. This is later needed to recognize whether we're in a single-client or in a multi-client setting.
|
2016-06-26 22:15:52 -07:00 |
|
Sebastian Messmer
|
1c654305a8
|
In integrity violations, use individual messages saying which check exactly failed.
|
2016-06-26 22:15:52 -07:00 |
|
Sebastian Messmer
|
f066b45954
|
VersionCountingBlockStore::forEachBlock() has an integrity check checking that all expected blocks are still existing.
|
2016-06-26 22:15:52 -07:00 |
|
Sebastian Messmer
|
491b277cee
|
In case of an integrity violation, tell the user how they can reset integrity data.
|
2016-06-26 22:15:52 -07:00 |
|
Sebastian Messmer
|
839a511c4d
|
When an integrity violation is detected, any further access to the file system is blocked. This ensures that the user notices the integrity violation.
|
2016-06-26 22:15:52 -07:00 |
|
Sebastian Messmer
|
c84a230afd
|
VersionCountingBlockStore is initialized with a flag, saying whether missing blocks are integrity errors. This flag is meant to be activated in a single-client scenario to have this additional integrity check.
|
2016-06-26 22:15:52 -07:00 |
|
Sebastian Messmer
|
e02021ecdc
|
Throw an IntegrityViolationError exception instead of just logging integrity violations. This makes sure the user notices.
|
2016-06-26 22:15:52 -07:00 |
|
Sebastian Messmer
|
d306f4872e
|
Fix merge error
|
2016-06-26 22:15:52 -07:00 |
|
Sebastian Messmer
|
fd5dd2c6e1
|
New way of handling block deletions: Set last-update-client-id to zero instead of increasing version number. This has the same effect (i.e. invalidates all known block versions) and it additionally allows for checking in a local list whether a block is deleted or not.
|
2016-06-26 22:15:52 -07:00 |
|
Sebastian Messmer
|
ee1f0f4d3a
|
Warn in migration that it might take a while
|
2016-06-26 22:15:52 -07:00 |
|
Sebastian Messmer
|
86c2144a37
|
Implement migration from file systems without version numbers
|
2016-06-26 22:15:52 -07:00 |
|
Sebastian Messmer
|
f9a10eea4d
|
Fix VersionCountingBlock::resize()
|
2016-06-26 22:15:52 -07:00 |
|
Sebastian Messmer
|
473e9cc8bb
|
Fix compiler errors
|
2016-06-26 22:15:52 -07:00 |
|
Sebastian Messmer
|
1a72d3c226
|
Deleting a block doesn't set the version number to MAX_UINT64, but just increases it by one. This makes sure, that if (in case of a synchronization conflict) the block is reintroduced by another client, this is accepted.
|
2016-06-26 22:15:52 -07:00 |
|
Sebastian Messmer
|
5534b56ce3
|
When increasing the version number, don't only look at the version number in the block, but also at the last version number given to it by the current client. Ensure, the new version number is higher than both.
|
2016-06-26 22:15:52 -07:00 |
|
Sebastian Messmer
|
577c697856
|
When a client deletes a block, it will never again accept it as a valid block.
|
2016-06-26 22:15:52 -07:00 |
|
Sebastian Messmer
|
9c1a12c063
|
Fix merge
|
2016-06-26 22:15:52 -07:00 |
|
Sebastian Messmer
|
e7ac9bec57
|
Implement BlockStore::forEachBlock
|
2016-06-26 22:15:52 -07:00 |
|
Sebastian Messmer
|
4d1f7a46b9
|
* Prevent rollback to the "newest" version of a client when this version was superseded by a version from a different client.
* Use mutex/locks to secure access to KnownBlockVersions
|
2016-06-26 22:15:52 -07:00 |
|
Sebastian Messmer
|
263c540cd0
|
Make sure, ClientIdAndBlockKey.h includes all its dependencies
|
2016-06-26 22:15:52 -07:00 |
|
Sebastian Messmer
|
ea3de7360c
|
* Block versions don't have to be globally nondecreasing, but only per client id. This solves potential synchronization conflicts in a multi-client setting.
* Use cpputils::Serializer and cpputils::Deserializer instead of std::ifstream for storing/loading the block version list
|
2016-06-26 22:15:52 -07:00 |
|
Sebastian Messmer
|
de692c1ee4
|
Store number of entries at beginning of integrity file, so loading can be faster using unordered_map::reserve()
|
2016-06-26 22:15:52 -07:00 |
|
Sebastian Messmer
|
adb10343d2
|
Only hash key once if key not found, i.e. use [] instead of find() and then emplace()
|
2016-06-26 22:15:52 -07:00 |
|
Sebastian Messmer
|
9da30e3c17
|
Remove redundant nullbyte
|
2016-06-26 22:15:52 -07:00 |
|
Sebastian Messmer
|
57af168cfd
|
Store own client id into the block next to the version number. This will be used to fix synchronization conflicts where the version number on one clients progresses slower than on another client, but synchronizes later.
|
2016-06-26 22:15:52 -07:00 |
|
Sebastian Messmer
|
13411c4e59
|
Integrity data from KnownBlockVersions is persisted to a file
|
2016-06-26 22:15:52 -07:00 |
|
Sebastian Messmer
|
d7f547dd47
|
Implement a VersionCountingBlockStore that checks that version numbers are nondecreasing. Currently, the block store is not used yet and the state is not stored over different runs of CryFS. This comes with future commits.
|
2016-06-26 22:15:52 -07:00 |
|
Sebastian Messmer
|
f4a7878e37
|
Compatibility with GCC 6
|
2016-06-26 21:26:08 -07:00 |
|
Sebastian Messmer
|
acedbbd0c4
|
Fix EncryptedBlock::resize() and add test cases for it
|
2016-06-23 21:20:36 -07:00 |
|
Sebastian Messmer
|
fcbca9ddea
|
Improve performance of InMemoryBlockStore and FakeBlockStore by using std::unordered_map instead of std::map and the direct representation of a block Key as Key instead of a string representation.
|
2016-06-23 12:26:47 -07:00 |
|
Sebastian Messmer
|
285dd05072
|
When counting blocks, ignore cryfs.config. This already worked on Linux, but not on Mac. After this commit, it is also fixed for Mac.
|
2016-05-03 22:21:28 -07:00 |
|
Sebastian Messmer
|
5aaba511bd
|
Fix migration step for new ciphertext layout
|
2016-04-29 12:35:17 -07:00 |
|
Sebastian Messmer
|
9ed7bd0b41
|
Ciphertext blocks are split into subdirectories (before, all were on top level) to reduce number of files per directory. Some unix tools don't work well with directories with too many entries.
|
2016-04-29 12:21:02 -07:00 |
|
Sebastian Messmer
|
c403ec6b48
|
Fix a potential deadlock in the cache
|
2016-04-26 18:09:42 -07:00 |
|
Sebastian Messmer
|
037b59634e
|
Implemented BlockStore::blockSizeFromPhysicalBlockSize. This will be used to let the user configure physical block size instead of virtual block size.
|
2016-03-16 18:56:06 +00:00 |
|
Sebastian Messmer
|
80f63969c7
|
Add TODOs
|
2016-02-17 13:09:03 +01:00 |
|
Sebastian Messmer
|
25b93ebe41
|
When deleting a block fails, log the error but don't crash (i.e. continue to delete other blocks of the same blob on a best-effort basis)
|
2016-02-17 13:01:12 +01:00 |
|
Sebastian Messmer
|
bb54c2f879
|
When removing a block, check whether it was successful
|
2016-02-17 11:03:06 +01:00 |
|
Sebastian Messmer
|
35f4723b65
|
Fix compiler error
|
2016-02-17 00:24:40 +01:00 |
|
Sebastian Messmer
|
74288c81e4
|
Replace sysctl on Mac with a syscall that is supported in all kernels
|
2016-02-16 23:24:11 +01:00 |
|
Sebastian Messmer
|
a05f51ee08
|
Replace sysctl on Mac with a syscall that is supported in all kernels
|
2016-02-16 22:27:07 +01:00 |
|
Sebastian Messmer
|
68acc27e88
|
Report file system statistics (like free space) to operating system
|
2016-02-15 14:20:21 +01:00 |
|
Sebastian Messmer
|
456b240e21
|
Add TODOs
|
2016-02-15 03:45:09 +01:00 |
|
Sebastian Messmer
|
e44c44334f
|
Fix compile on Debian 8 (link it to a library it wants)
|
2016-02-14 16:54:29 +01:00 |
|
Sebastian Messmer
|
56ccf781eb
|
Fix compiler warning on GCC 4.8
|
2016-02-13 15:26:16 +01:00 |
|
Sebastian Messmer
|
636445cd82
|
Introduce version flags for file system entities to allow future CryFS versions to be backwards-compatible even if the format changes.
|
2016-02-13 15:06:28 +01:00 |
|
Sebastian Messmer
|
31672e086c
|
Fix includes after merging repositories
|
2016-02-11 15:19:58 +01:00 |
|
Sebastian Messmer
|
31d52d3c8d
|
Merged blobstore repository
|
2016-02-11 14:50:18 +01:00 |
|
Sebastian Messmer
|
f3d614c633
|
Merged blockstore
|
2016-02-11 14:44:28 +01:00 |
|
Sebastian Messmer
|
0c851fa452
|
Switch to biicode
|
2015-02-17 00:23:33 +01:00 |
|
Sebastian Messmer
|
76b3724da4
|
Since blocks now store their keys, we don't need to store it somewhere else.
|
2015-01-24 22:27:14 +01:00 |
|
Sebastian Messmer
|
196b543cbb
|
A block stores its key
|
2015-01-24 22:08:41 +01:00 |
|
Sebastian Messmer
|
f4398dfeec
|
Implemented copying of blocks
|
2015-01-24 01:24:37 +01:00 |
|
Sebastian Messmer
|
1bbe0532dc
|
Introduce DataNodeStore and refactor DataNode
|
2014-12-13 17:43:02 +01:00 |
|
Sebastian Messmer
|
c5d0e2c245
|
Add Key::ToBinary, Key::FromBinary and more test cases for Key
|
2014-12-13 11:59:48 +01:00 |
|
Sebastian Messmer
|
0d6dcd5b8b
|
Introduce Data::copy() and test case for it, refactor FakeBlobStore using Data::copy()
|
2014-12-11 01:41:08 +01:00 |
|
Sebastian Messmer
|
15b10feeaf
|
Create a FakeBlockStore and use it instead of InMemoryBlockStore in tests
|
2014-12-11 01:31:21 +01:00 |
|
Sebastian Messmer
|
1924c936a4
|
Remove Key::CreateDummyKey()
|
2014-12-09 20:57:10 +01:00 |
|
Sebastian Messmer
|
e7aea38f83
|
Create dedicated Key class for addressing blocks
|
2014-12-09 20:36:32 +01:00 |
|
Sebastian Messmer
|
868c06a5f1
|
Rename blobstore to blockstore
|
2014-12-09 17:19:59 +01:00 |
|