-
-
Notifications
You must be signed in to change notification settings - Fork 1.5k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Refactor and add support classes for KDBX 4 #1179
Conversation
Travis is failing due to too old libgcrypt. |
nice work ! |
Since you renamed the old KeePass2 format read/write to Kdbx3... would it make sense to make the new one Kdbx4.... Really we should be instantiating a single common read/write shell class and passing the specific reader/writer as a constructor parameter. This would ensure adding a new format would be painless and independent. We can implement that more advanced usage later, but it might be smart to rename KeePass2.cpp to Kdbx4Reader/Writer.cpp |
@droidmonkey the Kdbx4 classes are in the next PR this is only for the initial renaming |
sweet! Please disregard my post 😄 |
src/core/Database.cpp
Outdated
} | ||
} | ||
if (updateTransformSalt) { | ||
m_data.kdf->randomizeSalt(); |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
@angelsl We could name this function updateTranformSalt
, or rename the option randomizeSalt
, just to be consistent.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
randomizeTransformSalt()
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Done.
if (error) { | ||
qWarning("Gcrypt error (ctor): %s", gcry_strerror(error)); | ||
qWarning("Gcrypt error (ctor): %s", gcry_strsource(error)); | ||
} | ||
Q_ASSERT(error == 0); // TODO: error handling |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
@angelsl not sure how we should be handling those errors, but I guess we should at some point. Right now it'll just fail silently unless the application was built in debug mode.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Yeah, but I think this PR is starting to be quite large..
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Ideally the calling function would be notified that there was an error so it could display a notification to the user. Perhaps we could use a generic "cryptoError" signal that includes the error string and can be listened to and acted upon.
src/crypto/CryptoHash.cpp
Outdated
: d_ptr(new CryptoHashPrivate()) | ||
{ | ||
Q_D(CryptoHash); | ||
|
||
Q_ASSERT(Crypto::initalized()); | ||
|
||
int algoGcrypt; | ||
int algoGcrypt = -1; | ||
unsigned int flagsGcrypt = 0; |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
@angelsl What about using the GCRY_MD_FLAG_SECURE
flag by default? From the doc:
Allocate all buffers and the resulting digest in "secure memory". Use this is the hashed data is highly confidential.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Added that.
In preparation for multiple KDFs in KDBX 4
… class This class will in future select Kdbx4{R,W} as appropriate.
return m_cipher.init(CryptoHash::hash(key, CryptoHash::Sha256), | ||
KeePass2::INNER_STREAM_SALSA20_IV); | ||
switch (m_cipher.algorithm()) { | ||
case SymmetricCipher::Salsa20: |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
@angelsl case and switch on same indentation levels
KeePass2::ProtectedStreamAlgo KeePass2::idToProtectedStreamAlgo(quint32 id) | ||
{ | ||
switch (id) { | ||
case static_cast<quint32>(KeePass2::ArcFourVariant): |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
@angelsl case and switch on same indentation level
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Overall this is tremendous work. Needs to be simplified (kdf fields!) and heavily tested.
@@ -374,6 +323,7 @@ void Database::setEmitModified(bool value) | |||
void Database::copyAttributesFrom(const Database* other) | |||
{ |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
This function is only used in one instance (ToDbExporter) which is not used in the code at all. I recommend in PR2 that we remove this function and the ToDbExporter code.
@@ -479,3 +429,38 @@ QString Database::saveToFile(QString filePath) | |||
return saveFile.errorString(); | |||
} | |||
} | |||
|
|||
Kdf* Database::kdf() { |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Recommend rename to getKdf() and also return const Kdf*.
@@ -139,7 +139,7 @@ QByteArray int16ToBytes(qint16 num, QSysInfo::Endian byteOrder) | |||
qToLittleEndian<qint16>(num, reinterpret_cast<uchar*>(ba.data())); | |||
} | |||
else { | |||
qToBigEndian<qint64>(num, reinterpret_cast<uchar*>(ba.data())); | |||
qToBigEndian<qint16>(num, reinterpret_cast<uchar*>(ba.data())); |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Wow great catch!
QByteArray sha512Test = CryptoHash::hash("abcdbcdecdefdefgefghfghighijhijkijkljklmklmnlmnomnopnopq", | ||
CryptoHash::Sha512); | ||
|
||
if (sha512Test != QByteArray::fromHex("204a8fc6dda82f0a0ced7beb8e08a41657c16ef468b228a8279be331a703c33596fd15c13b1b07f9aa1d3bea57789ca031ad85c7a71dd70354ec631238ca3445")) { |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Please post the reference that confirms this is the correct sequence. Other crypto tests pulled known test strings from RFC standards docs. https://tools.ietf.org/html/rfc4231#section-4.1
@@ -285,3 +308,30 @@ bool Crypto::testSalsa20() | |||
|
|||
return true; | |||
} | |||
|
|||
bool Crypto::testChaCha20() { | |||
QByteArray chacha20Key = QByteArray::fromHex("0000000000000000000000000000000000000000000000000000000000000000"); |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Same, please add comment that points to the reference that confirms the test/result strings. https://tools.ietf.org/html/rfc7539#appendix-A.2
quint32 version() const; | ||
BaseKeePass2Reader* reader() const; | ||
private: | ||
QScopedPointer<BaseKeePass2Reader> m_reader; |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I am confused why this class both inherits from BaseKeePass2Reader and implements a worker of type BaseKeePass2Reader. Either use the worker or use yourself, pick one.
virtual QString errorString() override; | ||
|
||
private: | ||
QScopedPointer<BaseKeePass2Writer> m_writer; |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Same comment, don't inherit and implement the same base class.
} | ||
} | ||
|
||
void DatabaseSettingsWidget::displayKdf(const Kdf& kdf) |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
This function drives me bonkers. Needs to be implemented as a form and when fields get refactored... simplified immensely.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Yes, please, please make this a from. Don't convolute the CPP files with GUI boilerplate code.
|
||
#include "TestKeePass2XmlReader.h" | ||
|
||
QTEST_GUILESS_MAIN(TestKdbx3XmlReader) |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Where did the rest of these tests go?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Look at CMakeLists.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
OK... so there are two cpp files, one is basically blank. Why are the TestKdbx3XmlReader member functions defined in TestKeePass2XmlReader.cpp instead of it's own cpp file??
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I guess you didn't take a look at the 4 commits that were left out of this PR.
TestKeePass2XmlReader
has tests that instantiate KeePass2XmlReader
directly like this one. In order to avoid having to duplicate the test functions between Kdbx3XmlReader
and Kdbx4XmlReader
, I pulled out the XML read and writes into virtual functions that use the correct XmlReader
class. I decided to keep them in the same source file so that if these functions are ever changed, it is less likely that one half is forgotten, and so that it is easier to verify that they are identical aside from the different XmlReader
class used.
The better solution, arguably, would have been to use templates, but Qt moc
does not support templates.
Another solution would have been making the two XmlReader
classes a subclass of a base XmlReader
class like the other Reader
classes, but since the XmlReader
s are only ever used by the Reader
classes themselves, I didn't think that was appropriate: the XmlReader
classes are implementation details of the Reader
classes and it does not make sense for them to be sibling classes. The only way in which they are similar is that they both take XML data (that happens to be similarly structured) and spit out a database. Arguably the XmlReader
classes should be nested classes of the Reader
classes, but I digress.
Finally, in case you are wondering why I didn't stick both QTEST_GUILESS_MAIN
in the TestKeePass2XmlReader
file, QTEST_GUILESS_MAIN
is a macro that expands to a definition of int main
.
@@ -512,6 +552,10 @@ void TestKeePass2XmlReader::testRepairUuidHistoryItem() | |||
QVERIFY(!entry->uuid().isNull()); | |||
QVERIFY(!historyItem->uuid().isNull()); | |||
QCOMPARE(historyItem->uuid(), entry->uuid()); | |||
|
|||
if (db) { |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
You can just delete db
even if its nullptr (its a noop)
The point of the KDF fields thing is to avoid having these details hardcoded in the GUI. If you feel that is OK, then yes, we can drop the whole list-of-fields thing, but I will not personally author that. |
I will work on the rest of the comments in a few days. It would do you good to look at the rest of the PR, @droidmonkey, so you can understand why some things were done the way they were done, so that I don't have to explain myself repeatedly. We went through some of this in March already, in case you don't remember. Some of the inconsistencies are there because the original file had them. I'm trying to minimise code churn in this PR. Perhaps you'd like to fix the formatting and whatnot first in a separate PR, and then I, or someone else, can rebase this again. I'm not very motivated to get this merged, especially if I keep getting comments that result from not fully understanding the codebase. Just laying it out there. |
TBH, things that don't become clear from the code itself, but only from looking at previous discussion about that code need to be documented. Although I know that you will refuse to do that, I would generally prefer doc blocks above every function. I also know that this is a requirement the rest of our code doesn't meet (and therefore I won't refuse to merge because of this), but we want to change that. |
I spent a solid 2.5 days diving into your contribution. If a comment is invalid or I'm being a jerk address it in that specific comment. I'm not unreasonable, this contrib is basically open heart surgery on this app. If this does not work flawlessly, the app is worthless and potentially thousands of peoples databases can be corrupted or have flawed security. |
FYI - I deleted the atEnd() comments since there was no action required |
I suggest splitting this even more by leaving all KDBX format related changes out for now (including renames) so the things that everyone agree on can be merged in sooner making future work easier to review and comment on. I have added a custom KDF in C style from OpenSSH in my PR and it would be really nice to refactor it into a class API that could be defined by this PR. I also used QCryptographicHash for SHA-512 and again this PR could lay the work to bring SHA-512 into the main crypto classes and I could switch to use that instead. In my eyes this is great work towards the bigger picture and getting even half of the blood and sweat that has gone into it merged will help advance that. Keep up the good work! |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
This is already a lot closer to a mergeable state. Thanks for your continued work on this.
My main complaint is also the fields concept, which just needs to be dropped altogether. Also the Kdf/AesKdf class structure needs a bit of restructuring. Apart from these major complaints and a small handful of medium problems, most requested changes are rather minor style issues.
} | ||
|
||
if (m_data.kdf != nullptr) { | ||
delete m_data.kdf; |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Why not use a QSharedPointer instead?
QByteArray transformedMasterKey; | ||
Kdf* kdf; |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
See above.
@@ -88,25 +90,18 @@ class Database : public QObject | |||
|
|||
Uuid cipher() const; | |||
Database::CompressionAlgorithm compressionAlgo() const; | |||
QByteArray transformSeed() const; | |||
quint64 transformRounds() const; | |||
Kdf* kdf(); |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Method should be const (see @droidmonkey's comment for this method's implementation).
@@ -120,6 +115,7 @@ class Database : public QObject | |||
* Returns a unique id that is only valid as long as the Database exists. | |||
*/ | |||
Uuid uuid(); | |||
bool changeKdf(Kdf* kdf); |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
The difference between setKdf and changeKdf isn't easily understandable for the user of this API without looking at the implementation. Please document. I would also move this up a few lines to group it with the setter method.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
These functions probably shouldn't exist at all. The database class should create and maintain the kdf object with external entities merely telling the database which kdf to use.
@@ -42,6 +42,9 @@ namespace Endian { | |||
QByteArray int16ToBytes(qint16 num, QSysInfo::Endian byteOrder); | |||
QByteArray int32ToBytes(qint32 num, QSysInfo::Endian byteOrder); | |||
QByteArray int64ToBytes(qint64 num, QSysInfo::Endian byteOrder); | |||
QByteArray uint16ToBytes(quint16 num, QSysInfo::Endian byteOrder); | |||
QByteArray uint32ToBytes(quint32 num, QSysInfo::Endian byteOrder); | |||
QByteArray uint64ToBytes(quint64 num, QSysInfo::Endian byteOrder); |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I agree. The implementation of all these methods is basically identical.
ba.resize(2|4|8);
can be rewritten as ba.resize(sizeof(T));
} | ||
} | ||
|
||
void DatabaseSettingsWidget::displayKdf(const Kdf& kdf) |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Yes, please, please make this a from. Don't convolute the CPP files with GUI boilerplate code.
|
||
void DatabaseSettingsWidget::clearKdfWidgets() | ||
{ | ||
m_benchmarkField = nullptr; |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Drop this function together with the one above.
if (m_error) { | ||
return -1; | ||
} | ||
else if (m_eof) { |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Remember to not insert newlines before else.
|
||
m_buffer.clear(); | ||
} | ||
m_blockIndex++; |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
++m_blockIndex;
QByteArray seed() const; | ||
|
||
bool setRounds(quint64 rounds); | ||
bool setSeed(const QByteArray& seed); |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
These four methods should be move to the parent class as pure virtual methods. They are generic enough to be applicable to any kind of KDF.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
This would remove the need for fields.
I merged your work into a working branch in our repository. You are very welcome to continue working on this with us, but overall, we'll take it from here. Thank you very much for your contribution! |
I haven't had time to look at this until now. Sorry about that.
My comment about some things having been discussed back in March was about certain choices I made, which (I think) would have been clear had this PR not been chopped in half.
Indeed, I do not generally think there is a need to document every single internal function, but I would've agreed to add that.
I did.
I got pretty frustrated with these comments
because you appear to completely dismiss the fields abstraction without appearing to consider at all why I did it that way. My original intention was to support arbitrary KDFs, if more are ever added, without having to modify the GUI code. It is clear both you and @phoerious don't think that is necessary. So be it.
I made an effort to make sure (almost?) every single commit in this PR compiles and passes the full test suite. So there's that. In any case, I honestly don't feel like these changes are terribly major. A lot of it was copy and paste, or adding glue code to the crypto classes to support ciphers already supported by gcrypt, etc. A visual review plus manual testing plus the unit test suite should uncover any bugs — or perhaps I'm being too optimistic. Perhaps it is time you reformat the entire codebase once through, so that everything is consistent and you can spend less time commenting on stylistic issues. The reason I did weird things like newline-before- I guess it is for the better that you guys take over the PR. Be sure to pick up the rest of the KDBX 4 changes as well. Feel free to contact me if you need any clarifications or help. |
- Add support for KDBX 4.0, Argon2 and ChaCha20 [#148, #1179, #1230, #1494] - Add SSH Agent feature [#1098, #1450, #1463] - Add preview panel with details of the selected entry [#879, #1338] - Add more and configurable columns to entry table and allow copying of values by double click [#1305] - Add KeePassXC-Browser API as a replacement for KeePassHTTP [#608] - Deprecate KeePassHTTP [#1392] - Add support for Steam one-time passwords [#1206] - Add support for multiple Auto-Type sequences for a single entry [#1390] - Adjust YubiKey HMAC-SHA1 challenge-response key generation for KDBX 4.0 [#1060] - Replace qHttp with cURL for website icon downloads [#1460] - Remove lock file [#1231] - Add option to create backup file before saving [#1385] - Ask to save a generated password before closing the entry password generator [#1499] - Resolve placeholders recursively [#1078] - Add Auto-Type button to the toolbar [#1056] - Improve window focus handling for Auto-Type dialogs [#1204, #1490] - Auto-Type dialog and password generator can now be exited with ESC [#1252, #1412] - Add optional dark tray icon [#1154] - Add new "Unsafe saving" option to work around saving problems with file sync services [#1385] - Add IBus support to AppImage and additional image formats to Windows builds [#1534, #1537] - Add diceware password generator to CLI [#1406] - Add --key-file option to CLI [#816, #824] - Add DBus interface for opening and closing KeePassXC databases [#283] - Add KDBX compression options to database settings [#1419] - Discourage use of old fixed-length key files in favor of arbitrary files [#1326, #1327] - Correct reference resolution in entry fields [#1486] - Fix window state and recent databases not being remembered on exit [#1453] - Correct history item generation when configuring TOTP for an entry [#1446] - Correct multiple TOTP bugs [#1414] - Automatic saving after every change is now a default [#279] - Allow creation of new entries during search [#1398] - Correct menu issues on macOS [#1335] - Allow compilation on OpenBSD [#1328] - Improve entry attachments view [#1139, #1298] - Fix auto lock for Gnome and Xfce [#910, #1249] - Don't remember key files in file dialogs when the setting is disabled [#1188] - Improve database merging and conflict resolution [#807, #1165] - Fix macOS pasteboard issues [#1202] - Improve startup times on some platforms [#1205] - Hide the notes field by default [#1124] - Toggle main window by clicking tray icon with the middle mouse button [#992] - Fix custom icons not copied over when databases are merged [#1008] - Allow use of DEL key to delete entries [#914] - Correct intermittent crash due to stale history items [#1527] - Sanitize newline characters in title, username and URL fields [#1502] - Reopen previously opened databases in correct order [#774] - Use system's zxcvbn library if available [#701] - Implement various i18n improvements [#690, #875, #1436]
@phoerious were these changes ever picked up? If not, are they are still needed? |
This PR is merged..... |
I realize that. I just thought those changes in the linked repo might have been forgotten about. If they weren't then disregard my comment. |
Yes we completely overhauled this implementation. It's been in since 2.3.0 |
The remaining changes are here. I will open a PR once this gets merged.
Description
Refer to commit log
Motivation and context
#148
How has this been tested?
Screenshots (if appropriate):
Types of changes
Checklist:
-DWITH_ASAN=ON
. [REQUIRED]