Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

refactor: Move epsilon_closure and nfa_to_dfa from Lexer class to Dfa class. #62

Open
wants to merge 442 commits into
base: main
Choose a base branch
from
Open
Changes from 24 commits
Commits
Show all changes
442 commits
Select commit Hold shift + click to select a range
59cc6cd
Add nullptr checks.
SharafMohamed Nov 7, 2024
8097a69
Merge branch 'meaningful-tags' into fixed-tagged-nfa
SharafMohamed Nov 7, 2024
9fc41c0
Change Tag class functionality to reflect how registers will be used.
SharafMohamed Nov 7, 2024
6e5c968
Add register class.
SharafMohamed Nov 7, 2024
e185fe2
Seperate classes from RegexDFA.hpp and RegexDFA.tpp into their own .h…
SharafMohamed Nov 7, 2024
54357af
Cleanup NFA and DFA classes.
SharafMohamed Nov 7, 2024
2379280
Change RegexDFA to Dfa.
SharafMohamed Nov 7, 2024
8fe9d1e
Change REGEX_DFA to DFA.
SharafMohamed Nov 7, 2024
a768ae3
Change DFAStateType to DfaStateType.
SharafMohamed Nov 7, 2024
9fcd7cc
RegexNFA -> Nfa.
SharafMohamed Nov 7, 2024
b112a25
UTF8 -> Utf8.
SharafMohamed Nov 7, 2024
653038f
Remove unneeded this->.
SharafMohamed Nov 7, 2024
815ac73
Use auto where possible.
SharafMohamed Nov 7, 2024
cc532c8
Remove nonterminal namespace inside of nonterminal.
SharafMohamed Nov 7, 2024
1a1655a
Merge branch 'remove-redundant-typenames' into remove-regex-prefix
SharafMohamed Nov 7, 2024
dae03e5
Merge branch 'remove-regex-prefix' into remove-uneeded-this
SharafMohamed Nov 7, 2024
76ef76f
Merge branch 'remove-uneeded-this' into use-auto
SharafMohamed Nov 7, 2024
70362d1
Use auto where possible for all = operators.
SharafMohamed Nov 7, 2024
039eec9
Change positive end transitions from a vector to an optional scalar.
SharafMohamed Nov 7, 2024
d060bc6
Temp fix for unit-test until future PR where Tag ptrs are stored in v…
SharafMohamed Nov 11, 2024
f041a37
Swap from set to vector to tag pointers to ensure determinism.
SharafMohamed Nov 11, 2024
f72e120
Better test coverage for tag class.
SharafMohamed Nov 11, 2024
d5ac1ad
Use constant iterators for elements that should not change.
SharafMohamed Nov 12, 2024
30f03ed
Use braced intiailization in test-tag.cpp.
SharafMohamed Nov 12, 2024
d386fc0
Use const& for insertion function that can't use move semantics.
SharafMohamed Nov 12, 2024
d77d17d
Refactor nfa_to_dfa().
SharafMohamed Nov 13, 2024
4024c3e
Have get_name() return string_view; Update headers.
SharafMohamed Nov 13, 2024
22c3b82
Remove const from member variable.
SharafMohamed Nov 13, 2024
ed55534
Remove const from member variable.
SharafMohamed Nov 13, 2024
534afce
Run linter.
SharafMohamed Nov 13, 2024
61fdb5d
Add move semantic test cases.
SharafMohamed Nov 13, 2024
78e5fe8
Add PositiveTaggedTransition docstring and make m_tag throw if ever n…
SharafMohamed Nov 13, 2024
630d882
Delete unused operators.
SharafMohamed Nov 13, 2024
543f8af
Move null check into intiailizer list for NegativeTaggedTransition co…
SharafMohamed Nov 13, 2024
ec342fc
Remove position vectors from Tag, as they arent used in the AST.
SharafMohamed Nov 13, 2024
af86281
RegexASTCapture enforces non-null arguments; Add docstring to RegexAS…
SharafMohamed Nov 13, 2024
738becd
Capitalize exceptions.
SharafMohamed Nov 13, 2024
789263e
Use () to fix linting issue.
SharafMohamed Nov 13, 2024
1f15ca7
Keep default copy assignment.
SharafMohamed Nov 14, 2024
7688c24
Move @throw to constructor docstrings.
SharafMohamed Nov 14, 2024
27618b2
Merge branch 'meaningful-tags' into fixed-tagged-nfa
SharafMohamed Nov 14, 2024
867d27c
Merge branch 'fixed-tagged-nfa' into register
SharafMohamed Nov 14, 2024
aff5bca
Merge branch 'register' into individual-dfa-files
SharafMohamed Nov 14, 2024
7195f57
Merge branch 'individual-dfa-files' into remove-redundant-typenames
SharafMohamed Nov 14, 2024
1b1460b
Merge branch 'remove-redundant-typenames' into remove-regex-prefix
SharafMohamed Nov 14, 2024
486190a
Do string_viee comparisomn in lexer test.
SharafMohamed Nov 14, 2024
ac75909
Use string_view compares in tag tests.
SharafMohamed Nov 14, 2024
090f18c
Update headers in TaggedTransition.hpp.
SharafMohamed Nov 15, 2024
c7cfc10
Seperate copy and move constructor unit-tests.
SharafMohamed Nov 15, 2024
91b8b51
Use NOTE for class requirements.
SharafMohamed Nov 15, 2024
fcb1a76
Use NOTE for class requirements.
SharafMohamed Nov 15, 2024
9b09e19
Use NOTE for class requirements.
SharafMohamed Nov 15, 2024
2f712e6
Merge branch 'meaningful-tags' into fixed-tagged-nfa
SharafMohamed Nov 18, 2024
75aecc4
Update install-catch2.sh to compile catch2 with c++17.
SharafMohamed Nov 18, 2024
9302b94
Merge branch 'main' into fixed-tagged-nfa
SharafMohamed Nov 18, 2024
97caabb
Merge branch 'catch2-install-fix' into fixed-tagged-nfa
SharafMohamed Nov 18, 2024
507a7d3
Loop over end_transitions correctly.
SharafMohamed Nov 18, 2024
34c227b
Add TagPositions class.
SharafMohamed Nov 18, 2024
27c8560
Remove new class, going to add it later.
SharafMohamed Nov 18, 2024
86caa9b
Add const back in.
SharafMohamed Nov 18, 2024
338638e
Add more const back in.
SharafMohamed Nov 18, 2024
a742601
Add more const back in.
SharafMohamed Nov 18, 2024
d358713
Linter.
SharafMohamed Nov 18, 2024
43870ea
Add more const back in.
SharafMohamed Nov 18, 2024
b827a6c
Merge branch 'fixed-tagged-nfa' into register
SharafMohamed Nov 18, 2024
f941607
Use `auto`.
SharafMohamed Nov 19, 2024
aad9eb3
Fix spacing.
SharafMohamed Nov 19, 2024
a801bf8
Add diagram for capture group NFA.
SharafMohamed Nov 19, 2024
08b7548
Add const for consitency with constructor.
SharafMohamed Nov 19, 2024
449133e
Update positive end transition to be optional instead of a vector.
SharafMohamed Nov 19, 2024
7b837bf
Rename new_state function correctly.
SharafMohamed Nov 19, 2024
f0eb56b
Update capture group AST state creation.
SharafMohamed Nov 19, 2024
a945915
Encapsulate new state for capture group.
SharafMohamed Nov 19, 2024
c757ded
Fix compiler error.
SharafMohamed Nov 19, 2024
2eb7477
Use singular for end transition getter function.
SharafMohamed Nov 20, 2024
08060ed
Void to auto -> void.
SharafMohamed Nov 20, 2024
0c2c1d1
Update new_capture_group_start_states to new_capture_group_states to …
SharafMohamed Nov 20, 2024
b0b951a
Linter.
SharafMohamed Nov 20, 2024
3c2a2ab
Update docstring for .
SharafMohamed Nov 20, 2024
98c5b95
Rename to new_start_and_end_states_with_positively_tagged_transitions.
SharafMohamed Nov 20, 2024
f59cf41
Rename to capture_X_state.
SharafMohamed Nov 20, 2024
85a2d69
Update docstring.
SharafMohamed Nov 20, 2024
4c602d4
Updated diagram to match vars used in code.
SharafMohamed Nov 20, 2024
2b01433
Rename vars to serialized_X.
SharafMohamed Nov 20, 2024
e37b29a
Run Linter.
SharafMohamed Nov 20, 2024
c5beca3
Fix typo.
SharafMohamed Nov 20, 2024
fe4a7b3
Update diagram for capture group NFA.
SharafMohamed Nov 20, 2024
8993088
Merge branch 'fixed-tagged-nfa' into register
SharafMohamed Nov 26, 2024
aaf720a
Merge branch 'main' into register
SharafMohamed Nov 26, 2024
0017512
Add register unit-tests, add PrefixTree with unit-tests.
SharafMohamed Nov 26, 2024
336f2ae
Finished with initial register implementation.
SharafMohamed Nov 26, 2024
3449df2
Linter.
SharafMohamed Nov 26, 2024
ef62df1
Linter.
SharafMohamed Nov 26, 2024
a085650
Docstring fixes.
SharafMohamed Nov 27, 2024
2be06c0
Add boundry test case.
SharafMohamed Nov 27, 2024
9ec01dd
Improve test cases for setting positions in prefix tree.
SharafMohamed Nov 27, 2024
019e675
Improve test cases for setting invalid positions in prefix tree.
SharafMohamed Nov 27, 2024
83a411a
Remove confusing description; Remove unused include.
SharafMohamed Nov 27, 2024
c88fbb5
Add edge case test to register unit-tests.
SharafMohamed Nov 27, 2024
7c91ddc
Update docstring for PrefixTreeNode.
SharafMohamed Nov 27, 2024
4c50769
Add comments to test-case; Add new test case for setting root value.
SharafMohamed Nov 27, 2024
98200b4
Update docstring to make it clear that any negative value of m_positi…
SharafMohamed Nov 27, 2024
afaf01a
Fix header gaurd.
SharafMohamed Nov 27, 2024
8dea476
Fix typo.
SharafMohamed Nov 27, 2024
dbb1e16
Remove newline in docstring.
SharafMohamed Nov 27, 2024
e054825
Improve throw consistency.
SharafMohamed Nov 27, 2024
792ce96
Update prefix tree insertion test cases.
SharafMohamed Nov 27, 2024
cab6e81
Fix test case.
SharafMohamed Nov 27, 2024
ffda5e6
Fix @throws doscstring for consistency; Improve insert() docstring.
SharafMohamed Nov 27, 2024
ff11672
Improve register handler test coverage.
SharafMohamed Nov 27, 2024
536b50b
Fix == ordering in test-cases; Fix vector initialization to remove re…
SharafMohamed Nov 27, 2024
77c20f7
Add const for consistency.
SharafMohamed Nov 27, 2024
f43759c
Add _HPP to header guards; Remove unused include.
SharafMohamed Nov 27, 2024
01e8881
Fix typo.
SharafMohamed Nov 27, 2024
fbb3d36
Remove blank line.
SharafMohamed Nov 27, 2024
e1f2b18
Rename to m_prefix_tree; Remove unused include.
SharafMohamed Nov 27, 2024
a51b49d
Add param descriptions to docstrings.
SharafMohamed Nov 27, 2024
002577e
Improve out of range check to be consistent.
SharafMohamed Nov 27, 2024
52a155c
Update set docstring.
SharafMohamed Nov 27, 2024
a6beafc
Punctuate docstrings.
SharafMohamed Nov 27, 2024
ec1f757
Update PregixTreeNode docstring.
SharafMohamed Nov 28, 2024
f35741f
Improve docstring for PrefixTree.
SharafMohamed Nov 28, 2024
e8e5e55
Change to use auto -> void; Punctuate out_of_range throws.
SharafMohamed Nov 28, 2024
f1ece30
Update Register docstring.
SharafMohamed Nov 28, 2024
08997ae
Update PrefixTree docstring.
SharafMohamed Nov 28, 2024
0910c62
Grammar fix.
SharafMohamed Nov 28, 2024
ede680e
Grammar fix.
SharafMohamed Nov 28, 2024
c7b047c
Use auto where possible.
SharafMohamed Nov 28, 2024
6fa8fcb
Use uniform initialization.
SharafMohamed Dec 2, 2024
18b9160
Add missing header.
SharafMohamed Dec 2, 2024
3f08fa3
Linter.
SharafMohamed Dec 2, 2024
e281f04
Fix spacing.
SharafMohamed Dec 2, 2024
a03734e
Make Node a member of PrefixTree.
SharafMohamed Dec 2, 2024
9123c7a
Rename index to prefix_tree_node_id.
SharafMohamed Dec 2, 2024
fe35fe0
Make it clear indicies in add_register are refering to prefix_tree no…
SharafMohamed Dec 3, 2024
de58e08
Linter.
SharafMohamed Dec 3, 2024
1426179
rename to reg_id.
SharafMohamed Dec 3, 2024
3301f14
Rename to reg_id.
SharafMohamed Dec 3, 2024
c9b1369
Use at().
SharafMohamed Dec 3, 2024
e2aee66
Remove Register class and use uint32_t instead; Rename vers to xxx_re…
SharafMohamed Dec 3, 2024
36c1810
Rename to reg_id.
SharafMohamed Dec 3, 2024
48df8b0
Remove unused header.
SharafMohamed Dec 3, 2024
a8605fc
Change pred index to be optional and nullopt for root.
SharafMohamed Dec 3, 2024
15cb1b6
Add and use node_id_t.
SharafMohamed Dec 3, 2024
6b787d0
Add position_t.
SharafMohamed Dec 3, 2024
cd8f4e3
Change to id_t.
SharafMohamed Dec 3, 2024
72da50c
Add is_root().
SharafMohamed Dec 4, 2024
3fc7ea7
Add missing header.
SharafMohamed Dec 4, 2024
6443d66
Update PrefixTree docstring.
SharafMohamed Dec 4, 2024
63aec4d
Removing node docstring as its redundant.
SharafMohamed Dec 4, 2024
295f3ee
Combine private section in PrefixTree.
SharafMohamed Dec 4, 2024
1186666
Add missing header; Remove copy paste error.
SharafMohamed Dec 4, 2024
06ee38e
Rename to node_id and parent_node_id.
SharafMohamed Dec 4, 2024
e103011
Update get_reversed_positions' docstring.
SharafMohamed Dec 4, 2024
31b0346
Update get_reversed positions' docstring to clarify exlcusivity of th…
SharafMohamed Dec 4, 2024
4005e41
Grammar fix.
SharafMohamed Dec 4, 2024
e38940c
Add maybe_unusued.
SharafMohamed Dec 4, 2024
d71368d
Update src/log_surgeon/finite_automata/RegisterHandler.hpp
SharafMohamed Dec 4, 2024
dd4b6e1
Update test case names to document code names better.
SharafMohamed Dec 4, 2024
7322852
Implicitily use auto in vectors.
SharafMohamed Dec 4, 2024
dba1a18
Explicitily use position_t for vectors.
SharafMohamed Dec 4, 2024
ee6efab
Update tests/test-register-handler.cpp
SharafMohamed Dec 4, 2024
9ba980c
Switch to size_t.
SharafMohamed Dec 4, 2024
27b324c
Clang-tidy: Remove magic numbers + Fix headers.
SharafMohamed Dec 4, 2024
f651a24
Reduce complexity for clang-tidy.
SharafMohamed Dec 4, 2024
fc6f426
Add negative pos test case in test-register-handler.cpp.
SharafMohamed Dec 4, 2024
c8fb570
Alternate b/w positive and negative positions in test-prefix-tree neg…
SharafMohamed Dec 4, 2024
1f66918
Add cRootId and size() to PrefixTree.
SharafMohamed Dec 4, 2024
a388c80
Update note.
SharafMohamed Dec 4, 2024
340eaf7
Update docstring.
SharafMohamed Dec 4, 2024
22cf931
Fix typo.
SharafMohamed Dec 4, 2024
e75c888
Merge branch 'register' into individual-dfa-files
SharafMohamed Dec 4, 2024
d5b20fe
Merge branch 'individual-dfa-files' into remove-redundant-typenames
SharafMohamed Dec 4, 2024
69a7ad1
Merge branch 'remove-redundant-typenames' into remove-regex-prefix
SharafMohamed Dec 4, 2024
0b7ef3c
Merge branch 'remove-regex-prefix' into remove-uneeded-this
SharafMohamed Dec 4, 2024
ada697f
Merge branch 'remove-uneeded-this' into use-auto
SharafMohamed Dec 4, 2024
4e08e2c
Merge branch 'use-auto' into refactor-nfa-to-dfa
SharafMohamed Dec 4, 2024
5916d33
Fix errors created by merge, need to still double check all the funct…
SharafMohamed Dec 5, 2024
c61f2d9
Update header for size_t.
SharafMohamed Dec 5, 2024
417bde8
Update src/log_surgeon/finite_automata/PrefixTree.hpp
SharafMohamed Dec 5, 2024
738876d
Update src/log_surgeon/finite_automata/PrefixTree.hpp
SharafMohamed Dec 5, 2024
93c03a0
Update src/log_surgeon/finite_automata/RegisterHandler.hpp
SharafMohamed Dec 5, 2024
6481e5f
Update tests/test-prefix-tree.cpp
SharafMohamed Dec 5, 2024
6a9a4a4
Clean up register initialization helper; Fix typo.
SharafMohamed Dec 5, 2024
052d86f
Update get_parent_id to clarify its unsafe and suppress warning.
SharafMohamed Dec 5, 2024
ed70bd5
Move constants in test-register-handler.hpp to minimize scope.
SharafMohamed Dec 5, 2024
fab801f
Merge branch 'register' into individual-dfa-files
SharafMohamed Dec 5, 2024
db84cb7
Merge branch 'individual-dfa-files' into remove-redundant-typenames
SharafMohamed Dec 5, 2024
7f6fcd9
Merge branch 'remove-redundant-typenames' into remove-regex-prefix
SharafMohamed Dec 5, 2024
a346104
Merge branch 'remove-regex-prefix' into remove-uneeded-this
SharafMohamed Dec 5, 2024
e90ae14
Merge branch 'remove-uneeded-this' into use-auto
SharafMohamed Dec 5, 2024
4d30509
Merge branch 'use-auto' into refactor-nfa-to-dfa
SharafMohamed Dec 5, 2024
1671e39
Move constants into scope for test-prefix-tree.cpp.
SharafMohamed Dec 5, 2024
748dfc5
Rename to handler_init and return handler.
SharafMohamed Dec 5, 2024
8abf35a
Add docstring for get_parent_id_unsafe().
SharafMohamed Dec 5, 2024
1e5fdcc
Linter.
SharafMohamed Dec 5, 2024
effac53
Merge branch 'individual-dfa-files' into remove-redundant-typenames
SharafMohamed Dec 5, 2024
43aa3be
Linter + Merge branch 'remove-redundant-typenames' into remove-regex-…
SharafMohamed Dec 5, 2024
c0920bf
Merge branch 'remove-regex-prefix' into remove-uneeded-this
SharafMohamed Dec 5, 2024
19fe130
Linter.
SharafMohamed Dec 5, 2024
61ceba6
Merge branch 'remove-uneeded-this' into use-auto
SharafMohamed Dec 5, 2024
0e1e2b2
Linter.
SharafMohamed Dec 5, 2024
9302ecb
Merge branch 'use-auto' into refactor-nfa-to-dfa
SharafMohamed Dec 5, 2024
71d926d
Merge branch 'register' into individual-dfa-files
SharafMohamed Dec 6, 2024
66ed13b
Merge branch 'main' into individual-dfa-files
SharafMohamed Dec 6, 2024
a12a360
Fix comment length.
SharafMohamed Dec 6, 2024
244d122
Initialize byte transitions.
SharafMohamed Dec 6, 2024
176391b
Use const* in place of unique_ptr reference; Update docstrings.
SharafMohamed Dec 7, 2024
012f61f
Update intersect test to compile.
SharafMohamed Dec 7, 2024
96a6363
Update next() docstring.
SharafMohamed Dec 7, 2024
7cd39f0
Merge branch 'individual-dfa-files' into remove-redundant-typenames
SharafMohamed Dec 7, 2024
a4a93b4
Rename to state_type.
SharafMohamed Dec 8, 2024
421c3de
Update headers.
SharafMohamed Dec 8, 2024
ecd4e4e
Merge branch 'individual-dfa-files' into remove-redundant-typenames
SharafMohamed Dec 8, 2024
1b945a1
Update Lexer headers.
SharafMohamed Dec 8, 2024
70cd43d
Merge branch 'individual-dfa-files' into remove-redundant-typenames
SharafMohamed Dec 8, 2024
78c4125
Add header for conditional_t.
SharafMohamed Dec 8, 2024
3ce0b30
Merge branch 'individual-dfa-files' into remove-redundant-typenames
SharafMohamed Dec 8, 2024
33623fa
Linter.
SharafMohamed Dec 8, 2024
bae0557
Merge branch 'individual-dfa-files' into remove-redundant-typenames
SharafMohamed Dec 8, 2024
5bbeafc
Linter.
SharafMohamed Dec 8, 2024
0decaf5
Change ! to false ==.
SharafMohamed Dec 8, 2024
0e2d593
Merge branch 'individual-dfa-files' into remove-redundant-typenames
SharafMohamed Dec 8, 2024
cf5980d
Merge branch 'remove-redundant-typenames' into remove-regex-prefix
SharafMohamed Dec 8, 2024
6e65a3e
LALR1Parser to Lalr1Parser.
SharafMohamed Dec 8, 2024
9c2ad81
Linter.
SharafMohamed Dec 8, 2024
c4fc96b
Rename templates to TypedDfaState and TypedNfaState.
SharafMohamed Dec 8, 2024
a6bbaef
Rename to Utf8*State and Byte*State.
SharafMohamed Dec 8, 2024
ff2dac3
Remove RegexDFAStateType.hpp.
SharafMohamed Dec 8, 2024
7a8982d
Linter.
SharafMohamed Dec 8, 2024
7b0a86c
Linter again.
SharafMohamed Dec 8, 2024
2632616
Add missing alogrithm header; Update test-NFA.
SharafMohamed Dec 8, 2024
992a2ec
Update intersect-test.cpp with new names.
SharafMohamed Dec 8, 2024
a5419f0
Linter on intersect-test.cpp.
SharafMohamed Dec 8, 2024
ae64f64
Merge branch 'remove-regex-prefix' into remove-uneeded-this
SharafMohamed Dec 9, 2024
9153a7c
Switch this->m_lexer to m_lexer by using Parser::m_lexer.
SharafMohamed Dec 9, 2024
fa0c098
Linter.
SharafMohamed Dec 9, 2024
845b088
Merge branch 'remove-uneeded-this' into use-auto
SharafMohamed Dec 9, 2024
2dd6f45
Lint.
SharafMohamed Dec 9, 2024
ee46719
More auto changes.
SharafMohamed Dec 9, 2024
4cebc06
Merge branch 'use-auto' into refactor-nfa-to-dfa
SharafMohamed Dec 11, 2024
827d39c
Merge branch 'main' into remove-redundant-typenames
SharafMohamed Dec 11, 2024
80979de
Remove old epsilon_closure after double checking it matches the moved…
SharafMohamed Dec 11, 2024
b38b6d7
Merge dfa_to_nfa() into Dfa::Dfa().
SharafMohamed Dec 11, 2024
1c2d7b3
Merge branch 'remove-redundant-typenames' into remove-regex-prefix
SharafMohamed Dec 11, 2024
11da5d5
Merge branch 'remove-regex-prefix' into remove-uneeded-this
SharafMohamed Dec 11, 2024
3f6966f
Merge branch 'remove-uneeded-this' into use-auto
SharafMohamed Dec 11, 2024
11eab2b
Merge branch 'use-auto' into refactor-nfa-to-dfa
SharafMohamed Dec 11, 2024
538ae8b
Fix intersect_test compiler errrors.
SharafMohamed Dec 11, 2024
74841b0
Add const.
SharafMohamed Dec 11, 2024
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion examples/intersect-test.cpp
Original file line number Diff line number Diff line change
@@ -42,7 +42,7 @@ auto get_intersect_for_query(
}
Nfa<NfaByteState> nfa(std::move(rules));
auto dfa2 = ByteLexer::nfa_to_dfa(nfa);
auto schema_types = dfa1->get_intersect(dfa2);
auto schema_types = dfa1->get_intersect(dfa2.get());
std::cout << search_string << ":";
for (auto const& schema_type : schema_types) {
std::cout << m_id_symbol[schema_type] << ",";
1 change: 1 addition & 0 deletions src/log_surgeon/Lexer.hpp
Original file line number Diff line number Diff line change
@@ -12,6 +12,7 @@

#include <log_surgeon/Constants.hpp>
#include <log_surgeon/finite_automata/Dfa.hpp>
#include <log_surgeon/finite_automata/DfaState.hpp>
#include <log_surgeon/finite_automata/Nfa.hpp>
#include <log_surgeon/finite_automata/RegexAST.hpp>
#include <log_surgeon/LexicalRule.hpp>
26 changes: 11 additions & 15 deletions src/log_surgeon/finite_automata/Dfa.hpp
Original file line number Diff line number Diff line change
@@ -13,27 +13,24 @@ template <typename DfaStateType>
class Dfa {
public:
/**
* Creates a new DFA state based on a set of NFA states and adds it to
* m_states
* @param nfa_state_set
* @return DfaStateType*
* Creates a new DFA state based on a set of NFA states and adds it to `m_states`.
* @param nfa_state_set The set of NFA states represented by this DFA state.
* @return A pointer to the new DFA state.
*/
template <typename NfaStateType>
auto new_state(std::set<NfaStateType*> const& nfa_state_set) -> DfaStateType*;

auto get_root() const -> DfaStateType const* { return m_states.at(0).get(); }

/**
* Compares this dfa with dfa_in to determine the set of schema types in
* this dfa that are reachable by any type in dfa_in. A type is considered
* reachable if there is at least one string for which: (1) this dfa returns
* a set of types containing the type, and (2) dfa_in returns any non-empty
* set of types.
* @param dfa_in
* @return The set of schema types reachable by dfa_in
* Compares this dfa with `dfa_in` to determine the set of schema types in this dfa that are
* reachable by any type in `dfa_in`. A type is considered reachable if there is at least one
* string for which: (1) this dfa returns a set of types containing the type, and (2) `dfa_in`
* returns any non-empty set of types.
* @param dfa_in The dfa with which to take the intersect.
* @return The set of schema types reachable by `dfa_in`.
*/
[[nodiscard]] auto get_intersect(std::unique_ptr<Dfa> const& dfa_in
) const -> std::set<uint32_t>;
[[nodiscard]] auto get_intersect(Dfa const* dfa_in) const -> std::set<uint32_t>;

private:
std::vector<std::unique_ptr<DfaStateType>> m_states;
@@ -53,8 +50,7 @@ auto Dfa<DfaStateType>::new_state(std::set<NfaStateType*> const& nfa_state_set)
}

template <typename DfaStateType>
auto Dfa<DfaStateType>::get_intersect(std::unique_ptr<Dfa> const& dfa_in
) const -> std::set<uint32_t> {
auto Dfa<DfaStateType>::get_intersect(Dfa const* dfa_in) const -> std::set<uint32_t> {
std::set<uint32_t> schema_types;
std::set<DfaStatePair<DfaStateType>> unvisited_pairs;
std::set<DfaStatePair<DfaStateType>> visited_pairs;
31 changes: 19 additions & 12 deletions src/log_surgeon/finite_automata/DfaState.hpp
Original file line number Diff line number Diff line change
@@ -5,8 +5,10 @@
#include <cstdint>
#include <memory>
#include <tuple>
#include <type_traits>
#include <vector>

#include <log_surgeon/Constants.hpp>
#include <log_surgeon/finite_automata/DfaStateType.hpp>
#include <log_surgeon/finite_automata/UnicodeIntervalTree.hpp>

@@ -17,11 +19,15 @@ class DfaState;
using DfaByteState = DfaState<DfaStateType::Byte>;
using DfaUtf8State = DfaState<DfaStateType::Utf8>;

template <DfaStateType stateType>
template <DfaStateType state_type>
class DfaState {
public:
using Tree = UnicodeIntervalTree<DfaState*>;

DfaState() {
std::fill(std::begin(m_bytes_transition), std::end(m_bytes_transition), nullptr);
}

auto add_matching_variable_id(uint32_t const variable_id) -> void {
m_matching_variable_ids.push_back(variable_id);
}
@@ -30,30 +36,31 @@ class DfaState {
return m_matching_variable_ids;
}

[[nodiscard]] auto is_accepting() const -> bool { return !m_matching_variable_ids.empty(); }
[[nodiscard]] auto is_accepting() const -> bool {
return false == m_matching_variable_ids.empty();
}

auto add_byte_transition(uint8_t const& byte, DfaState* dest_state) -> void {
m_bytes_transition[byte] = dest_state;
}

/**
* Returns the next state the DFA transitions to on input character (byte or utf8).
* @param character
* @return DfaState<stateType>*
* @param character The character (byte or utf8) to transition on.
* @return A pointer to the DFA state reached after transitioning on `character`.
*/
[[nodiscard]] auto next(uint32_t character) const -> DfaState*;

private:
std::vector<uint32_t> m_matching_variable_ids;
DfaState* m_bytes_transition[cSizeOfByte];
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

⚠️ Potential issue

Initialize m_bytes_transition to prevent undefined behaviour

The array m_bytes_transition contains pointers that are not automatically initialized, which may lead to undefined behaviour when accessing uninitialized pointers. It's crucial to initialize this array to ensure all elements start as nullptr.

Apply this diff to add a constructor that initializes m_bytes_transition:

+    DfaState() {
+        std::fill(std::begin(m_bytes_transition), std::end(m_bytes_transition), nullptr);
+    }
📝 Committable suggestion

‼️ IMPORTANT
Carefully review the code before committing. Ensure that it accurately replaces the highlighted code, contains no missing lines, and has no issues with indentation. Thoroughly test & benchmark the code to ensure it meets the requirements.

Suggested change
DfaState* m_bytes_transition[cSizeOfByte];
DfaState() {
std::fill(std::begin(m_bytes_transition), std::end(m_bytes_transition), nullptr);
}
DfaState* m_bytes_transition[cSizeOfByte];

// NOTE: We don't need m_tree_transitions for the `stateType == DfaStateType::Byte` case,
// so we use an empty class (`std::tuple<>`) in that case.
std::conditional_t<stateType == DfaStateType::Utf8, Tree, std::tuple<>> m_tree_transitions;
// NOTE: We don't need m_tree_transitions for the `state_type == DfaStateType::Byte` case, so we
// use an empty class (`std::tuple<>`) in that case.
std::conditional_t<state_type == DfaStateType::Utf8, Tree, std::tuple<>> m_tree_transitions;
};

template <DfaStateType stateType>
auto DfaState<stateType>::next(uint32_t character) const -> DfaState* {
if constexpr (DfaStateType::Byte == stateType) {
template <DfaStateType state_type>
auto DfaState<state_type>::next(uint32_t character) const -> DfaState* {
if constexpr (DfaStateType::Byte == state_type) {
return m_bytes_transition[character];
} else {
if (character < cSizeOfByte) {
@@ -62,7 +69,7 @@ auto DfaState<stateType>::next(uint32_t character) const -> DfaState* {
std::unique_ptr<std::vector<typename Tree::Data>> result
= m_tree_transitions.find(Interval(character, character));
assert(result->size() <= 1);
if (!result->empty()) {
if (false == result->empty()) {
return result->front().m_value;
}
return nullptr;
3 changes: 2 additions & 1 deletion src/log_surgeon/finite_automata/DfaStatePair.hpp
Original file line number Diff line number Diff line change
@@ -1,10 +1,11 @@
#ifndef LOG_SURGEON_FINITE_AUTOMATA_DFA_STATE_PAIR
#define LOG_SURGEON_FINITE_AUTOMATA_DFA_STATE_PAIR

#include <cstdint>
#include <set>
#include <vector>

#include <log_surgeon/finite_automata/DfaState.hpp>
#include <log_surgeon/Constants.hpp>

namespace log_surgeon::finite_automata {
/**
5 changes: 5 additions & 0 deletions src/log_surgeon/finite_automata/PrefixTree.hpp
Original file line number Diff line number Diff line change
@@ -65,6 +65,11 @@ class PrefixTree {

[[nodiscard]] auto is_root() const -> bool { return false == m_parent_id.has_value(); }

/**
* Gets the parent ID without checking if it's `std::nullopt`.
* NOTE: This method should only be used if the caller has checked the node is not the root.
* @return The ID of the parent node in the prefix tree.
*/
[[nodiscard]] auto get_parent_id_unsafe() const -> id_t {
// NOLINTNEXTLINE(bugprone-unchecked-optional-access)
return m_parent_id.value();
63 changes: 33 additions & 30 deletions tests/test-prefix-tree.cpp
Original file line number Diff line number Diff line change
@@ -10,22 +10,11 @@ using log_surgeon::finite_automata::PrefixTree;
using id_t = PrefixTree::id_t;
using position_t = PrefixTree::position_t;

constexpr auto cRootId{PrefixTree::cRootId};
constexpr id_t cInvalidNodeId{100};
constexpr position_t cInsertPos1{4};
constexpr position_t cInsertPos2{7};
constexpr position_t cInsertPos3{9};
constexpr position_t cMaxPos{std::numeric_limits<position_t>::max()};
constexpr position_t cNegativePos1{-1};
constexpr position_t cNegativePos2{-100};
constexpr position_t cSetPos1{10};
constexpr position_t cSetPos2{12};
constexpr position_t cSetPos3{15};
constexpr position_t cSetPos4{20};
constexpr position_t cTreeSize1{4};
constexpr position_t cTreeSize2{8};

TEST_CASE("`PrefixTree` operations", "[PrefixTree]") {
constexpr auto cRootId{PrefixTree::cRootId};
constexpr position_t cInitialPos1{4};
constexpr position_t cSetPos1{10};

SECTION("Newly constructed tree works correctly") {
PrefixTree const tree;

@@ -34,16 +23,24 @@ TEST_CASE("`PrefixTree` operations", "[PrefixTree]") {
}

SECTION("Inserting nodes into the prefix tree works correctly") {
constexpr position_t cInitialPos2{7};
constexpr position_t cInitialPos3{9};
constexpr position_t cMaxPos{std::numeric_limits<position_t>::max()};
constexpr position_t cNegativePos1{-1};
constexpr position_t cNegativePos2{-100};
constexpr position_t cTreeSize1{4};
constexpr position_t cTreeSize2{8};

PrefixTree tree;

// Test basic insertions
auto const node_id_1{tree.insert(cRootId, cInsertPos1)};
auto const node_id_2{tree.insert(node_id_1, cInsertPos2)};
auto const node_id_3{tree.insert(node_id_2, cInsertPos3)};
REQUIRE(std::vector<position_t>{cInsertPos1} == tree.get_reversed_positions(node_id_1));
REQUIRE(std::vector<position_t>{cInsertPos2, cInsertPos1}
auto const node_id_1{tree.insert(cRootId, cInitialPos1)};
auto const node_id_2{tree.insert(node_id_1, cInitialPos2)};
auto const node_id_3{tree.insert(node_id_2, cInitialPos3)};
REQUIRE(std::vector<position_t>{cInitialPos1} == tree.get_reversed_positions(node_id_1));
REQUIRE(std::vector<position_t>{cInitialPos2, cInitialPos1}
== tree.get_reversed_positions(node_id_2));
REQUIRE(std::vector<position_t>{cInsertPos3, cInsertPos2, cInsertPos1}
REQUIRE(std::vector<position_t>{cInitialPos3, cInitialPos2, cInitialPos1}
== tree.get_reversed_positions(node_id_3));
REQUIRE(cTreeSize1 == tree.size());

@@ -53,12 +50,12 @@ TEST_CASE("`PrefixTree` operations", "[PrefixTree]") {

// Test insertion with negative position values
auto const node_id_5{tree.insert(cRootId, cNegativePos1)};
auto const node_id_6{tree.insert(node_id_5, cInsertPos1)};
auto const node_id_6{tree.insert(node_id_5, cInitialPos1)};
auto const node_id_7{tree.insert(node_id_6, cNegativePos2)};
REQUIRE(std::vector<position_t>{cNegativePos1} == tree.get_reversed_positions(node_id_5));
REQUIRE(std::vector<position_t>{cInsertPos1, cNegativePos1}
REQUIRE(std::vector<position_t>{cInitialPos1, cNegativePos1}
== tree.get_reversed_positions(node_id_6));
REQUIRE(std::vector<position_t>{cNegativePos2, cInsertPos1, cNegativePos1}
REQUIRE(std::vector<position_t>{cNegativePos2, cInitialPos1, cNegativePos1}
== tree.get_reversed_positions(node_id_7));
REQUIRE(cTreeSize2 == tree.size());
}
@@ -67,7 +64,7 @@ TEST_CASE("`PrefixTree` operations", "[PrefixTree]") {
PrefixTree tree;
REQUIRE_THROWS_AS(tree.get_reversed_positions(tree.size()), std::out_of_range);

tree.insert(cRootId, cInsertPos1);
tree.insert(cRootId, cInitialPos1);
REQUIRE_THROWS_AS(tree.get_reversed_positions(tree.size()), std::out_of_range);

REQUIRE_THROWS_AS(
@@ -77,13 +74,17 @@ TEST_CASE("`PrefixTree` operations", "[PrefixTree]") {
}

SECTION("Set position for a valid index works correctly") {
constexpr position_t cSetPos2{12};
constexpr position_t cSetPos3{15};
constexpr position_t cSetPos4{20};

PrefixTree tree;
// Test that you can set the root node for sanity, although this value is not used
tree.set(cRootId, cSetPos1);

// Test updates to different nodes
auto const node_id_1{tree.insert(cRootId, cInsertPos1)};
auto const node_id_2{tree.insert(node_id_1, cInsertPos1)};
auto const node_id_1{tree.insert(cRootId, cInitialPos1)};
auto const node_id_2{tree.insert(node_id_1, cInitialPos1)};
tree.set(node_id_1, cSetPos1);
tree.set(node_id_2, cSetPos2);
REQUIRE(std::vector<position_t>{cSetPos1} == tree.get_reversed_positions(node_id_1));
@@ -105,13 +106,15 @@ TEST_CASE("`PrefixTree` operations", "[PrefixTree]") {
}

SECTION("Set position for an invalid index throws correctly") {
constexpr id_t cInvalidNodeId{100};

PrefixTree tree;

// Test setting position before any insertions
REQUIRE_THROWS_AS(tree.set(cInvalidNodeId, cSetPos4), std::out_of_range);
REQUIRE_THROWS_AS(tree.set(cInvalidNodeId, cSetPos1), std::out_of_range);

// Test setting position just beyond valid range
auto const node_id_1{tree.insert(cRootId, cInsertPos1)};
REQUIRE_THROWS_AS(tree.set(node_id_1 + 1, cSetPos4), std::out_of_range);
auto const node_id_1{tree.insert(cRootId, cInitialPos1)};
REQUIRE_THROWS_AS(tree.set(node_id_1 + 1, cSetPos1), std::out_of_range);
}
}
17 changes: 9 additions & 8 deletions tests/test-register-handler.cpp
Original file line number Diff line number Diff line change
@@ -12,17 +12,19 @@ using position_t = log_surgeon::finite_automata::PrefixTree::position_t;

namespace {
/**
* @param handler The register handler that will contain the new registers.
* @param num_registers The number of registers to initialize.
* @param num_registers The number of registers managed by the handler.
* @return The newly initialized register handler.
*/
auto registers_init(RegisterHandler& handler, size_t num_registers) -> void;
[[nodiscard]] auto handler_init(size_t num_registers) -> RegisterHandler;

auto registers_init(RegisterHandler& handler, size_t const num_registers) -> void {
auto handler_init(size_t const num_registers) -> RegisterHandler {
constexpr position_t cDefaultPos{0};

RegisterHandler handler;
for (size_t i{0}; i < num_registers; ++i) {
handler.add_register(i, cDefaultPos);
}
return handler;
}
} // namespace

@@ -32,13 +34,12 @@ TEST_CASE("`RegisterHandler` tests", "[RegisterHandler]") {
constexpr size_t cRegId1{0};
constexpr size_t cRegId2{1};

RegisterHandler handler;

SECTION("Initial state is empty") {
REQUIRE_THROWS_AS(handler.get_reversed_positions(cRegId1), std::out_of_range);
RegisterHandler empty_handler{handler_init(0)};
REQUIRE_THROWS_AS(empty_handler.get_reversed_positions(cRegId1), std::out_of_range);
}

registers_init(handler, cNumRegisters);
RegisterHandler handler{handler_init(cNumRegisters)};

SECTION("Set register position correctly") {
handler.set_register(cRegId1, cInitialPos1);