-
Notifications
You must be signed in to change notification settings - Fork 5.5k
Macro wrapper for memcpy #13770
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Merged
Merged
Macro wrapper for memcpy #13770
Changes from 23 commits
Commits
Show all changes
122 commits
Select commit
Hold shift + click to select a range
df6d49f
Macro 'SAFE_MEMCPY added according to comments on issue#9328'
6a9d849
WIP: SAFE_MEMCPY modified with memmove, and some minor changes made.
5a29b9a
macros.h format fixed
814894b
testing CI pipeline
45565c2
static_assert deleted from macro SAFE_MEMCPY to avoid troubles with r…
2ad12d4
list of dependencies changed
22d2bfa
label corrected in .../mongo_proxy/BUILD
9d72d11
casts added and deleted var val included
1b43b53
format corrected
0aab7bc
cast added to avoid using void* in MemBlockBuilder
d34e261
Initializing Span before calling appendData
6e3a257
Span constructor corrected
d4b3c50
Revert "Span constructor corrected"
34d9ecb
Span constructor corrected
50eee3c
style corrected.
1bc7916
SAFE_MACRO modified to assert input arrays' sizes
d68aa9e
Get size of the array
4d331cd
use of envoy ASSERT macro in SAFE_MEMCPY
5372f86
Use static_assert again in SAFE_MEMCPY. Changes in proxy_protocol.cc
5517c53
proxy protocol header parse changed
455ceb7
getSockAddr public method added to PipeInstance
d188a94
comparison corrected in for loop
b20de9a
reinterpret_cast deleted from dns_parser. Some changes added to buffe…
cfb8145
buffer BUILD file changed
87c59b3
Cast added to copy data
22b2ee4
proxy protocol implementation back in place. Method parseV2Header mod…
e1dd071
changes to mem_block_builder applied, data_ type is a raw pointer poi…
57916ed
format fixed
c71445f
check_format changed as suggested in issue #9328
f51819e
NOTLINT added to the line in which memcpy is used
b0960d3
memcpy changed for strcpy
74e19f1
memcpy reviewed in the dir source/
dc0d096
Other applications of memcpy reviewed
a980bbd
assert error corrected for extensions/filters/network/kafka/serializa…
d1a3898
SAFE_MEMCPY reviewed due to difference in array sizes
f594444
minor corrections
f75af26
Merge branch 'master' into issue-9328
rialg ead0075
Format fixed
327d8d7
Merge branch 'issue-9328' of https://github.com/grial1/envoy into iss…
f3b29f6
wrong file submitted. dns_parser.cc fix added
4e502c8
changes suggested by jmarantz were added and some tests approved.
42105e0
format fixed
d714e2e
address_impl.cc compilation error fixed
b7f136c
comment fixed in address_impl
596c784
initial set of changes. Added MemBlockBuilder in proxy protocol imple…
6f0305f
proposed changes applied
671a5f1
format fixed
c51bbf2
bug fixed
882752a
format fixed
fecccfe
SAFE_MEMCPY deleted from macros.h
3692166
SAFE_MEMCPY macro changed for safe_memcpy template function
e3f469b
safe_memcpy added to common/network/cidr_range.cc
40dc126
proxy_protocol_header.cc: function generateV2Header uses memcpy in a …
616f36d
format fixed
2a8c457
explicit types deleted from safe_memcpy, check_format.py updated, and…
c4211ca
format fixed
9488f4e
C library added for memcpy
3b44b88
dereference applied and safe_memcpy_lib registered for hot_restarting…
cdcecd3
coding style error fixed: safe_memcpy signature changed to safeMemcpy
6fc7264
memcpy error message changed in check_format.py
b7dcf04
two simple tests added for safeMemcpy
93622eb
format corrected and application of safeMemcpy in common/grpc/common.cc
ade5461
address taken from frame->ping.opaque_data in codec_impl*.cc
da34c9d
safeMemcpy switch back to memcpy in function parseDnsObject due to st…
ab19cbd
dns_filter_lib reverted due to inability to compile with MSVC
5f76f5e
suggested changes applied
95c6815
Merge remote-tracking branch 'upstream/master'
86d1024
Merge remote-tracking branch 'upstream/master'
60f3da6
Merge branch 'master' into issue-9328
rialg 5f06fc0
Merge remote-tracking branch 'upstream/master'
10f4596
Merge remote-tracking branch 'upstream/master'
57ad370
Chnages suggested by htuch: safeMemcpy applied to proxy_protocol_head…
6e56f97
reinterpreting DnsHeaderFlags as uint16_t ptr
31f428a
typo fixed
f9d8c1f
Changes suggested by lizan@ and jmarantz@ added
b1cc18b
adding IPv6 address to the output corrected in generateV2Header
4a8b709
dns_parser reverted
d662d87
format fixed in dns_parser.cc
4406fa4
missing parentheses when safeMemcpy was used in dns_parser.cc
3dac1e3
dns_parser.cc reverted to memcpy
03520ed
Merge branch 'master' into issue-9328
rialg fff4fe1
Merge branch 'master' into issue-9328
rialg be23cf1
safeMemcpySrc & safeMemcpyDst refactored to match comments (drop size)
51c59d9
first set of changes: Do not use reinterpret_cast with safeMemcpy. Pe…
3f19f09
Merge branch 'issue-9328' of https://github.com/grial1/envoy into iss…
422b0e9
format fixed
23a5cf7
second set of changes: replace, where possible, memcpy for a safeMemc…
c789969
format fixed
d79baa3
address() function changed in address_impl.cc
3661d0a
error in address_impl.cc
27e3ccf
format fixed
83bd5dc
Revert "format fixed"
4263158
format fixed
89bc7ed
names of the variants of safeMemcpy changed
89e53c2
format fixed
08007f4
tests corrected
021efce
test corrected
9159f29
test approved
b561e2f
format message updated
ed2a892
tests corrected
a792eec
test/common/common/safe_memcpy_test.cc
f6ebfa6
tmpfile to set in motion pipeline
cb3ec94
temporary file deleted
33ef5a8
changed safeMemcpyUnsafeSrc and safeMemcpyUnsafeDst to function templ…
0630eee
typo fixed
c8f1808
void* removed
a2dfabb
address() function modified to use safeMemcpyUnsafeSrc
3b59def
Merge branch 'master' into issue-9328
rialg b8dd3e0
Merge branch 'master' into issue-9328
rialg e6760fc
Merge remote-tracking branch 'upstream/master' into issue-9328
4da526b
merged with master: fixed buffer_impl.cc
c133f5b
merge conflict solved
fc7aec5
merge
2b603f2
codec_impl_legacy.cc deleted
2632163
requested changes added
6454e2c
SafeMemcpyUnsafeDstTest modified to use basic C++
6d6aba8
safe_memcpy_test fixed
aca7254
Merge branch 'master' of https://github.com/envoyproxy/envoy into iss…
1151c7a
Tests coverage acceptable threshold reduced by 0.1 for source/extensi…
8bc497b
merge conflicts resolved
2cd8aaf
merge conflicts resolved
ddeed70
merge conflicts resolved
File filter
Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
There are no files selected for viewing
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
this does not look correct. Assigning this pointer is not going to move the bytes.
Really I think the problem here is I gave a suggestion for how I would build SAFE_MEMCPY that is not quite sufficient for all use-cases. Maybe we can use MemBlockBuilder for these dynamic cases, but we need to figure out how to do it without incurring extra copies, or performance will suffer.
For example, maybe we can have a variant of MemBlockBuilder (a subclass, perhaps) where the target memory and its size is provided to the constructor.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I agree on limiting the amount of temporary memory used to make the copy, however, would you consider that modifying the MemBlockBuilder with new methods could accomplish this task? why is it necessary a child class? I thought that MemBlockBuilder already meant the target. So, the issue is to find the an efficient way move the data in order to be accessible from the pointer in the code (i.e. dest).
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I haven't fully thought it through. But currently MemBlockBuilder owns its memory, and while you can release it, it's possible that some of the call-sites you have would be better off supplying an already-allocated buffer to MemBlockBuilder.
Determining the memory management policy is the reason I thought of using a subclass. But it's not worth making it virtual.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I just looked at MemBlockBuilder again. And have two thoughts:
If we do want MemBlockBuilder to populate an already-allocated buffer, I think it's not too hard. We need an extra constructor and an extra member variable. We can use the unique_ptr to hold data allocated by the current constructor. Let's call that
owned_data_, and makedata_instead be a raw pointer. In the current constructor, owned_data_ will be allocated, and data_ will be set to that. And in the new constructor, a passed-in pointer will be provided for initializing data_, andowned_data_will never be populated, thus never freed.Let's double-check that we really need to do this. Maybe we can change the classes that already allocate their own storage to simply delegate that responsibility to MemBlockBuilder, which can be used as an instance variable.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Ok, those proposed changes seem feasible, in this case, to be sure I have got you right, the data_ member would be a pointer to an already allocated external buffer, isn’t it? Hence, the above snippet could be something like
By the way, answering to a previous comment, I've started to compile the project locally, so I expected to push new changes at sometime during the next week.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
data_would point toowned_data_.get()in this case of the current constructor. And you would add a new constructor that would setdata_to a passed-in external pointer.