Skip to content

xds: keep a reference to old config in the filter config provider#12479

Merged
mattklein123 merged 6 commits intoenvoyproxy:masterfrom
kyessenov:fix_ecds_slots
Aug 6, 2020
Merged

xds: keep a reference to old config in the filter config provider#12479
mattklein123 merged 6 commits intoenvoyproxy:masterfrom
kyessenov:fix_ecds_slots

Conversation

@kyessenov
Copy link
Contributor

Signed-off-by: Kuat Yessenov kuat@google.com

Commit Message: Some filter factories allocate TLS slots shared via shared pointers. On a filter config update, the last filter factory reference happens to be deleted on a worker thread, which causes a runtime failure since TLS slots must be deleted on the main thread. The solution is to prolong the life of the filter factory using main thread completion callback.
Fixes envoyproxy/envoy-wasm#601
Additional Description:
Risk Level: low
Testing: manual
Docs Changes:
Release Notes:

Signed-off-by: Kuat Yessenov <kuat@google.com>
@kyessenov
Copy link
Contributor Author

cc @jplevyak

@kyessenov
Copy link
Contributor Author

I'll add a test if we can agree on the approach. I'd need to mock a filter with a slot to be truthful to the original issue

jplevyak
jplevyak previously approved these changes Aug 5, 2020
@mattklein123 mattklein123 self-assigned this Aug 5, 2020
Copy link
Member

@mattklein123 mattklein123 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think this is a fine approach. The history here around why TLS gets deleted on the main thread is around simplicity. It's come up a few times to allow this to happen from any thread. I'm not sure if you want to tackle that or not. Thank you for fixing either way!

/wait

}
return previous;
},
[old_config]() {});
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can you add more comments in the final change?

Signed-off-by: Kuat Yessenov <kuat@google.com>
Signed-off-by: Kuat Yessenov <kuat@google.com>
@kyessenov
Copy link
Contributor Author

PTAL. Tested with --runs_per_test =1000 and found that the previous attempt didn't work (it was capturing n-2 version). The new attempt appears to pass.

Signed-off-by: Kuat Yessenov <kuat@google.com>
@kyessenov
Copy link
Contributor Author

Fixes #12315

Signed-off-by: Kuat Yessenov <kuat@google.com>
Copy link
Member

@mattklein123 mattklein123 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks LGTM with small comments.

/wait


FilterConfigSubscriptionSharedPtr subscription_;
const std::set<std::string> require_type_urls_;
absl::optional<Envoy::Http::FilterFactoryCb> config_{absl::nullopt};
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

nit: I don't think this needs to be an optional? It can just be a null lambda? Also can we call it current_config_ or something like that with a comment about what this member is for?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think it's good for consistency with the rest. Lambda pointer is rather confusing and doesn't own the underlying values which is the intent. I'll change the name and comment.

Comment on lines +48 to +49
auto filter_config = std::make_shared<SetResponseCodeFilter>(proto_config.prefix(),
proto_config.code(), context);
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

nit: can we make this more realistic. The filter should be allocated inside the lambda below, with the TLS slot allocated outside. This could be done by just allocating the slot outside and passing it, or have a config object and passing that to the filter as is done elsewhere.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I was just copying what wasm filter did. I guess it's not idiomatic.

Signed-off-by: Kuat Yessenov <kuat@google.com>
Copy link
Member

@mattklein123 mattklein123 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Awesome, thank you!

@mattklein123 mattklein123 merged commit df7c167 into envoyproxy:master Aug 6, 2020
istio-testing pushed a commit to istio/envoy that referenced this pull request Aug 6, 2020
* merge fix

Signed-off-by: Kuat Yessenov <kuat@google.com>

* format fix

Signed-off-by: Kuat Yessenov <kuat@google.com>
chaoqin-li1123 pushed a commit to chaoqin-li1123/envoy that referenced this pull request Aug 7, 2020
…voyproxy#12479)

Some filter factories allocate TLS slots shared via shared pointers. On a filter config update, the last filter factory reference happens to be deleted on a worker thread, which causes a runtime failure since TLS slots must be deleted on the main thread. The solution is to prolong the life of the filter factory using main thread completion callback.

Signed-off-by: Kuat Yessenov <kuat@google.com>
chaoqin-li1123 pushed a commit to chaoqin-li1123/envoy that referenced this pull request Aug 7, 2020
…voyproxy#12479)

Some filter factories allocate TLS slots shared via shared pointers. On a filter config update, the last filter factory reference happens to be deleted on a worker thread, which causes a runtime failure since TLS slots must be deleted on the main thread. The solution is to prolong the life of the filter factory using main thread completion callback.

Signed-off-by: Kuat Yessenov <kuat@google.com>
Signed-off-by: chaoqinli <chaoqinli@google.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

crash in filter config update

3 participants