Skip to content
This repository has been archived by the owner on Aug 2, 2022. It is now read-only.

Improve logging in new SQL engine #912

Merged

Conversation

dai-chen
Copy link
Member

@dai-chen dai-chen commented Dec 10, 2020

Issue #, if available: #817

Description of changes:

  1. Avoid dump request object which contains original query: Because we've logged sanitized query with request ID, actually there is no need to dump it again.
  2. Add log for fall back reason: To make our development easier, added info log for fall back reason when explain queries. And also update README for user who wants to figure out this too.

Examples:

  1. Logging for request handled by new SQL engine:
POST _opendistro/_sql
{
  "query": " SELECT age FROM accounts "
}
[aa74566f-ad79-4c3c-a221-5e1f6c6b88ed] Incoming request /_opendistro/_sql?pretty=true: ( SELECT identifier FROM table )
[aa74566f-ad79-4c3c-a221-5e1f6c6b88ed] Request is handled by new SQL query engine
  1. Logging for explaining fallback due to unsupported syntax exception (ex. LIMIT) thrown by ANTLR parser:
POST _opendistro/_sql/_explain
{
  "query": " SELECT age FROM accounts LIMIT 1"
}
... Request is falling back to old SQL engine due to: Failed to parse query due to offending symbol [LIMIT] at: '
      SELECT age FROM accounts LIMIT' <--- HERE... More details: Expecting tokens in {<EOF>, ';'}
  1. Logging for explaining fallback due to semantic exception (exception thrown manually from Analyzer):
POST _opendistro/_sql/_explain
{
  "query": " SELECT projects FROM employees "
}
... Request is falling back to old SQL engine due to: Identifier [projects] of type [ARRAY] is not supported yet

By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.

@dai-chen dai-chen added the SQL label Dec 10, 2020
@dai-chen dai-chen self-assigned this Dec 10, 2020
@codecov
Copy link

codecov bot commented Dec 10, 2020

Codecov Report

Merging #912 (ad66ca2) into develop (d170da3) will increase coverage by 0.00%.
The diff coverage is n/a.

Impacted file tree graph

@@            Coverage Diff             @@
##             develop     #912   +/-   ##
==========================================
  Coverage      99.85%   99.85%           
  Complexity      2149     2149           
==========================================
  Files            216      216           
  Lines           4851     4852    +1     
  Branches         323      323           
==========================================
+ Hits            4844     4845    +1     
  Misses             5        5           
  Partials           2        2           
Impacted Files Coverage Δ Complexity Δ
...ssion/operator/arthmetic/MathematicalFunction.java 100.00% <0.00%> (ø) 122.00% <0.00%> (ø%)

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update d170da3...ad66ca2. Read the comment docs.

@dai-chen dai-chen marked this pull request as ready for review December 11, 2020 17:56
@dai-chen dai-chen merged commit 6f9102f into opendistro-for-elasticsearch:develop Dec 12, 2020
@dai-chen dai-chen deleted the fix-log-security-issue branch December 12, 2020 01:17
penghuo pushed a commit that referenced this pull request Dec 15, 2020
* Remove sensitive log and add log for fallback

* Update readme
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants