HBASE-29090: Add server-side load metrics to client results #6623
Conversation
This comment has been minimized.
This comment has been minimized.
| @@ -274,6 +278,7 @@ message Scan { | |||
| } | |||
| optional ReadType readType = 23 [default = DEFAULT]; | |||
| optional bool need_cursor_result = 24 [default = false]; | |||
| optional bool resultMetricsEnabled = 25 [default = false]; | |||
There was a problem hiding this comment.
Nit; enable_result_metrics matches the existing fields better (and same note for other protobuf changes)
| optional bool resultMetricsEnabled = 25 [default = false]; | |
| optional bool enable_result_metrics = 25 [default = false]; |
There was a problem hiding this comment.
updated. this file has both camel & snake case littered everywhere I should probably avoid adding additional inconsistencies
| * Statistics about the Result's server-side metrics | ||
| */ | ||
| message ResultMetrics { | ||
| required uint64 blockBytesScanned = 1; |
There was a problem hiding this comment.
Not sure what the standard is for HBase, but I know that optional values with a default are more future proof than required fields, and I think that would be better
There was a problem hiding this comment.
wasn't familiar with the move away from required fields, I'm happy to make this optional.
| @@ -1434,6 +1439,10 @@ public static ClientProtos.Result toResult(final Result result, boolean encodeTa | |||
| builder.setStale(result.isStale()); | |||
| builder.setPartial(result.mayHaveMoreCellsInRow()); | |||
|
|
|||
| if (result.getMetrics() != null) { | |||
There was a problem hiding this comment.
Seems like the result metrics will get lost for exists calls; thoughts on handling that more explicitly? Maybe throwing an error if someone tries to enable metrics on an Get with checkExistenceOnly == true
There was a problem hiding this comment.
Throwing an error might be a bit aggressive, though I'd be curious to know what others think. I'm not sure if we're happy throwing an unchecked exception from setCheckExistenceOnly and if we wanted to throw an IOException, we'd have to change the signature.
Maybe it's enough that the metics returned will be null
| @@ -95,6 +95,7 @@ public Get(Get get) { | |||
| this.setFilter(get.getFilter()); | |||
| this.setReplicaId(get.getReplicaId()); | |||
| this.setConsistency(get.getConsistency()); | |||
| this.setResultMetricsEnabled(get.isResultMetricsEnabled()); | |||
There was a problem hiding this comment.
Nit; Thoughts on the name QueryMetrics? I think that ResultMetrics could be interpreted as "metrics about the result" instead of "metrics about the query operation"
There was a problem hiding this comment.
Query metrics makes sense, I mostly just wanted a way to distinguish between the existing scan metrics, and these new metrics
| @@ -3425,6 +3438,12 @@ private void scan(HBaseRpcController controller, ScanRequest request, RegionScan | |||
| } | |||
| boolean mayHaveMoreCellsInRow = scannerContext.mayHaveMoreCellsInRow(); | |||
| Result r = Result.create(values, null, stale, mayHaveMoreCellsInRow); | |||
|
|
|||
| if (request.getScan().getResultMetricsEnabled()) { | |||
There was a problem hiding this comment.
What's the reason for not directly setting it on the result for scans?
There was a problem hiding this comment.
good question, scans are a bit tricky just due to how the results are constructed on the client side. you can take a look at how the response converter builds these responses. They're created from the cells, so we need to send the metrics separately through the wire in this case and hydrate them client-side.
This comment has been minimized.
This comment has been minimized.
hgromer
force-pushed
the
HBASE-29090
branch
2 times, most recently
from
3d96194 to
8680147
Compare
This comment has been minimized.
This comment has been minimized.
This comment has been minimized.
This comment has been minimized.
|
I'm actively working through the unit test failures that come up |
This comment has been minimized.
This comment has been minimized.
This comment has been minimized.
This comment has been minimized.
| @@ -44,21 +44,21 @@ public void itSerializesScan() { | |||
| Scan scan = new Scan(); | |||
| scan.withStartRow(Bytes.toBytes(123)); | |||
| scan.withStopRow(Bytes.toBytes(456)); | |||
| String expectedOutput = | |||
| "{\n" + " \"startTime\": 1,\n" + " \"processingTime\": 2,\n" + " \"queueTime\": 3,\n" | |||
There was a problem hiding this comment.
it seems like adding a field completely changed the order of the serialized fields. all I did here was add the queryMetricsEnabled field
|
🎊 +1 overall
This message was automatically generated. |
|
🎊 +1 overall
This message was automatically generated. |
No description provided.