PHOENIX-7528 Add metrics for overall wall clock query wait time in Phoenix client thread pool and total time spent in executing HBase scan tasks #2077
Conversation
| long vpk3 = 1000; | ||
| try(Connection conn = DriverManager.getConnection(getUrl())) { | ||
| PreparedStatement stmt = conn.prepareStatement(upsertRows); | ||
| stmt.execute(creatTableDdl); |
There was a problem hiding this comment.
This is a slightly weird pattern. Creating a table with a PreparedStatement for upsert rows. Generally, we just do conn.createStatement().execute(ddl)
| SecurityException, IllegalArgumentException, IllegalAccessException { | ||
| private void changeInternalStateForTesting(PhoenixResultSet rs, | ||
| ReadMetricQueue testMetricsQueue) throws | ||
| NoSuchFieldException, SecurityException, IllegalArgumentException, |
There was a problem hiding this comment.
Maybe just change it to throws Exception
| @@ -1298,5 +1301,225 @@ public Connection call() throws Exception { | |||
| } | |||
| } | |||
|
|
|||
| @Test | |||
| public void testPhoenixClientQueueWaitTimeAndEndToEndTime() throws SQLException, | |||
| NoSuchFieldException, | |||
There was a problem hiding this comment.
Just change it to throws Exception
| } | ||
| getRows += ")"; | ||
| final String upsertRows = "UPSERT INTO " + tableName + " VALUES(?, ?, ?, ?, ?, ?)"; | ||
| String creatTableDdl = "CREATE TABLE IF NOT EXISTS " + tableName + " (\n" + |
There was a problem hiding this comment.
Generally we put the table schema first and then the upsert statements and the select statements. This makes it easy to understand the code.
| int[] taskQueueWaitTime = {10, 5, 20, 7}; | ||
| int[] taskEndToEndTime = {20, 15, 41, 16}; |
There was a problem hiding this comment.
Pass these as parameters in the constructor
|
@sanjeet006py We need more testing around different iterators like Serial, MergeSort and also different kind of queries like Aggregate |
|
Summary of the change:
JobCallableinRoundRobinResultItr.when submitting next batches of scans as scanner cache of previous batches of scan has been exhausted. By using JobCallable we can track total wall clock time spent by a query waiting in Phoenix client thread pool queue along with total wall clock time spent in executing tasks (end to end).