-
Notifications
You must be signed in to change notification settings - Fork 205
Clickhouse Native(Binary) client #931
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Open
smagellan
wants to merge
186
commits into
eclipse-vertx:master
Choose a base branch
from
smagellan:feature/clickhouse-native-client
base: master
Could not load branches
Branch not found: {{ refName }}
Loading
Could not load tags
Nothing to show
Loading
Are you sure you want to change the base?
Some commits from the old base branch may be removed from the timeline,
and old review comments may become outdated.
Open
Changes from 70 commits
Commits
Show all changes
186 commits
Select commit
Hold shift + click to select a range
8101d06
Added clickhouse InitCommand handler
smagellan 622b2fb
Fixed typo
smagellan 95a624e
implemented SimpleQueryCodec
smagellan 74ed583
added unit test
smagellan 240cb5a
added UInt16, UInt64 types support
smagellan 50a11ac
renamed nItems to nRows, removed reflection API calls
smagellan 3be7ed8
initial writeSettings implementation; initial PreparedStatement imple…
smagellan 52da104
refactored PacketReader
smagellan 057ca73
added ExtendedClickhouseTest
smagellan ab430b3
added testIntegerRanges test
smagellan 9df29ad
added Int128 type
smagellan 9edb5e1
basic lz4 compression/decompression [WIP]
smagellan 1b90509
added DateTime/DateTime64 column support
smagellan b6403a3
added UUID column support
smagellan e36625f
added Decimal column support
smagellan 88e3b2c
added TABLE_COLUMNS packet-type support
smagellan 217be04
added Replay test
smagellan 837bb65
added Enum column support
smagellan c4a4c66
added LowCardinality column support
smagellan d345fa5
Array support [WIP]
smagellan 0d48b63
skip bytes if element is NULL
smagellan b898cf9
Added Date,Float32,Float64 columns; added ClickhouseNativeBinaryDataT…
smagellan 0004994
added ClickhouseNativeCollectorTest
smagellan 331072a
added ClickhouseNativeConnectionTest
smagellan 010cfe2
added ClickhouseNativeConnectionAutoRetryTest
smagellan 98aaaa4
added ClickhouseNativeBinaryDataTypeEncodeTest and ClickhouseNativeDr…
smagellan ec3da62
reworked ArrayColumn to recode elements if needed
smagellan 940df68
reworked ArrayColumn to handle partial reads
smagellan f1c615d
initial batch insert support
smagellan 623cb2f
splitted columns into readers and writers
smagellan f386e03
finished tck test set
smagellan bb1c42f
added LowCardinality encoder
smagellan e8ebbf6
added Array encoder
smagellan 1234e7c
added DateColumnWriter and DateTimeColumnWriter
smagellan 6a11da8
reworked AllTypesTest to use ColumnChecker
smagellan c65f52b
reworked ArrayColumnReader
smagellan 070aafc
reworked AllTypesTest
smagellan 91632f5
added IPv4 column test
smagellan 3330970
added IPv6 column test
smagellan 712b52b
added Int16/32/64 tests
smagellan e79de39
fixed fragmented read issue
smagellan 0a3839e
added DateTime64 test
smagellan 4af3384
added Enum8Test
smagellan be34c67
added Enum16Test
smagellan 429e4de
added Interval array tests
smagellan 07e3f94
added enum_resolution option
smagellan ad2c556
added FixedString test
smagellan c421deb
added UInt16/32/64 test
smagellan f68a178
added Decimal32/64 and Float32 tests
smagellan 108a965
added Decimal128/256 tests
smagellan 7f8e10a
fixed TODO: reuse cached key columns for LowCardinality reader
smagellan 225ced8
fixed TODO: do not store array descriptor as lined list of nested types
smagellan f0005e4
do not do extra nested array copy
smagellan 66e6212
removed recursion from ArrayColumnWriter
smagellan e9136d5
do not reslice every row if recoding is required
smagellan 1395271
added StringCache for LowCardinality columns
smagellan 6fdbe02
added IntervalArray tests
smagellan 225dd3d
added array deduplication test
smagellan b7be852
reworked array deduplication test
smagellan cef98c2
use columnreader as accessor if array recoding is reqired
smagellan da9062c
use less reflection for empty arrays
smagellan 831b3a1
added more LowCardinality checks, fixed TODOs
smagellan ce4fdf0
moved INSERT info extraction into QueryInfo
smagellan 0643b1f
added NestedExceptionsTest
smagellan ee30664
replaced LOG.info with LOG.debug where needed
smagellan da8e34c
added copyright headers
smagellan e9af16e
fixed copyright headers
smagellan 5735b68
fixed copyright headers, reworked array to use int[] arrays as slices
smagellan 8a45ce2
fixed TODOs in pom.xml
smagellan 1e0e95a
reworked ArrayColumnReader to support primitive array
smagellan dbd1084
Merge branch 'eclipse-vertx:master' into feature/clickhouse-native-cl…
smagellan daea1c0
added charset parameter to readPascalString
smagellan c8b538d
removed TestRunner
smagellan 9212824
rebasing onto master
smagellan 5d9a37a
Merge branch 'eclipse-vertx:master' into feature/clickhouse-native-cl…
smagellan 4329849
Merge branch 'feature/clickhouse-native-client' of https://github.com…
smagellan eede5dd
rebasing onto master
smagellan 413caf9
throw ClassCastException in get(Class<T> type, int position) if can't…
smagellan fd972ed
added column index into ClassCastException message
smagellan ad76c64
replaced confusing 'native' to 'binary'
smagellan 5ff7b1f
replaced confusing 'native' to 'binary' in pom.xml
smagellan ba517dd
replaced confusing 'native' to 'binary' in tests and package-info
smagellan 3157dab
replaced confusing 'native' to 'binary' in tests and package-info
smagellan 1491473
Merge branch 'eclipse-vertx:master' into feature/clickhouse-native-cl…
smagellan 59be4d2
replaced confusing 'native' to 'binary' in pom description
smagellan ffe427e
replaced confusing 'native' to 'binary' in pom description
smagellan f39a54c
Merge branch 'eclipse-vertx:master' into feature/clickhouse-native-cl…
smagellan 7a6f899
Added clickhouse InitCommand handler
smagellan f68bb39
Fixed typo
smagellan 3e7c29c
implemented SimpleQueryCodec
smagellan 372f6d8
added unit test
smagellan 1386a5d
added UInt16, UInt64 types support
smagellan a2e88ce
renamed nItems to nRows, removed reflection API calls
smagellan e4c97cc
initial writeSettings implementation; initial PreparedStatement imple…
smagellan f7a096e
refactored PacketReader
smagellan 05411b4
added ExtendedClickhouseTest
smagellan 27fdd76
added testIntegerRanges test
smagellan c46f3e0
added Int128 type
smagellan 91a0371
basic lz4 compression/decompression [WIP]
smagellan da0ee4c
added DateTime/DateTime64 column support
smagellan d5106f3
added UUID column support
smagellan 8841c24
added Decimal column support
smagellan 957ece6
added TABLE_COLUMNS packet-type support
smagellan 92f38b1
added Replay test
smagellan e16e995
added Enum column support
smagellan fac35ba
added LowCardinality column support
smagellan 500f03f
Array support [WIP]
smagellan f1e0636
skip bytes if element is NULL
smagellan d97b16b
Added Date,Float32,Float64 columns; added ClickhouseNativeBinaryDataT…
smagellan cc27188
added ClickhouseNativeCollectorTest
smagellan 2871542
added ClickhouseNativeConnectionTest
smagellan d820e90
added ClickhouseNativeConnectionAutoRetryTest
smagellan be0635f
added ClickhouseNativeBinaryDataTypeEncodeTest and ClickhouseNativeDr…
smagellan cf8c171
reworked ArrayColumn to recode elements if needed
smagellan d13538c
reworked ArrayColumn to handle partial reads
smagellan 22bb068
initial batch insert support
smagellan d37f743
splitted columns into readers and writers
smagellan e16e1ab
finished tck test set
smagellan 672b998
added LowCardinality encoder
smagellan c15b884
added Array encoder
smagellan 233fd4f
added DateColumnWriter and DateTimeColumnWriter
smagellan 4c6f944
reworked AllTypesTest to use ColumnChecker
smagellan 41c68fd
reworked ArrayColumnReader
smagellan 13267df
reworked AllTypesTest
smagellan 888985f
added IPv4 column test
smagellan 0ef566d
added IPv6 column test
smagellan c7228b2
added Int16/32/64 tests
smagellan 4ddb4d4
fixed fragmented read issue
smagellan f43cf87
added DateTime64 test
smagellan dff9104
added Enum8Test
smagellan 82c9ec6
added Enum16Test
smagellan b4fce3d
added Interval array tests
smagellan 923f44b
added enum_resolution option
smagellan 8456b0f
added FixedString test
smagellan 8aa88a5
added UInt16/32/64 test
smagellan 73369b3
added Decimal32/64 and Float32 tests
smagellan 08be2bf
added Decimal128/256 tests
smagellan 14ae1c6
fixed TODO: reuse cached key columns for LowCardinality reader
smagellan 7c92fbe
fixed TODO: do not store array descriptor as lined list of nested types
smagellan 930a424
do not do extra nested array copy
smagellan 3851d06
removed recursion from ArrayColumnWriter
smagellan 73113d8
do not reslice every row if recoding is required
smagellan 32ddad9
added StringCache for LowCardinality columns
smagellan a2f099e
added IntervalArray tests
smagellan 8569966
added array deduplication test
smagellan 7e206ff
reworked array deduplication test
smagellan 2252717
use columnreader as accessor if array recoding is reqired
smagellan 33b839c
use less reflection for empty arrays
smagellan d0185ec
added more LowCardinality checks, fixed TODOs
smagellan af722ff
moved INSERT info extraction into QueryInfo
smagellan d974e2e
added NestedExceptionsTest
smagellan 4472783
replaced LOG.info with LOG.debug where needed
smagellan 4254305
added copyright headers
smagellan 4c7cb93
fixed copyright headers
smagellan 75cdc8a
fixed copyright headers, reworked array to use int[] arrays as slices
smagellan 57f1d3b
fixed TODOs in pom.xml
smagellan ad58bce
reworked ArrayColumnReader to support primitive array
smagellan df4df78
added charset parameter to readPascalString
smagellan 4eff81e
removed TestRunner
smagellan 009ec4d
rebasing onto master
smagellan 8724b82
rebasing onto master
smagellan 77f21e6
throw ClassCastException in get(Class<T> type, int position) if can't…
smagellan 39714dd
added column index into ClassCastException message
smagellan 9a076bc
replaced confusing 'native' to 'binary'
smagellan 8b1b28a
replaced confusing 'native' to 'binary' in pom.xml
smagellan 6d3e8e3
replaced confusing 'native' to 'binary' in tests and package-info
smagellan 6c9faf1
replaced confusing 'native' to 'binary' in tests and package-info
smagellan 152261c
replaced confusing 'native' to 'binary' in pom description
smagellan 3d91c33
replaced confusing 'native' to 'binary' in pom description
smagellan 7eb874f
rebase to 4.2.0-SNAPSHOT
smagellan e9c19a6
Merge remote-tracking branch 'origin/feature/clickhouse-native-client…
smagellan a0dd2a1
handle 'dangling' exceptions (e.g. non-existent DB exception after su…
smagellan 5cba677
Merge branch 'eclipse-vertx:master' into feature/clickhouse-native-cl…
smagellan ec30819
rebase over master
smagellan 6e686cc
Merge branch 'eclipse-vertx:master' into feature/clickhouse-native-cl…
smagellan 365247c
1) rebase over 4.3.5-SNAPSHOT; 2) fix arrays deserialization
smagellan f081249
Merge branch 'eclipse-vertx:master' into feature/clickhouse-native-cl…
smagellan b15d646
rebase over 4.4.0-SNAPSHOT
smagellan f366718
add Boolean type support
smagellan 86cf5e4
separate Signed/Unsigned column readers
smagellan b5b120c
separate Signed/Unsigned column readers
smagellan a3c95d0
finalize initial DB switching machinery
smagellan d7e6296
enable non-compressed connections again
smagellan df25675
remove traces of old Exception-after-HELLO(Metadata) handling code
smagellan fbee08e
simplify DB switch logic
smagellan 5a0cb44
Merge branch 'eclipse-vertx:master' into feature/clickhouse-native-cl…
smagellan File filter
Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
There are no files selected for viewing
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,209 @@ | ||
<?xml version="1.0"?> | ||
<!-- | ||
~ Copyright (C) 2017 Julien Viet | ||
~ | ||
~ Licensed under the Apache License, Version 2.0 (the "License"); | ||
~ you may not use this file except in compliance with the License. | ||
~ You may obtain a copy of the License at | ||
~ | ||
~ http://www.apache.org/licenses/LICENSE-2.0 | ||
~ | ||
~ Unless required by applicable law or agreed to in writing, software | ||
~ distributed under the License is distributed on an "AS IS" BASIS, | ||
~ WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. | ||
~ See the License for the specific language governing permissions and | ||
~ limitations under the License. | ||
~ | ||
--> | ||
<project xmlns="http://maven.apache.org/POM/4.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd"> | ||
|
||
<modelVersion>4.0.0</modelVersion> | ||
|
||
<parent> | ||
<groupId>io.vertx</groupId> | ||
<artifactId>vertx-sql-client-parent</artifactId> | ||
<version>4.0.3-SNAPSHOT</version> | ||
</parent> | ||
|
||
<artifactId>vertx-clickhouse-native-client</artifactId> | ||
|
||
<name>Vertx Clickhouse native Client</name> | ||
<url>https://github.com/eclipse-vertx/vertx-sql-client</url> | ||
<description>The Reactive Clickhouse Client</description> | ||
|
||
<properties> | ||
<doc.skip>false</doc.skip> | ||
<docs.dir>${project.basedir}/src/main/docs</docs.dir> | ||
<generated.dir>${project.basedir}/src/main/generated</generated.dir> | ||
<!-- Set to a value for testing with a specific database --> | ||
<embedded.clickhouse.version /> | ||
<log4j.version>2.14.1</log4j.version> | ||
</properties> | ||
|
||
<dependencies> | ||
|
||
<!-- Vert.x dependencies --> | ||
<dependency> | ||
<groupId>io.vertx</groupId> | ||
<artifactId>vertx-core</artifactId> | ||
</dependency> | ||
<dependency> | ||
<groupId>io.vertx</groupId> | ||
<artifactId>vertx-codegen</artifactId> | ||
<optional>true</optional> | ||
</dependency> | ||
<dependency> | ||
<groupId>io.vertx</groupId> | ||
<artifactId>vertx-docgen</artifactId> | ||
<optional>true</optional> | ||
</dependency> | ||
<dependency> | ||
<groupId>io.vertx</groupId> | ||
<artifactId>vertx-sql-client</artifactId> | ||
</dependency> | ||
|
||
<dependency> | ||
<groupId>org.lz4</groupId> | ||
<artifactId>lz4-java</artifactId> | ||
<version>1.7.1</version> | ||
</dependency> | ||
|
||
<dependency> | ||
<groupId>io.vertx</groupId> | ||
<artifactId>vertx-sql-client</artifactId> | ||
<type>test-jar</type> | ||
<scope>test</scope> | ||
</dependency> | ||
|
||
<dependency> | ||
<groupId>com.fasterxml.jackson.dataformat</groupId> | ||
<artifactId>jackson-dataformat-yaml</artifactId> | ||
<scope>test</scope> | ||
</dependency> | ||
|
||
<dependency> | ||
<groupId>org.testcontainers</groupId> | ||
<artifactId>clickhouse</artifactId> | ||
<version>${testcontainers.version}</version> | ||
<scope>test</scope> | ||
</dependency> | ||
|
||
<dependency> | ||
<groupId>ru.yandex.clickhouse</groupId> | ||
<artifactId>clickhouse-jdbc</artifactId> | ||
<version>0.2.6</version> | ||
<scope>test</scope> | ||
</dependency> | ||
|
||
<dependency> | ||
<groupId>org.slf4j</groupId> | ||
<artifactId>slf4j-api</artifactId> | ||
<version>1.7.30</version> | ||
<scope>test</scope> | ||
</dependency> | ||
|
||
<dependency> | ||
<groupId>org.apache.logging.log4j</groupId> | ||
<artifactId>log4j-slf4j-impl</artifactId> | ||
<version>${log4j.version}</version> | ||
<scope>test</scope> | ||
</dependency> | ||
|
||
<dependency> | ||
<groupId>org.apache.logging.log4j</groupId> | ||
<artifactId>log4j-api</artifactId> | ||
<version>${log4j.version}</version> | ||
<scope>test</scope> | ||
</dependency> | ||
|
||
<dependency> | ||
<groupId>org.apache.logging.log4j</groupId> | ||
<artifactId>log4j-web</artifactId> | ||
<version>${log4j.version}</version> | ||
<scope>test</scope> | ||
</dependency> | ||
|
||
<dependency> | ||
<groupId>org.apache.logging.log4j</groupId> | ||
<artifactId>log4j-core</artifactId> | ||
<version>${log4j.version}</version> | ||
<scope>test</scope> | ||
</dependency> | ||
|
||
<dependency> | ||
<groupId>org.apache.logging.log4j</groupId> | ||
<artifactId>log4j-jul</artifactId> | ||
<version>${log4j.version}</version> | ||
<scope>test</scope> | ||
</dependency> | ||
|
||
|
||
<dependency> | ||
<groupId>org.openjdk.jmh</groupId> | ||
<artifactId>jmh-core</artifactId> | ||
<version>${jmh.version}</version> | ||
<scope>test</scope> | ||
</dependency> | ||
<dependency> | ||
<groupId>org.openjdk.jmh</groupId> | ||
<artifactId>jmh-generator-annprocess</artifactId> | ||
<version>${jmh.version}</version> | ||
<scope>test</scope> | ||
</dependency> | ||
</dependencies> | ||
|
||
<build> | ||
<pluginManagement> | ||
<plugins> | ||
<plugin> | ||
<artifactId>maven-surefire-plugin</artifactId> | ||
<configuration> | ||
<argLine>-Xmx1024M</argLine> | ||
<systemPropertyVariables> | ||
<target.dir>${project.build.directory}</target.dir> | ||
<embedded.clickhouse.version>${embedded.clickhouse.version}</embedded.clickhouse.version> | ||
<connection.uri>${connection.uri}</connection.uri> | ||
<!-- TODO: implement encryption --> | ||
<tls.connection.uri>${tls.connection.uri}</tls.connection.uri> | ||
</systemPropertyVariables> | ||
</configuration> | ||
</plugin> | ||
</plugins> | ||
</pluginManagement> | ||
<plugins> | ||
<plugin> | ||
<groupId>org.bsc.maven</groupId> | ||
<artifactId>maven-processor-plugin</artifactId> | ||
<executions> | ||
<execution> | ||
<id>generate-sources</id> | ||
<configuration> | ||
<optionMap> | ||
<docgen.source>${project.basedir}/../vertx-sql-client/src/main/asciidoc/*.adoc,${asciidoc.dir}/*.adoc</docgen.source> | ||
</optionMap> | ||
</configuration> | ||
</execution> | ||
</executions> | ||
</plugin> | ||
<plugin> | ||
<artifactId>maven-assembly-plugin</artifactId> | ||
<executions> | ||
<!-- Override sources to use custom descriptor to have all adoc files --> | ||
<execution> | ||
<id>package-sources</id> | ||
<configuration> | ||
<descriptors> | ||
<descriptor>${project.basedir}/../assembly/sources.xml</descriptor> | ||
</descriptors> | ||
<descriptorRefs> | ||
<descriptorRef>none</descriptorRef> | ||
</descriptorRefs> | ||
<ignoreMissingDescriptor>true</ignoreMissingDescriptor> | ||
</configuration> | ||
</execution> | ||
</executions> | ||
</plugin> | ||
</plugins> | ||
</build> | ||
|
||
</project> |
29 changes: 29 additions & 0 deletions
29
...nerated/io/vertx/clickhouse/clickhousenative/ClickhouseNativeConnectOptionsConverter.java
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,29 @@ | ||
package io.vertx.clickhouse.clickhousenative; | ||
|
||
import io.vertx.core.json.JsonObject; | ||
import io.vertx.core.json.JsonArray; | ||
import io.vertx.core.json.impl.JsonUtil; | ||
import java.time.Instant; | ||
import java.time.format.DateTimeFormatter; | ||
|
||
/** | ||
* Converter and mapper for {@link io.vertx.clickhouse.clickhousenative.ClickhouseNativeConnectOptions}. | ||
* NOTE: This class has been automatically generated from the {@link io.vertx.clickhouse.clickhousenative.ClickhouseNativeConnectOptions} original class using Vert.x codegen. | ||
*/ | ||
public class ClickhouseNativeConnectOptionsConverter { | ||
|
||
|
||
public static void fromJson(Iterable<java.util.Map.Entry<String, Object>> json, ClickhouseNativeConnectOptions obj) { | ||
for (java.util.Map.Entry<String, Object> member : json) { | ||
switch (member.getKey()) { | ||
} | ||
} | ||
} | ||
|
||
public static void toJson(ClickhouseNativeConnectOptions obj, JsonObject json) { | ||
toJson(obj, json.getMap()); | ||
} | ||
|
||
public static void toJson(ClickhouseNativeConnectOptions obj, java.util.Map<String, Object> json) { | ||
} | ||
} |
Oops, something went wrong.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
can we stick to the current test container version ?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I was not able to start clickhouse testcontainer with 1.12.4. Should not be a problem since it is already upgraded in master: https://github.com/eclipse-vertx/vertx-sql-client/blob/master/pom.xml#L62