Skip to content

Commit d7a933e

Browse files
authored
GetQueryStatisticsScript implementation, runner parallelization
1 parent 9722975 commit d7a933e

Some content is hidden

Large Commits have some content hidden by default. Use the searchbox below for content that may be hidden.

42 files changed

+1860
-706
lines changed

Directory.Build.props

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -1,5 +1,5 @@
11
<Project>
22
<PropertyGroup>
3-
<Version>0.3.0</Version>
3+
<Version>0.4.0</Version>
44
</PropertyGroup>
55
</Project>

README.md

Lines changed: 162 additions & 30 deletions
Original file line numberDiff line numberDiff line change
@@ -1,35 +1,41 @@
11
# QAToolKit Engine Database library
2+
23
[![Build .NET Library](https://github.com/qatoolkit/qatoolkit-engine-database-net/workflows/Build%20.NET%20Library/badge.svg)](https://github.com/qatoolkit/qatoolkit-engine-database-net/actions)
34
[![CodeQL](https://github.com/qatoolkit/qatoolkit-engine-database-net/workflows/CodeQL%20Analyze/badge.svg)](https://github.com/qatoolkit/qatoolkit-engine-database-net/security/code-scanning)
45
[![Sonarcloud Quality gate](https://github.com/qatoolkit/qatoolkit-engine-database-net/workflows/Sonarqube%20Analyze/badge.svg)](https://sonarcloud.io/dashboard?id=qatoolkit_qatoolkit-engine-database-net)
56
[![NuGet package](https://img.shields.io/nuget/v/QAToolKit.Engine.DataBase?label=QAToolKit.Engine.Database)](https://www.nuget.org/packages/QAToolKit.Engine.Database/)
67
[![Discord](https://img.shields.io/discord/787220825127780354?color=%23267CB9&label=Discord%20chat)](https://discord.gg/hYs6ayYQC5)
78

89
## Description
9-
`QAToolKit.Engine.Database` is a .NET standard library, which can be used to do database fitness tests. For example, if you want to test that table is present in database, or certain number of records exist in specific table or if a record exists.
10+
11+
`QAToolKit.Engine.Database` is a .NET standard library, which can be used to do database fitness tests. For example, if
12+
you want to test that table is present in database, or certain number of records exist in specific table or if a record
13+
exists.
1014

1115
`DatabaseTestType` enumeration currently described those three test types:
16+
1217
- `ObjectExits`: Check if table, view or stored procedure exists.
1318
- `RecordCount`: Check if record count in specific table equals an expression.
1419
- `RecordExist`: Check if a record exists in specific table.
15-
- `CustomScript`: Check if a custom script returns results. You can write a custom select query and check if it has any results.
20+
- `CustomScript`: Check if a custom script returns results. You can write a custom select query and check if it has any
21+
results.
1622

1723
Currently supports only relational databases: `SQLServer`, `MySQL` and `PostgreSQL`.
1824

1925
Get in touch with me on:
2026

2127
[![Discord](https://img.shields.io/discord/787220825127780354?color=%23267CB9&label=Discord%20chat)](https://discord.gg/hYs6ayYQC5)
2228

23-
## Sample
29+
## Generators and Runners
2430

2531
```csharp
2632
var generator = new SqlServerTestGenerator(options =>
2733
{
2834
options.AddDatabaseObjectExitsRule(new string[] { "mytable" }, DatabaseObjectType.Table);
2935

30-
options.AddDatabaseRecordExitsRule(new List<DatabaseRecordExistRule>()
36+
options.AddDatabaseRecordExitsRule(new List<RecordExistRule>()
3137
{
32-
new DatabaseRecordExistRule()
38+
new RecordExistRule()
3339
{
3440
TableName = "mytable",
3541
ColumnName = "name",
@@ -38,9 +44,9 @@ var generator = new SqlServerTestGenerator(options =>
3844
}
3945
});
4046

41-
options.AddDatabaseRecordsCountRule(new List<DatabaseRecordCountRule>()
47+
options.AddDatabaseRecordsCountRule(new List<RecordCountRule>()
4248
{
43-
new DatabaseRecordCountRule()
49+
new RecordCountRule()
4450
{
4551
TableName = "mytable",
4652
Count = 100,
@@ -54,19 +60,39 @@ var generator = new SqlServerTestGenerator(options =>
5460
"SELECT * FROM [table] WHERE timestamp = '2016-05-31'",
5561
"SELECT * FROM [table] WHERE value < 1"
5662
});
63+
64+
//Currently only for SQL Server
65+
options.CaptureQueryStatistics(new string[]
66+
{
67+
@"SELECT * FROM myTable1 INNER JOIN myTable ON myTable1.id=myTable.sampleID;"
68+
},
69+
new QueryStatisticsType[]
70+
{
71+
QueryStatisticsType.Time, QueryStatisticsType.Io
72+
});
5773
});
5874

5975
List<DatabaseTest> scripts = await generator.Generate();
6076
```
61-
The code above will generate a SQLServer `DatabaseTest` list, which will be used by runner to run the tests against database.
77+
78+
The code above will generate a SQLServer `DatabaseTest` list, which will be used by runner to run the tests against
79+
database.
6280

6381
Above example adds all three test types to the generator:
82+
6483
- `AddDatabaseObjectExitsRule`: will check if a table `mytable` exists in the database.
6584
- `AddDatabaseRecordExitsRule`: will check if a record in table `mytable` with `name` equals `myname` exists.
6685
- `AddDatabaseRecordsCountRule`: will check if there is exactly 100 records in the `mytable` table.
67-
- `AddCustomSqlRule`: will check if a custom script returns any results. This way you have a lot of freedom to define your own queries. Please note, that the query you specify is wrapped in the `EXISTS` clause for a specific database vendor.
86+
- `AddCustomSqlRule`: will check if a custom script returns any results. This way you have a lot of freedom to define
87+
your own queries. Please note, that the query you specify is wrapped in the `EXISTS` clause for a specific database
88+
vendor.
89+
- `CaptureQueryStatistics`: capture the `time` and/or `IO` statistics by running the specified queries. Runner will
90+
collect the results from
91+
the [InfoMessage event](https://docs.microsoft.com/en-us/dotnet/api/system.data.sqlclient.sqlconnection.infomessage?view=dotnet-plat-ext-5.0)
92+
. This is currently implemented only for SQL Server.
6893

69-
Alternatively if you want to use `MySQL` or `PostgreSQL` generators, you can use MySqlTestGenerator` or `PostgresqlTestGenerator` respectively.
94+
Alternatively if you want to use `MySQL` or `PostgreSQL` generators, you can use MySqlTestGenerator` or `
95+
PostgresqlTestGenerator` respectively.
7096

7197
To run the tests, we create a `SqlServerTestRunner` runner:
7298

@@ -79,34 +105,140 @@ var runner = new SqlServerTestRunner(scripts, options =>
79105
List<DatabaseTestResult> results = await runner.Run();
80106
```
81107

82-
Alternatively if you want to use `MySQL` or `PostgreSQL` runners, you can use MySqlTestRunner` or `PostgresqlTestRunner` respectively.
108+
With version 0.4.0 you can run runners in parallel, by specifying the number in Run() method:
109+
110+
```csharp
111+
//Run in 5 parallels
112+
var results = runner.Run(5);
113+
```
114+
115+
Alternatively if you want to use `MySQL` or `PostgreSQL` runners, you can use MySqlTestRunner` or `PostgresqlTestRunner`
116+
respectively.
117+
118+
Please note that **your user must have correct database permissions**. I suggest a read-only permissions that can also
119+
access `sys` or `information_schema` schemas.
120+
121+
Below is a sample of database test result:
122+
123+
```json
124+
[
125+
{
126+
"Id": "c9ac5d7f-d4fd-44e5-a743-837be2740b72",
127+
"Hash": "0x4E263AF99F6C0D3AB0268A249CC14E27",
128+
"RunAt": "2021-03-07T12:39:16.4181917+01:00",
129+
"DatabaseResult": null,
130+
"Variable": null,
131+
"Script": "SET STATISTICS TIME ON;SET STATISTICS IO ON;SELECT * FROM myTable1 INNER JOIN myTable ON myTable1.id=myTable.sampleID;SET STATISTICS TIME OFF;SET STATISTICS IO OFF;",
132+
"DatabaseTestType": 4,
133+
"DatabaseKind": 1,
134+
"ServerCpuTime": 234,
135+
"ServerElapsedTime": 494,
136+
"TotalElapsedTime": 887,
137+
"Statistics": {
138+
"Workfile": {
139+
...
140+
},
141+
"Worktable": {
142+
...
143+
},
144+
"myTable": {
145+
"ScanCount": 1,
146+
"LogicalReads": 1674,
147+
"PhysicalReads": 0,
148+
"PageServerReads": 0,
149+
"ReadAheadReads": 0,
150+
"PageServerReadAheadReads": 0,
151+
"LobLogicalReads": 0,
152+
"LobPhysicalReads": 0,
153+
"LobPageServerReads": 0,
154+
"LobReadAheadReads": 0,
155+
"LobPageServerReadAheadReads": 0
156+
},
157+
"myTable1": {
158+
"ScanCount": 1,
159+
"LogicalReads": 218,
160+
"PhysicalReads": 0,
161+
"PageServerReads": 0,
162+
"ReadAheadReads": 0,
163+
"PageServerReadAheadReads": 0,
164+
"LobLogicalReads": 0,
165+
"LobPhysicalReads": 0,
166+
"LobPageServerReads": 0,
167+
"LobReadAheadReads": 0,
168+
"LobPageServerReadAheadReads": 0
169+
}
170+
}
171+
}
172+
]
173+
```
174+
A bit of explanations:
175+
176+
- `Id`: A unique GUID generated on the fly for every result.
177+
- `Hash`: is a MurMurHash of the script string.
178+
- `Statistics`: those are collected only if a `DatabaseTest` with `CaptureQueryStatistics` is used. It contains the parsed information retrieved from the SQL server (currently there is no support PostgreSQL or MySQL).
179+
180+
## Asserters
181+
182+
If you want to assert query statistics in version `0.4.0` and above, you can assert the results with `TestAsserter`.
183+
Runner will return `DatabaseResult = null`, because it can not assert the vast arrays of results when executing a query.
184+
You need to do that after you run your queries.
185+
186+
Sample code for asserter:
187+
188+
```csharp
189+
var generator = new SqlServerTestGenerator(options =>
190+
{
191+
options.CaptureQueryStatistics(new string[]
192+
{
193+
@"SELECT * FROM myTable1 INNER JOIN myTable ON myTable1.id=myTable.sampleID;"
194+
},
195+
new QueryStatisticsType[]
196+
{
197+
QueryStatisticsType.Time, QueryStatisticsType.Io
198+
});
199+
});
83200

84-
Please note that **your user must have correct database permissions**. I suggest a read-only permissions that can also access `sys` or `information_schema` schemas.
201+
List<DatabaseTest> scripts = await generator.Generate();
202+
var runner = new SqlServerTestRunnerFactory(scripts,
203+
options => options.AddSQLServerConnection(
204+
"server=localhost;user=sa;password=password;Initial Catalog=MyTable"));
205+
var results = await runner.Run();
206+
207+
var asserter = new TestAsserter(result)
208+
.EvaluateTotalElapsedTime(x => x > 0 && x < 1500)
209+
.EvaluateLogicalReads(x => x < 2000)//, "MyTable")
210+
.WithCustomProperty(new KeyValuePair<string, string>("script",result.Script))
211+
.AssertAll();
212+
```
85213

86214
## To-Do
87215

88-
- Add more test types if necessary.
216+
- Collect time / IO statistics for `PostgreSQL` and `MySQL` (currently throws `NotImplementedException`).
217+
218+
## Breaking changes
219+
220+
### version 0.3.0 to 0.4.0
221+
222+
- Renamed model names, by removing `Database` prefix from names, for example `DatabaseRecordExistRule` was renamed
223+
to `RecordExistRule`
224+
- Renamed SQL runner to support parallel running. For example `SqlServerTestRunner` was renamed
225+
to `SqlServerTestRunnerFactory`.
89226

90227
## License
91228

92229
MIT License
93230

94231
Copyright (c) 2020-2021 Miha Jakovac
95232

96-
Permission is hereby granted, free of charge, to any person obtaining a copy
97-
of this software and associated documentation files (the "Software"), to deal
98-
in the Software without restriction, including without limitation the rights
99-
to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
100-
copies of the Software, and to permit persons to whom the Software is
101-
furnished to do so, subject to the following conditions:
102-
103-
The above copyright notice and this permission notice shall be included in all
104-
copies or substantial portions of the Software.
105-
106-
THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
107-
IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
108-
FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
109-
AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
110-
LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
111-
OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
112-
SOFTWARE.
233+
Permission is hereby granted, free of charge, to any person obtaining a copy of this software and associated
234+
documentation files (the "Software"), to deal in the Software without restriction, including without limitation the
235+
rights to use, copy, modify, merge, publish, distribute, sublicense, and/or sell copies of the Software, and to permit
236+
persons to whom the Software is furnished to do so, subject to the following conditions:
237+
238+
The above copyright notice and this permission notice shall be included in all copies or substantial portions of the
239+
Software.
240+
241+
THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE
242+
WARRANTIES OF MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR
243+
COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR
244+
OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE.

src/QAToolKit.Engine.Database.Test/DatabaseTestGeneratorOptionsTests.cs

Lines changed: 0 additions & 107 deletions
This file was deleted.

0 commit comments

Comments
 (0)