benchmark testing: Want to know how to do bench mark testing

[nagarajtrivedi]

Hi all, I want to know how to do a bench marking testing of golang rule engine.

I want to test how much time it takes to execute 100, 100, 1000 , 10000 and 20000 rules.

Let me know how it is different from other open source rule engines available and what are its advantages over others.

Expecting an early answer to these queries.

Thanks and Regards

N.M.Trivedi

[jinagamvasubabu]

https://github.com/hyperjumptech/grule-rule-engine/tree/master/examples/benchmark
https://github.com/hyperjumptech/grule-rule-engine/blob/master/docs/Benchmarking_en.md

[nagarajtrivedi]

Hi Vasu, I followed it and executed the command but it is giving error that open 5000_dummy_rules.grl: permission denied
Below is the command I followed and the output I got

/go/grule_engine/grule-rule-engine-master/examples/benchmark$ go test -bench=. -benchmem
--- FAIL: TestSerializationPerformanceOnFile (0.04s)
GRBPerformance_test.go:28: GENERATING DUMMY RULES
GRBPerformance_test.go:32: LOADING GRL
GRBPerformance_test.go:40:
Error Trace: GRBPerformance_test.go:40
Error: Received unexpected error:
open 5000_dummy_rules.grl: permission denied
Test: TestSerializationPerformanceOnFile
GRBPerformance_test.go:43: SAVING BINARY INTO GRB
GRBPerformance_test.go:56: LOADING BINARY FROM GRB
GRBPerformance_test.go:74: -------------------------------------------
GRBPerformance_test.go:75: Load GRL duration : 0 ms
GRBPerformance_test.go:76: Saving GRB duration : 0 ms
GRBPerformance_test.go:77: Load GRB duration : 0 ms
FAIL
exit status 1

[newm4n]

@nagarajtrivedi

The GRBPerformance_test.go test is a short test. It was made short because this test is environment dependent. Running this test outside the development environment may cause it to fail.

You should temporary rename this file to become something that is not a golang source code. (e.g. GRBPerforance_test.tmpgo)
And re-run your benchmark.

[newm4n]

Answering your question...

Let me know how it is different from other open source rule engines available and what are its advantages over others.

I don't know any other "expression based" rule engine that is like Grule in Golang and Drools in Java. If you know any, than I definitely misses it, most probably because they're not golang or java based. The language Im familiar to.

Other, so called "rule engine" made in Golang that I can find, they're actually an expression evaluation engine, where you give a single string expression and the engine evaluate it against a fact and give you the result. Here a list of known open source, so called, rule engines. For golang, you can look for names like Gengine and Yql.

These are all open source project and we, the developer, provide you with a solution as it is, without receiving a single dime. You should do your own home work to find out which of open source products suits your need, and should not expect us to do them for you.

[nagarajtrivedi]

Hi, this w.r.t the clarification you have provided about why the bench mark testing is failing. You have mentioned that
The GRBPerformance_test.go test is a short test. It was made short because this test is environment dependent. Running this test outside the development environment may cause it to fail.
I have executed it within your development environment, basically the entire grule engine software downloaded from the github.
The directory in which I was executing was grule_engine/grule-rule-engine-master
I also tried running the make command make test under the directory grule_engine/grule-rule-engine-master and it gave the same problem saying that open 5000_dummy_rules.grl: permission denied
Let me know if I want to execute 5000 dummy rules then what I should do

Thanks and Regards
Nagaraj Trivedi

[newm4n]

In GRBPerformance_test.go file,
you should make sure the following files should point to an absolute path to a location that can be written and read by the the current user.

grlFile = "5000_dummy_rules.grl"
grbFile = "5000_dummy_rules.grb"

into e.g.

grlFile = "/path/to/canwrite/canread/5000_dummy_rules.grl"
grbFile = "/path/to/canwrite/canread/5000_dummy_rules.grb"

Make sure that /path/to/canwrite/canread is exist.

[newm4n]

And i need to point you out. that the generated rules are NOT EXECUTABLE. this test case is to prove that the GRL can be saved as GRB and loaded back as GRL. It will show the performance of saving and loading rules from binary format. Its not to show the execution performance.

If you seek the execution performance, you should look into benchmark created by @jinagamvasubabu

[jinagamvasubabu]

@nagarajtrivedi
simplest thing to do is comment out GRB performance tests or use absolute path as @newm4n said those are not related to see the performance of the engine.

I suggest you to comment the below GRB performance related tests.

func TestSerializationPerformanceOnFile(t *testing.T) {

and then run this command in benchmark folder to see the performance benchmarks for loading/executing against 100 or 1000 rules.

go test -bench=. -benchmem