grule didn't match correctly some of the rules in high demand service !!

[ManouchehrRasoulli]

Hello, i was using grule in our newly developed service, and we use it as a service for checkin limitations,
we have a scenario which we have near 1200 request/second and grule just works fine and all of the rules matchs in the way which we defined !! but when we increase r/s to 8000 and more it starts to giving wrong answers for our rules, and didn't match some obvius rules which we defined and it have to be matched in any scenario !!! do you have any idea about the correctness of grule rule matching !?

[niallnsec]

The results from the engine should be consistent regardless of your level of concurrency.

Are you able to provide us with a minimal example which demonstrates the issue? There is not much inside of the engine which could be affected by race conditions caused by high concurrency, however, if you are using externally defined variables in your own structs or external functions there could be a race condition in those.

If you aren't able to provide a minimal example, you could try building your service with the -race option to enable the race detector and run it with the 1200 req/sec data flow. This should highlight many cases of possible data races in the code.

[ManouchehrRasoulli]

no goroutins are using same knowledge base, but it dosent cost effecient, and it's time cosuming for building knwoledge base for each request !!

[niallnsec]

Yeah I wouldn't expect you to rebuild the knowledge base for every request, that would be massively resource intensive.

When you are using Grule Engine from multiple goroutines there are three common patterns:

1. You use 1 instance of the rule engine and control access with a sync.Mutex object so that only one goroutine is calling Execute at a given moment. This is the simplest setup but can result in performance issues with massively concurrent workloads.
2. You create a worker pool where each worker has a dedicated instance of the rule engine. This is useful for bound workloads, such as consuming messages from a pool.
3. You create a pool of engine instances and implement a simple acquire/release semantic. This option is most useful when you have an unbound number for requests incoming, such as in an HTTP service.

Whatever approach you take, it is essential that you never call Execute on a single instance of the engine from two goroutines otherwise you will get unexpected results.

[ManouchehrRasoulli]

Yes that's true, I take the second way and it's working, but its using up to 4GB of memory and I think we have to rethink about our design approach, any way thank you for your answers 🙏

[niallnsec]

No problem, glad I could help 😃

[ManouchehrRasoulli]

And that's the results with third approach which is promising but i think we need more replication of this service.