Statistical logic of indicators

[Echo-C-Yi]

I want get some help.

Q1：Isn't metrics aggregation affected by sample rate? I invoke a method 10 times in 1 minute, but the sample rate is 1/10000. Should the CPM be 10 or 0? If sample rate doesn't affect metrics aggregation, please show me where the metrics aggregation logic is.

Q2: Do you guys has a plan to support min time sample policy? Like in 1 second at least take at least 1 sample.

Q3: If i wanna use about 3 machine to build oap-cluster to support billion level of data, how large spec do you suggest? 16C64G will be fine?

Q4: If sample rate will affect metrics aggregate, how large spec can support 100% sample rate with billion data level? And any suggestion about accurate statistics of metrics like CPM?

[wu-sheng]

Re,
Q1, If you are applying agent-side sampling, then yes, the statistics got impacted. Because the client sampler would not forward traces to the OAP server. There is nothing to be analyzed. If you are using the OAP side sampling rate, then it doesn't impact the statistics.

Q2, resetting in 1s could have a huge impact on the CPU. You could implement a new sampling policy through a plugin. We don't have one in upstream, but it is not hard to do by yourself.

Q3, 3 billion is huge, I am highly doubting that 3 machines could support that. Usually, 10-15 high-spec machines for OAP and another 30 nodes for elastic may support a billion-level deployment. 32U64G spec is very common for high-payload deployment. We are trying to have BanyanDB 1.0, which should require much less than Elastic. But BanyanDB is a new thing, it is only suitable for people willing adopt new things.

Q4 has been covered by the answer to Q1.

[wu-sheng]

I think you could find it in topology? Metrics on he lines. There are two, one is on client side, the other for server side.

[Echo-C-Yi]

Yes，i got your point.
Topology can provide server side and client side metrics.
But i wanna implement one endpoint can find all the relation metrics.
I will figure it by myself.
Thank you.

[wu-sheng]

That doesn't make any sense. You have to provide both sides of a calling to get the metric.
From the client side, that is not an endpoint at all.

[Echo-C-Yi]

Yes, it's not about endpoint thing.
For example:
dubbo-provider1 has about 10 different consumer.
dubbo-consumer1 consume dubbo-provider1 and http endpoint is the entry.

I wanna know the performance of dubbo-consumer1 to dubbo-provider1 with this http endpoint but not the dubbo-provider1 performance.

Dubbo-provider1 performance will include all the 10 consumer performance then make it confuse in my situation.

[wu-sheng]

If performance per consumer, then it is metrics of service topo.