Data not accurate

[kirschem-fernride]

When comparing the data from the aws-cost-exporter to the Cost Explorer, there are some discrepancies.

I guess this is due to the fact, that the cost exporter always queries for the day before, on a given day. Let's say TODAY it queries for YESTERDAY.

The billing cycle for YESTERDAY ends at 23:59 UTC that day. It may take up to 24 hours for AWS to update the costs
(Source).

That means, the earliest point in time to get accurate data for YESTERDAY is TODAY at 23:59 UTC. If we don't poll at exactly this time, we either miss some data (if we poll earlier) or skip already to the next day (if we poll later).

Thus, I'd propose to change the logic (or at least make it configurable), to poll for TODAY - 2d instead of TODAY -1d.
Another option would be to enable a cron-based schedule instead of polling. This would be even more fitting given the AWS updating constraints outlined above.

[gespi1]

i believe its on purpose. #27 (comment)

[kirschem-fernride]

Currently, it's dependent on the timing of the polling. I'm pretty sure this could be improved by using a fixed cron schedule to poll at the correct time, which we know (24h after end of billing cycle)

[gluckzhang]

Hi @kirschem-fernride, thank you for pointing this out. In our case, we might be lucky, or AWS might be conservative about updating cost data. It seems fine to fetch costs every 6 hours in our previous experiments.

But yes, I agree it works better if the exporter can fetch the data at several UTC timepoints like 06:00, 12:00, ..., 23:59 so the costs for the previous day can be regularly updated, given that AWS billing cycle is based on UTC time. Configuring the target date for a query is also a good idea, though I am not sure if users tend to have TODAY-2d as it is a bit outdated (e.g., a high-cost alert comes after 2 days).

I will check how to implement the cron-based schedule. At the same time, you are more than welcome to make a PR for this. Thanks :)