Does Transforms support to get Data from APIs and do some business logic in vector?

[qwqmobile]

There is a requirement in which source is SQS, sink is redis, but Transforms needs to get some data from APIs and merge these data from APIs and source through some business logics.
In vector, could Transforms support the requirement? Do you have the plan to support it?

[spencergilbert]

👋

This isn't currently well supported. remap with enrichment_tables can enrich events based on a static CSV file today, and with the lua transform you can make external HTTP requests and use the results - there is a definite performance impact with that, of course.

We've been hesitant to add I/O heavy functionality to transforms to avoid performance degradation and hard to debug issues, but if I'd open a feature request issue with your use case and any details you can provide.

[qwqmobile]

Thanks very much for your quick reply. In our use case, Redis client has been created to read and write data and monitor the redis status, so we want to get redis data through APIs(client) and merge these data with source from SQS through some business logics in transforms, then the result can be written into redis or other storage through APIs in transform or sink operation.