Log request without blocking api response

[thomasaarholt]

I want to log requests and responses, to a database in order to track usage and what the response is. My logging function is slow, taking over a second to upload to a database.

Is there a way I can pass the request and response to a function without blocking the response being sent?

Stealing the bentoml readme example and modifying slightly:

from time import sleep

...

def slow_logging_function(request, response):
    sleep(secs=10)

@svc.api(input=Image(), output=NumpyNdarray(dtype="int64"))
def predict(input_img: PILImage):
    img_arr = np.array(input_img)/255.0
    input_arr = np.expand_dims(img_arr, 0).astype("float32")
    output_tensor = mnist_runner.predict.run(input_arr)

    response = output_tensor.numpy()
    # Slow logging function that would block `return response`
    slow_logging_function(request=input_img, response=response)
    return response

How can I make slow_logging_function non-blocking?

[ssheng]

@thomasaarholt the trade-off is the logging consistency. If returning the response before the log is committed to the database is acceptable, you can consider store the log in memory and run the slow_logging_function as a background task using asyncio. You can even batch the logging to database for better efficiency.