Decompressing input request

[ramkrishs]

Hello Everyone,

We are trying to use BentoML for our model inferencing while doing so when we send compressed JSON request, decompression fails in decompress_gzip_request method. When I debug further when the request body reaches the decompress method the binary wasn't a gzipped file anymore (it was transformed some step before). Due to that decompression fails with error msg "not a gzipped binary"

we are using before the function to parse JsonInput

@bentoml.api( input=JsonInput(), output=JsonOutput(), route='score' )

Can someone help me with debugging this issue. ? What is the right way to input the deserialization method should I pass in?

[parano]

Hi @ramkrishs could you share your HTTP client side code?

[ramkrishs]

Hi Parano,

Let's take this example, where we send gzipped JSON request that returns the back the same request. I am using BentoML == 0.13.1

from bentoml.adapters import JsonInput, JsonOutput
from bentoml.service.artifacts.pickle import PickleArtifact


@bentoml.artifacts([PickleArtifact('model')])
@bentoml.env(conda_env_yml_file='./environment.yml')
class PredictService(bentoml.BentoService):

    def __init__(self):
        super().__init__()

    @bentoml.api(
        input=JsonInput(),
        output=JsonOutput(),
        route='score'
    )
    def predict_with__json(self, parsed_json: dict):
        """
        End point with Json input.
        """
        return parsed_json 

[parano]

Could you share how did you send the request? e.g. the CURL command or code that sends the request? Did you specify the content-encoding header?

And could you share the full error message that you saw on the server logs?

[ramkrishs]

curl --location --request POST 'https://localhost:8080/score' \
--header 'Content-Encoding: gzip' \
--header 'Content-Type: application/json' \
--data-binary '~/Downloads/example-request.json.gz'

I was thrown OSError with Gzip decompression error. With Debugging the error by adding breakpoints when the message reaches the req.body = gzip.decompress(req.body) line req.body isn't in gzipped format anymore. Someplace in the stack trace, it was modified hence I got OSError with the exception of saying not a gzipped file.

[ramkrishs]

Am I missing anything with respect to configuration / request?