How fast should I expect this to be?

[brandostrong]

Using the default settings and the cogvlm script, processing is quite slow. It's running on my 4090,taking up about 14.4gb, and taking about 10 seconds per image.

[ProGamerGov]

@brandostrong Unfortunately you are limited by the VLM model being used. For the original CogVLM model your speed seems similar to what I got on a 3090, which was 8-9 seconds per image.