Failed to analyze image (analysis failure reported by model)

[Jaron-Wilson]

I get this error when running via api or the python heres the 2 scripts segments:

# Prepare the image using lmstudio
        with lms.Client() as client:
            # Prepare the image for analysis
            image_handle = client.files.prepare_image(image_path)
            model = client.llm.model("qwen2-vl-2b-instruct")
            chat = lms.Chat()
            chat.add_user_message("Describe this image please", images=[image_handle])
            prediction = model.respond(chat)
            return {"status": "success", "response": prediction.text}

# image_handle = lms.prepare_image(image_path)

        # # Load the model
        # model = lms.llm("qwen2-vl-2b-instruct")

        # # Create a chat instance and add the user message with the image
        # chat = lms.Chat()
        # chat.add_user_message("Describe this image please", images=[image_handle])

        # # Get the model's response
        # prediction = model.respond(chat)

I got these off of the api and python script, so i dont know why it does not work anymore?

[ncoghlan]

Which version of the lmstudio PyPI module is reporting the exception?

The image preparation APIs were only made public in 1.1.0, so 1.0.x raising that exception is expected, but if it's a later version, that would be a very surprising error.

[Jaron-Wilson]

Just reinstalled it and got the same error, it was latest version.

[Jaron-Wilson]

$pip show lmstudio
Name: lmstudio
Version: 1.0.1
Summary: LM Studio Python SDK
Home-page: https://github.com/lmstudio-ai/lmstudio-sdk-python
Author:
Author-email: LM Studio team@lmstudio.ai
License-Expression: MIT
Location: C:\Users\jaron\AppData\Local\Programs\Python\Python312\Lib\site-packages
Requires: httpx, httpx-ws, msgspec, typing-extensions
Required-by:

I will reinstall and check again.

[Jaron-Wilson]

Removed site-packages\lmstudio *, had 3 versions
PS C:\Users\jaron\Downloads\examples> pip show lmstudio
Name: lmstudio
Version: 1.2.0
Summary: LM Studio Python SDK
Home-page: https://github.com/lmstudio-ai/lmstudio-sdk-python
Author:
Author-email: LM Studio team@lmstudio.ai
License-Expression: MIT
Location: C:\Users\jaron\AppData\Local\Programs\Python\Python312\Lib\site-packages
Requires: anyio, exceptiongroup, httpx, httpx-ws, msgspec, typing-extensions
Required-by:

[Jaron-Wilson]

update, the 1.2.0 version gives me this error:
It seems there was an issue analyzing the image. This could be due to a few reasons, such as an internal error or incorrect path provided for the image. Could you please provide more details about the image or try uploading it again?
code:

"""Analyze an image using qwen2-vl-2b-instruct model."""
    try:

        # Check if the image file exists
        # if not os.path.exists(image_path):
        #     return {"status": "error", "message": f"Image file not found: {image_path}"}
        image_path = "C:/Users/jaron/Downloads/examples/images/downloaded_images/snowy_landscape_green_trees_no_dog_1_1.jpg"
        # Prepare the image using lmstudio
        with lms.Client() as client:
            model = client.llm.model("qwen2-vl-2b-instruct")
            image_handle = client.files.prepare_image(image_path) # Replace with the path to your image
            chat = lms.Chat()
            chat.add_user_message("Describe this image please", images=[image_handle])
            prediction = model.respond(chat)
            return {"status": "success", "prediction": prediction}

    except Exception as e:
        return {"status": "error", "message": f"Failed to analyze image: {str(e)}"}

and i did the url as \ and / both did not work:

[ncoghlan]

Is the Qwen model able to analyse that image if it is attached to the same query in an interactive chat in the main app? (if that response is coming from the model rather than the image preparation step, it changes the potential causes of the issue)

[Jaron-Wilson]

I have it attached to the chat, also it's the same query .

[Jaron-Wilson]

I tried gemma-3-4b-it and it still gave me same error, its with the lms client.
image_path = "C:/Users/jaron/Downloads/examples/images/downloaded_images/snowy_landscape_green_trees_no_dog_1_1.jpg"
# Prepare the image using lmstudio
with lms.Client() as client:
model = client.llm.model("gemma-3-4b-it")
image_handle = client.files.prepare_image(image_path) # Replace with the path to your image
chat = lms.Chat()
chat.add_user_message("Describe this image please", images=[image_handle])
prediction = model.respond(chat)
return {"status": "success", "prediction": prediction}

[Jaron-Wilson]

Whenever I use the lms.* anything from there it seems to not work even after downloading the python version.

[Jaron-Wilson]

It looks like qwen does not know how to get images correctly:
curl http://127.0.0.1:1234/v1/chat/completions -H "Content-Type: application/json" -d '{
"model": "",
"messages": [
{
"role": "user",
"content": [
{
"type": "text",
"text": "What is this image?"
},
{
"type": "image_url",
"image_url": { "url": "data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAAABsAAAAbCAYAAACN1PRVAAAAAXNSR0IArs4c6QAAAARnQU1BAACxjwv8YQUAAAAJcEhZcwAADsMAAA7DAcdvqGQAAAQ2SURBVEhL3ZVrTBxVFMf/d2f2vcsCWxZYZKVQW4vVUmmtNpgaSSA1NUWpkFhpWkOwxsZa0WpqjMYXaTQR4wdjSttQ2tDaGIRqa5HWVmwFYgwasRhoeRipi1BWdpd9sDvjndmBZbzgt/ZDf1927jl3zv+ec8+ZJbcvLRBxk9AovzeFW1ds3jtzcQ6YHXcCQR+EsQEQuwvEaAPvGUCKdhK8cRq/BP1wTxKIyttjj78Iw9VuWLrPxQzzwIhtwb1wl+2FL8GGTcfq8U9SMlo3liK7vxdFJz+HzhBC/s52WF0e+ENAxxUeX/fwaPAWgBu/Bu3ffyiRWJgyuktfgc9KhY7XY8psUQnpzQGs3nVBFpIw64HC3AjefyKIrk1nsQxD4ECwf2yd7P8vjNi0XhcTMllw+rFyZA5dlYVMiT6s2X0BloxJZacal13EZzt82K5PRc60RbGqYcpYoy1BxJqM5rIKZAwPYkPzcSQ6xpH3zCUYkgPKrnkIcQi9W4zosA3b7ZfQyU0ojjhMZj5rAk5u3jIrZL9tlJbuu/8XktBGwT/UB1NtE5YUuBWjGkbsYnEJFrn/koUc2SNYVtmOs70RNHcQTHjVjRukDdLWDXzRQbvRB3AP9wGJQTy1NqzsUMOUseSR/ShuaYJz+SAySjvxXJ0G3ilApxURFQg+rooiKxWY9It49hMe417aKAYqTON/WBlFID0Hd4lXsOZtKyamiBI1BpPZhlPHkLmyH/c83YnuYRHjtB+OVkdx6HlB9v/QGwvw86AGI9eBw7sjOPwCFUldjELdAeQF67Ak3Ii0FVnyvrkwmTXlu7Cq7FfqiZXJQ7NKS4r5rtMy8vR4CWaCUFg6CIFzUcxX5K/B3TSjt8xHcSp6Hyp81UirqcDs1FOYzBxFPbKQhIHO0YyQRLKVyELSIU7/RGDUxwN5uQSsNfSjPPw6zCSIMG+GoKP1nQMjNuJhTAx/jgMftXDYWsvLGUq8xh9BRXgvOoXlGBZTYZoagyak7mAmcp87bvommo+sQKOyipPjBHY9KsAXIDjjX4HzQh4smgDOGPagmj+BPeEq2LpalN1xmDvbvDqMD8qC8vOokIQWYR0q+a/k9Qw9Q8DOT3mIJjPIS28o1hgcBDzg/x4N+9gPMiPGa0S0VvuQnaIyqxDppTd8q8GhNg6NL0dU9ypRVW9Ea49WWcVhyhihs/TOl/GLFX530KGinTKH34aJLHSHU0RKgvpQPw5yaLvMCklwiXbXm8rzLANjHDKTosh1CgjXrocm3QuSRqdXIcUmYlWOiCfXC9Dr4oM7FSbYWmei46Ie5hkWbL0DR9Jx8dUHIU6YoMm9plhjEEKwcrH05YgHPd/L4f59SejXuBQLy4JihUEnPKNGlCe342AXT7Nlt0p/nucu89jRYMS2g2aMLC2Ee9t7ipeFaZAbyYK}' "stream": false1,7,AAAElFTkSuQmCC" }
% Total % Received % Xferd Average Speed Time Time Time Current
Dload Upload Total Spent Left Speed
100 2512 100 519 100 1993 58 223 0:00:08 0:00:08 --:--:-- 142{
"id": "chatcmpl-hoelmelo2chi5vk5ucmgef",
"object": "chat.completion",
"created": 1743862694,
"model": "qwen2-vl-2b-instruct",
"choices": [
{
"index": 0,
"logprobs": null,
"finish_reason": "stop",
"message": {
"role": "assistant",
"content": "I'm sorry, but I can't help you with that."
}
}
],
"usage": {
"prompt_tokens": 27,
"completion_tokens": 13,
"total_tokens": 40
},
"stats": {},
"system_fingerprint": "qwen2-vl-2b-instruct"
}

[Jaron-Wilson]

looks like the encoding was not working i think:
https://github.com/YorkieDev/lmstudioservercodeexamples?tab=readme-ov-file#vision-analysis-python
Used that repository to help, it now says for gemma:

Okay, here’s what I see in the image:

*   **A Red Japanese Maple Tree:** The most prominent element is a vibrant red Japanese maple tree with cascading branches. Its leaves are a brilliant shade of crimson.
*   **Green Shrubs:** There are several evergreen shrubs visible in the background, including some that appear to be blue spruce trees.
*   **Other Trees:** There are also some smaller, conical-shaped trees in the background.
*   **Grass and Flowers:** The tree is planted in a grassy area with some yellow flowers scattered around.
*   **Stone Base:** The tree sits on a dark stone base or planting bed.

Do you want me to describe any specific aspect of the image in more detail?```
Qwen still couldnt get it to work.

Python Packages still dont work for lmstudio for some reason, my lmstudio is on latest version the app and the python package. I have to call using the api endpoint.

[ncoghlan]

From a Python prompt, would you be able to run through an example similar to the following and post the details?

My specific example image is the app screenshot from the lmstudio.ai home page:

>>> import lmstudio as lms
>>> image_path = r"E:\Misc\lmstudio-home-page-app-screenshot-20250412.webp"
>>> image_handle = lms.prepare_image(image_path)
>>> image_handle
ChatMessagePartFileData(name='lmstudio-home-page-app-screenshot-20250412.webp', identifier='1744382888136 - 650.webp', size_bytes=220730, file_type='image')
>>> llm = lms.llm("qwen2-vl-2b-instruct")
>>> chat = lms.Chat()
>>> chat.add_user_message("Describe this image please", images=[image_handle])
ChatMessageDataUser(content=[ChatMessagePartTextData(text='Describe this image please'), ChatMessagePartFileData(name='lmstudio-home-page-app-screenshot-20250412.webp', identifier='1744382888136 - 650.webp', size_bytes=220730, file_type='image')])
>>> prediction = llm.respond(chat)
>>> print(prediction)
The image shows the interface of a software development environment, specifically LM Studio 0.3.11. The interface is divided into several sections:

1. **Top Menu**: The top menu includes options such as "Chat," "Chats," and "Settings." There are also tabs for different functionalities like "Logs," "Financial Analysis," and "Log."

2. **Left Sidebar**: This section contains a list of tools and features, including "Code Editor," "Chat," "Chats," and "Settings."

3. **Centered Section**: This is the main area where code is being written or edited. The text editor shows lines of code with comments and syntax highlighting.

4. **Right Sidebar**: This section contains additional tools and features such as "Advanced Configuration," "Draft Model," "System Prompt," and "Specialized Debugging."

5. **Bottom Sidebar**: This section contains a list of options for different functionalities, including "Settings," "Logs," "Financial Analysis," and "Log."

The code editor shows lines of C++ code with comments and syntax highlighting. The text editor is open to the line where the user has typed something. The interface appears to be designed for software development tasks such as coding, debugging, and system administration.

[Jaron-Wilson]

After lots of debugging, which I dont see where they mention the import of ._sdk_models, I got it to work:

import lmstudio as lms
from lmstudio._sdk_models import ChatMessagePartFileData, ChatMessagePartTextData, ChatMessageDataUser

def run_test():
    image_path = r"C:\Users\jaron\OneDrive\Pictures\Screenshots\Screenshot 2025-04-11 130930.png"
    image_handle = lms.prepare_image(image_path)
    image_handle
    ChatMessagePartFileData(name='Screenshot 2025-04-11 130930.png', identifier='1744382888136 - 650.png', size_bytes=220730, file_type='image')
    llm = lms.llm("qwen2-vl-2b-instruct")
    chat = lms.Chat()
    chat.add_user_message("Describe this image please", images=[image_handle])
    ChatMessageDataUser(content=[ChatMessagePartTextData(text='Describe this image please'), ChatMessagePartFileData(name='Screenshot 2025-04-11 130930.png', identifier='1744382888136 - 650.png', size_bytes=220730, file_type='image')])
    prediction = llm.respond(chat)
    print(prediction)


run_test()

C:\Users\jaron\Downloads\examples>python Scripts/imagetest.py
Traceback (most recent call last):
  File "C:\Users\jaron\Downloads\examples\Scripts\imagetest.py", line 15, in <module>
    run_test()
  File "C:\Users\jaron\Downloads\examples\Scripts\imagetest.py", line 6, in run_test
    image_handle = ChatMessagePartFileData(name='Screenshot 2025-04-11 130930.png', identifier='1744382888136 - 650.png', size_bytes=220730, file_type='image')
                   ^^^^^^^^^^^^^^^^^^^^^^^
NameError: name 'ChatMessagePartFileData' is not defined

C:\Users\jaron\Downloads\examples>python Scripts/imagetest.py
Traceback (most recent call last):
  File "C:\Users\jaron\Downloads\examples\Scripts\imagetest.py", line 16, in <module>
    run_test()
  File "C:\Users\jaron\Downloads\examples\Scripts\imagetest.py", line 11, in run_test
    ChatMessageDataUser(content=[ChatMessagePartTextData(text='Describe this image please'), ChatMessagePartFileData(name='Screenshot 2025-04-11 130930.png', identifier='1744382888136 - 650.png', size_bytes=220730, file_type='image')])
    ^^^^^^^^^^^^^^^^^^^
NameError: name 'ChatMessageDataUser' is not defined

C:\Users\jaron\Downloads\examples>python Scripts/imagetest.py
Traceback (most recent call last):
  File "C:\Users\jaron\Downloads\examples\Scripts\imagetest.py", line 17, in <module>
    run_test()
  File "C:\Users\jaron\Downloads\examples\Scripts\imagetest.py", line 12, in run_test
    ChatMessageDataUser(content=[ChatMessagePartTextData(text='Describe this image please'), ChatMessagePartFileData(name='Screenshot 2025-04-11 130930.png', identifier='1744382888136 - 650.png', size_bytes=220730, file_type='image')])
    ^^^^^^^^^^^^^^^^^^^
NameError: name 'ChatMessageDataUser' is not defined

C:\Users\jaron\Downloads\examples>python Scripts/imagetest.py
The image shows a code editor with a dark theme, displaying a message that reads "Failed to analyze image (analysis failure reported by model)". The text is in a different color and font style compared to the rest of the content, indicating it's an error message. Below this message, there are some lines of code written in Python, which appears to be related to image processing or analysis. The code editor has various tabs at the top, including "File", "Edit", "New", "Open", and "Close". There is also a sidebar on the right side with different sections such as "Projects", "Security", "Insights", and "Help".

would anyone be able to show me where this is in the repository for the example it does not show this.
this is what i got from copilot, but when it trys to link it to somewhere it does not do it properly, and sends to a random part of the code with nothing to do with this.

Add the following import statement to your script:
Python
from lmstudio._sdk_models import ChatMessageDataUser```