order of messages when sending multipile messages

[boynet]

just to make sure as I have few bugs with the models, is the messages array first element is the oldest one in the conversation?

like:
user: hello
agent: hi do you need help
user: yes I need help

when sending it to open router I send it like were 0 is the oldest message? (simplified of course)
$messages = [0=>'user: hello', 1=>'agent: hi do you need help', 2=>'user: yes I need help']

[boynet]

I think that some models like google/gemini-2.5-flash-preview-05-20:thinking cant see multiple messages, they combine them internally into one.
so with this system message: 'you have simple rules: return the first message you got in this conversation. '
and send this messages:
{
"role": "user",
"content": "blue blue"
},
{
"role": "user",
"content": "red red"
},
gemini return: blue blue red red

while gpt4.1 return the correct answer: blue blue

[moe-mizrak]

The maintaining the chat contiunity (historical messages) may vary for different models (It has nothing to do with OpenRouter). OpenRouter is mainly follows the best practices of well-known LLM providers - OpenAi.

This is basically how chat contiunity works - you send you previous conversation along with the new one:

$model = 'mistralai/mistral-7b-instruct:free';

$firstMessage = new MessageData(
    role: RoleType::USER,
    content: 'Is it going to rain today?'
);

$chatData = new ChatData(
    messages: [
        $firstMessage,
    ],
    model: $model,
);
// This is the chat which you want LLM to remember
$oldResponse = LaravelOpenRouter::chatRequest($chatData); // e.g. LLM responds as: 'No, it is not rainny today'

/*
* You can skip part above and just create your historical message below (maybe you retrieve historical messages from DB etc.)
*/

// Here adding historical response to new message
$historicalMessage = new MessageData(
    role: RoleType::ASSISTANT, // Set as assistant since it is a historical message retrieved previously
    content: Arr::get($oldResponse->choices[0], 'message.content'), // Historical response content retrieved from previous chat request
);
// This is your new message
$newMessage = new MessageData(
    role: RoleType::USER,
    content: 'So I should take my umbrella with me?',
);

$chatData = new ChatData(
    messages: [
        $firstMessage, // this is what we asked
        $historicalMessage, // this is what LLM responded
        $newMessage, // this is our new message
    ],
    model: $model,
);

$response = LaravelOpenRouter::chatRequest($chatData);

Without this, you can manipulate the chat contiunity with prompt-engineering as you do (red red blue blue one).

But generally speaking, all LLMs try to be coherent - follow the best practices of well know one (meaning that they generally handles things same/similar to OpenAi).

• If you are still having issue, try different LLM
• Try messages that make sense to keep chat contiunity so LLM will not ignore it
• There can be some limit for remembering historical chat for LLMs so at some point it can ignore/forget the old ones. So aiming for very long conversation history might not be good idea and it increases cost (sending all messages increases cost) and might effect the response quality
• If you are aiming for such long historical chat / chat contiunity then coming up different approaches might be beneficial such as using LLM to summarize the historical chat and sending the summarized message along with the new one etc.

Anyway, hopefully you figure it out, you are welcome to ask it here, test different scenarios and so on.

[boynet]

Thanks a lot! So, the messages array is sorted from oldest to newest.
I believe the issue is with Gemini, which merges multiple consecutive messages from the same role into a single response when there is no separation between them.
For example, given the following system instruction:
"Return the very first message you received in this conversation. It's important to return only the first message, no matter what."

And the following messages:

[
  { "role": "user", "content": "blue blue" },
  { "role": "user", "content": "red red" }
]

Gemini incorrectly returns:
"blue blue red red" (instead of just the first message)
However, if I send messages with an explicit separation:

[
  { "role": "user", "content": "blue blue" },
  { "role": "assistant", "content": "" },
  { "role": "user", "content": "red red" }
]

Gemini correctly returns:
"blue blue" (as expected)

[moe-mizrak]

correct, it goes like old message -> old response -> new message.
Also check open ai doc for better understanding: https://platform.openai.com/docs/guides/conversation-state?lang=javascript

Since OpenRouter is a unified interface for multiple LLMs, do it as in my sample because it might be slightly different for open ai.

you blu blu red red example, it might be possible that without "assistant", it does not understand them as historical messages, but as same message divided. so providing the LLM response in "assistant" might be working for that reason.

[boynet]

I saw your example but what about when you have this type of history:
system, user_message, user_message, user_message

when its not simply user->ASSISTANT->user->ASSISTANT
and you need the ai to be able to read each message separately to understand context
so I am not sure how to apply it to my case

[moe-mizrak]

I dont know, kind of an edge use-case; but as you alread did, sending empty message for assistant, or a dummy message like "ok, i am waiting for your next command" (maybe the first 'system' requires LLM to listen until the message includes "listen until i tell you to answer" ...); so basically best way to try and find the most doable method I think.