How does it perform with multiple documents about the same subject?

[ph0nky-town]

Has anyone tried to check how it performs when uploading multiple documents related to the same subject?
How does it select the best answer? Will it always cite the first document it finds with an answer?

[amsheldon]

It didn't do well on a year's worth of meeting minutes that have a similar structure in each of the 26 documents. If asked for a detailed summary of the year's meetings it would usually provide a summary of a just few meetings at random. I've tried the included model as well as several Hermes, Mistral, and a newer Mixtral model. You can get some usable information out of it if you specify a single document, but some of the models had trouble and gave an error that there was no context provided. The included model has done a decent job of providing information on loosely related documents so I think its utility depends on what you are putting in it. I've had far more luck using it as a repository for tech documents and procedures than for other more office admin-like tasks. I'd say give it a try and see what you think.