Is it possible to use the Gemini API instead of local models?

[NightMachinery]

Gemini provides a very generous free tier, and its models are excellent. Is it possible to use them instead of a local model?

The title model in particular seems to need a hack?

[rafjaf]

@NightMachinery FYI I had a go at it in a different repo: https://github.com/rafjaf/summarize

I hope it can be of use.

[NightMachinery]

@rafjaf Thanks, feel free to close this one. That repo looks promising.

[NightMachinery]

@rafjaf I tested that repo, and here are my initial observations:

• It needs to group short chapters together. (Because some detected chapters are things like the cover etc., and these make the model hallucinate, which will pollute everything until the end.)
• Retry and wait on errors (esp. quota errors)
• Add a cli arg --context-chapters=3, perhaps add a context mode where we summarize the current context ("compact" it) to always have a summary that starts from the beginning. Though with how big the context window of gemini is, I guess we could just put all the previous chapters into it.
• add --language to explicitly set the language of the summarization (mistakenly thought an English book was in spanish; probably because of the first short chapter issue above)
• It doesn't support HTTP_PROXY etc.

I might try solving some of these myself and sending PRs if I find the time later.

Thanks for your work!

[NightMachinery]

I'll reopen this issue as having this mature project support Gemini seems worthwhile.