bulk-chain-1.1.0

Release Notes

In this release we address on API customization for agents deployment using third-party services and for third-party apps.

Key release features:

• Introduction of the new API interface for inference.
  • Results in updated API for providers: infer_mode.
• LLM attempts is no longer a provider parameter but API inference setup.
  • Support launch async inference of a batch for LLM providers
• Example for Replicate IO.
  • Event-loop access and setup.
• Adapation to the third-party UI / frameworks like fastAPI (asyncio.get_running_loop()), tkInter, etc.

from bulk_chain.core.utils import dynamic_init
from bulk_chain.api import iter_content

content_it = iter_content(
    ... 
    # New parameters
    infer_mode="batch_async", 
    return_mode="batch",
    ...
)
    
for content in content_it:
    # Handle your LLM responses here ...

Batch streaming

Changes

• 🐛 Make chunk index global #96
• 🔧 Callback mechanism for streaming mode is no longer needed #85
• ✨ handle_missed_value_func: put this default handler into the core #82
• 🔥 work with async streams #84
• 🔧 Remove ask_core method #93
• 🔧 Move attempts into iter_content API #92
• 🔥 Support async streaming mode by queueing the output #95
• Add API usage in the readme section #89
• 🔥Simplify iterative modes #91

Minor

• dynamic_init remove class_dir parameter. #90
• ❌ remove dynamic and adapt passing the direct filepath to provider #88
• ❌ Drop the support for name method at BaseLM #94

Full Changelog: 1.0.0...1.1.0