bulk-chain-1.1.0

Release Notes

In this release we address on API customization for agents deployment using third-party services and for third-party apps.

Key release features:

• Introduction of the new API interface for inference.
  • Results in updated API for providers: infer_mode.
• LLM attempts is no longer a provider parameter but API inference setup.
  • Support launch async inference of a batch for LLM providers
• Example for Replicate IO.
  • Event-loop access and setup.
• Adapation to the third-party UI / frameworks like fastAPI (asyncio.get_running_loop()), tkInter, etc.

from bulk_chain.core.utils import dynamic_init
from bulk_chain.api import iter_content

content_it = iter_content(
    ... 
    # New parameters
    infer_mode="batch_async", 
    return_mode="batch",
    ...
)
    
for content in content_it:
    # Handle your LLM responses here ...

Batch streaming

Changes

• 🐛 Make chunk index global #96
• 🔧 Callback mechanism for streaming mode is no longer needed #85
• ✨ handle_missed_value_func: put this default handler into the core #82
• 🔥 work with async streams #84
• 🔧 Remove ask_core method #93
• 🔧 Move attempts into iter_content API #92
• 🔥 Support async streaming mode by queueing the output #95
• Add API usage in the readme section #89
• 🔥Simplify iterative modes #91

Minor

• dynamic_init remove class_dir parameter. #90
• ❌ remove dynamic and adapt passing the direct filepath to provider #88
• ❌ Drop the support for name method at BaseLM #94

Full Changelog: 1.0.0...1.1.0

bulk-chain-1.0.0

The first major release of a no-string API for adapting your LLM for Chain-of-Thought alike reasoning over records with large amount of parameters across large datasets.

📘 What's new [Updates]

• 🔧 Inference API only: #78
• 📺 Shell demo dispatching: #86

Full Changelog: 0.25.3...1.0.0

bulk-chain-0.25.3

📘 What's new [Updates]

The most important fix in this release relates to streaming.
Earlier it worked only in demo mode, while in other cases return generators.
Now this problem solved and you can manually handle streamed data.

NOTICE: This is intended to be the latest 0.x.x stable release. This and earlier series of releases include shell-based client.
📢 We expect big changes further and more on API side is what makes this project follow the concept of no-string dependencies. That highly would turn shell-based client into separated github project aka. DEMO 📺 with switch into major 1.0.0 release. Stand by guys for more further updates. 🌟

Changes

• 🐛 Streaming mode does not support callback and generator processing [Callback API mode support] #75
• 📺 DEMO: ModuleNotFoundError: No module named 'path.to.providers'` #76
• 🔥 Optional caching operation #64
• 🔥 SQLite Caching implementation does not support batching [cmd inference mode] #63
• 📺 Demo: use inference API #77
• 🔥 Handling none-values #80
• 🐛 Empty Dict Test: RuntimeError: dictionary changed size during iteration #79
• Next Release
• 🔥 Setup LLM adapter via (optional) adapters field in JSON schema #38
• 🔥 Optional prompt logging #28

Integration of streaming Clients

This is a replicate-powered CoT via bulk-chain inference engine integration examples for sentiment analysis schema :

1. For single input Shell-based (within the same release:
   

2. For iterable JSON data / other sources: Tk-sheet client demo project:
   

Full Changelog: 0.25.2...0.25.3

bulk-chain-0.25.2

Changeset

Fixed bugs

• 🔥 🐛 ask in batching mode results in multiple calls for ask / LLM inference #73
• 🐛Shell script: attempts arg may cause a bug #57
• Fix bug with reading arg with no param #72

bulk-translate-0.25.1

Changeset

Demo support

NOTE: Demo supports streaming!

python3 -m bulk_chain.demo \
    --schema "test/schema/thor_cot_schema.json" \
    --adapter "dynamic:replicate_104.py:Replicate" \
    %%m \
    --model_name "meta/meta-llama-3-70b-instruct" \
    --api_token "<REPLICATE-API-TOKEN>" \
    --stream

Fixed bugs

🐛 Batching mode -- Using Qwen2 provider results in list output. #58
🔥 Shell: use separated demo.sh #70
✨ Add test dataset for benchmarking 📊 #59
🐋 Set DeepSeek-R1 by default hosted by Replicate #62

Repository organization

Use wget for fetching content from nlp-thirdgate.
❌ Drop replicate.py provider in this repository and use the one from nlp-thirdgate #60
🐛 Batching mode -- Using Qwen2 provider results in list output. #58

Full Changelog: 0.25.0...0.25.1

bulk-chain-0.25.0

Changeset

📘 What's new in 0.25.0

🔥 API support

🔥 Optional Batching support with backport to existed models

Main Changes

• ⚠️ Non-framework based Native Batching Support [non-caching mode] #9
• 🔧 Optional Caching / No Caching for API. #43
• Simplified API usage with schema
• 🔧 Check that added arguments were not mentioned before #47

🔧 Bugs fixed

• 🔧 Prompt limiting should not be a part of inference modes #48
• 🔥 Optional prompt logging #28
• csv and tsv separation #29
• ✨ Default target path -- implement replace extention function #30
• 🔧 BaseLM -- remove naming #31
• Setup LLM adapter via JSON schema #38
• Bash script inference: --output may cause exception in the case when the file type is not SQLite #34
• 🐛 delimiter parameter that is not related to model does not supported by OpenAI. #40
• args.limit is expected to be outside of the iter_content call #45
• 🔥 🐛 data[c] records and duplicated call on parameters update. [DEBUGING] #42
• 🐛 Cache target should not be None for cached inference #50
• 🐛 API: no need for name parameter in schema #51
• Make source-iter as an optional package #53

Minor changes

• 🔥Switch FlanT5 to Replicate API by default #52

Full Changelog: 0.24.2...0.25.0

bulk-chain-0.24.2

📘 The complete list of implemeted issues: #33

• #32 Adapt readers from source_iter project
• #35 Support re-attempting inference after any raised exception during inference
• chat_with_llm logger initializes only when we launch the chat mode.

Full Changelog: 0.24.1...0.24.2

bulk-chain-0.24.1

This release represent bug fixes from the version 0.24.0

Details: #26

Full Changelog: 0.24.0...0.24.1

bulk-chain-0.24.0

This is an initial version of the project:
https://pypi.org/project/bulk-chain/0.24.0/

See more details: #8