v0.0.0beta18

What's Changed

• bump the llm engine pypi version by @ian-scale in #294
• Add A100e GPU type by @yunfeng-scale in #299
• bump client version by @yunfeng-scale in #300
• Add repetition_penalty, top_k, top_p to Completion by @francesy-scale in #295
• Smartly check safetensors vs. bin by @sam-scale in #296
• adding s3 session function instead of client function by @ian-scale in #302

New Contributors

• @francesy-scale made their first contribution in #295
• @sam-scale made their first contribution in #296

Full Changelog: v0.0.0beta15...v0.0.0beta18

v0.0.0beta17

What's Changed

• bump the llm engine pypi version by @ian-scale in #294
• Add A100e GPU type by @yunfeng-scale in #299
• bump client version by @yunfeng-scale in #300
• Add repetition_penalty, top_k, top_p to Completion by @francesy-scale in #295
• Smartly check safetensors vs. bin by @sam-scale in #296
• adding s3 session function instead of client function by @ian-scale in #302

New Contributors

• @francesy-scale made their first contribution in #295
• @sam-scale made their first contribution in #296

Full Changelog: v0.0.0beta15...v0.0.0beta17

v0.0.0b16

This release fixes a bug in the naming conventions of the last version release.

v0.0.0b15

What's Changed

• bump pypi version by @ian-scale in #284
• Ianmacleod/error handling by @ian-scale in #282
• Ianmacleod/add datadog tracing in training by @ian-scale in #287
• Add retries for flaky integration tests by @squeakymouse in #289
• Ianmacleod/pypi version nudge by @ian-scale in #290
• Propagate extra server args to the gunicorn command by @squeakymouse in #291
• Support AWQ for vLLM by @yunfeng-scale in #292
• Update issue templates by @ian-scale in #288
• Ian GitHub templates by @ian-scale in #293

Full Changelog: v0.0.0beta14...v0.0.0beta15

v0.0.0b14

What's Changed

• Add comments by @yunfeng-scale in #250
• Fix vllm docker tensor paralllel by @yunfeng-scale in #254
• Increase liveness timeout for main container by @yunfeng-scale in #255
• Mark batch jobs as not safe to evict by @saiatmakuri in #256
• removing timezone tzinfo in favor of utc by @ian-scale in #257
• invalid CSV input returns InvalidRequestException by @ian-scale in #258
• bumping image tag by @ian-scale in #262
• Integrate LightLLM by @yunfeng-scale in #273
• removing datadog interfaces logging by @ian-scale in #275
• Ianmacleod/vllm by default by @ian-scale in #274
• update docs by @ian-scale in #276
• Ianmacleod/add model delete by @ian-scale in #261
• fixing cacher, tested in prod version of cacher deployment in k8s and… by @ian-scale in #278
• Ianmacleod/fix cacher by @ian-scale in #279
• add vllm to inference framework enum by @ian-scale in #280
• Ianmacleod/update client enum with lightllm by @ian-scale in #281
• Some fixes for endpoints by @yunfeng-scale in #283
• Add actual pytests to integration tests by @phil-scale in #227

Full Changelog: v0.0.0beta13...v0.0.0beta14

v0.0.0beta13

What's Changed

• Update README.md by @saiatmakuri in #236
• Ianmacleod/fix download artifact gateway by @ian-scale in #237
• add peft config documentation by @saiatmakuri in #238
• Update client completion timeout by @seanshi-scale in #239
• Add nvidia.com/gpu in requests by @yunfeng-scale in #240
• Add new image to image cache by @seanshi-scale in #242
• Remove plugins from endpoint containers by @song-william in #241
• Add vLLM as an inference framework by @yunfeng-scale in #228
• Change max_input_length to half of max_total_tokens to work around potential tokenizer loading issue by @seanshi-scale in #244
• Validate Fine-tuning CSV headers by @saiatmakuri in #243
• Sync scale from zero part 2 by @seanshi-scale in #230
• Completions for vLLM endpoints by @yunfeng-scale in #245
• Download bin files for TGI also by @yunfeng-scale in #247
• update team label for fine-tunes by @saiatmakuri in #246
• Ianmacleod/completion sync error throws 4xx by @ian-scale in #234
• Some fixes by @yunfeng-scale in #248
• Higher concurrency limit for gunicorn by @yunfeng-scale in #249
• Pass labels to job config by @saiatmakuri in #251
• Bump python client version from 0.0.0beta12 to 0.0.0beta13 by @seanshi-scale in #253

Full Changelog: v0.0.0.beta12...v0.0.0beta13

v0.0.0.beta12

What's Changed

• Enhancements to hosting docs. by @acmatscale in #124
• Fix CI run_unit_tests_server by @ruizehung-scale in #123
• Updates to cookbook by @seanshi-scale in #127
• Update README.md by @rkaplan in #126
• Play With It section in self hosting doc by @ruizehung-scale in #128
• Documentation update for llama-v2 by @yunfeng-scale in #129
• Update cookbook for llamav2 by @seanshi-scale in #130
• Doc enhancements to use llama-2 by @acmatscale in #131
• Clarified current self-hosting features. by @acmatscale in #134
• Add more models to model zoo by @yunfeng-scale in #135
• Add back Model.create by @yunfeng-scale in #125
• Update allowed types on hyperparameter values by @seanshi-scale in #140
• Can create LLM endpoints by @ruizehung-scale in #132
• Refetch API key dynamically by @jihan-yin in #142
• Add fine_tuned_model field to Get/ListFineTuneResponse by @seanshi-scale in #145
• Update fine_tuning.md by @gargutsav in #147
• Bump pip package version from 0.0.0.beta3 -> 0.0.0.beta4 by @song-william in #154
• Example notebook for fine-tuning Llama-2 7B on ScienceQA by @jihan-yin in #148
• Bump version to 0.0.0b5 by @seanshi-scale in #158
• Add llm endpoint creation and inference sample code to self hosting d… by @ruizehung-scale in #153
• Patch Integration Test in CI by @song-william in #161
• updating fine_tune_id to id by @ian-scale in #174
• Update unit test instruction by @ruizehung in #176
• Add llm-engine suffix to Spellbook URL by @squeakymouse in #173
• adding required dependency installs to scienceqa by @ian-scale in #177
• fix fine_tune_id to be id in scienceqa example by @ian-scale in #179
• Bump version to 0.0.0.beta7 by @squeakymouse in #178
• updating api key settings so that we can set api key without environm… by @ian-scale in #180
• Correct A100 tag by @yunfeng-scale in #183
• Support checkpoint_path for endpoint creation by @yunfeng-scale in #181
• Fix typo in roles.py by @eltociear in #150
• Bump aiohttp from 3.8.4 to 3.8.5 in /clients/python by @dependabot in #151
• Add llama 2 70B in model zoo by @yunfeng-scale in #185
• Add CNAME to docs by @phil-scale in #191
• File API functions by @squeakymouse in #160
• Fix File documentation by @squeakymouse in #192
• Deploy docs from CI by @phil-scale in #190
• Some improvements to completions APIs by @yunfeng-scale in #194
• bump beta8 -> beta9 by @song-william in #195
• Link to HF greedy search by @yunfeng-scale in #198
• Integrate finetune with wandb by @yunfeng-scale in #199
• Bump version by @yunfeng-scale in #201
• Add documentation on pointing llmengine client to self-hosted infrastructure by @ruizehung-scale in #200
• adding status field to model get response by @ian-scale in #202
• Add integrations doc page by @yunfeng-scale in #203
• Update docs to reflect maximum suffix length by @seanshi-scale in #207
• adding download api to launch client, updating example by @ian-scale in #196
• Ianmacleod/update download docs by @ian-scale in #210
• Faster s5cmd download by @yunfeng-scale in #212
• Latest model-engine updates by @phil-scale in #214
• Bump aiohttp from 3.8.4 to 3.8.5 in /model-engine by @dependabot in #217
• Bump waitress from 2.0.0 to 2.1.2 in /model-engine/model_engine_server/inference by @dependabot in #215
• Bump certifi from 2023.5.7 to 2023.7.22 in /clients/python by @dependabot in #216
• Bump certifi from 2023.5.7 to 2023.7.22 in /model-engine by @dependabot in #218
• Add readme to model-engine folder by @phil-scale in #220
• Add pre-commit hooks for mypy, semgrep, and trufflehog by @phil-scale in #219
• Fix docs building by @phil-scale in #223
• Update service template by @yunfeng-scale in #222
• Ensure successful helm installation in integration test by @phil-scale in #224
• Update helm charts by @phil-scale in #226
• Fix API calls on Windows by @saiatmakuri in #225
• Sync scale from zero, part 1 by @seanshi-scale in #229
• Doc add fine tune support for llama 2 70b by @yunfeng-scale in #232
• add file upload guidance to fine-tune docs by @saiatmakuri in #231
• Update LICENSE by @saiatmakuri in #235

New Contributors

• @acmatscale made their first contribution in #124
• @jihan-yin made their first contribution in #142
• @ian-scale made their first contribution in #174
• @ruizehung made their first contribution in #176
• @eltociear made their first contribution in #150
• @saiatmakuri made their first contribution in #225

Full Changelog: https://github.com/scaleapi/llm-engine/commits/v0.0.0.beta12