v1.3.0

[ggerganov]

Overview

This release should be considered in Beta stage, since I haven't done a lot of testing and I am not sure if I didn't break something.
But overall, I believe both the performance and the quality are improved.

• Added Core ML support Core ML support #566
• Restored decoding fallbacks with default size of 2 instead of 5 (f19e23f)
• Pad the audio with zeros instead of the spectrogram (5108b30)
• Added talk-llama example
• Added whisper_state which allows parallel transcriptions with a single model in memory (Added whisper state + default state on the whisper_context #523)

The C-style API has been extended significantly to support the new whisper_state, bit in general, should be backwards compatible.
The only breaking change is in the callbacks signatures.

Please provide feedback in the discussion if you observe any issues.

What's Changed

• update csv output format to match OpenAI's Whisper dataframe output by @hykelvinlee42 in update csv output format to match OpenAI's Whisper dataframe output #552
• Go binding: NewContext now returns a clean context by @polarmoon in Go binding: NewContext now returns a clean context #537
• Added whisper state + default state on the whisper_context by @sandrohanea in Added whisper state + default state on the whisper_context #523
• whisper.android: Enable fp16 instrinsics (FP16_VA) which is supported by ARMv8.2 or later. by @tinoue in whisper.android: Enable fp16 instrinsics (FP16_VA) which is supported by ARMv8.2 or later. #572
• Add quality comparison helper by @venkr in Add quality comparison helper #569
• whisper.android: Support benchmark for Android example. by @tinoue in whisper.android: Support benchmark for Android example. #542
• Fix MUSL Linux build by @ggerganov in Fix MUSL Linux build #576
• Change default encoding to UTF-8 by @Kamilake in Change default encoding to UTF-8 #605
• Provide option for creating JSON output by @tuxpoldo in Provide option for creating JSON output #615
• readme : add react-native bindings by @jhen0409 in readme : add react-native bindings #619
• Fixed language auto-detection for state provided processing. by @sandrohanea in Fixed language auto-detection for state provided processing. #627
• xcodeproj : add -O3 -DNDEBUG in release mode by @jhen0409 in xcodeproj : add -O3 -DNDEBUG in release mode #640
• Nodejs Addon blocking main thread. Implemented Napi::AsyncWorker by @LucasZNK in Nodejs Addon blocking main thread. Implemented Napi::AsyncWorker #642
• Include link to R wrapper in README by @jwijffels in Include link to R wrapper in README #626
• Add a cmake flag to disable F16C by @a5huynh in Add a cmake flag to disable F16C #628
• Add talk-llama example by @ggerganov in Add talk-llama example #664
• Add Alpaca support to talk-llama example by @ejones in Add Alpaca support to talk-llama example #668
• Update README.md by @razodactyl in Update README.md #682
• issue armv7l crashes, out of box Illegal instruction, with tweaking Killed #470 - working 32-bit ARM by @clach04 in issue #470 - working 32-bit ARM #486
• whisper : add initial_prompt param by @jhen0409 in whisper : add initial_prompt param #645
• fix typo in JSON output by @egorFiNE in fix typo in JSON output #648
• Fix shell script ./models/download-ggml-model.sh to handle spaces and special characters in paths by @be-next in Fix shell script ./models/download-ggml-model.sh to handle spaces and special characters in paths #677
• Fixed test to new async implementation by @LucasZNK in Fixed test to new async implementation #686
• Minor: fixing usage message for talk-llama by @InconsolableCellist in Minor: fixing usage message for talk-llama #687
• Small typo by @ZiggerZZ in Small typo #688
• feat: add progress callback by @pajowu in feat: add progress callback #600
• ggml : fix q4_1 dot product types by @novag in ggml : fix q4_1 dot product types #759
• Exposed various parts to the Go Interface by @bmurray in Exposed various parts to the Go Interface #697
• Adds shell command example for --print-colors by @bocytko in Adds shell command example for --print-colors #710
• Makefile: disable avx in case f16c is not available by @duthils in Makefile: disable avx in case f16c is not available #706
• Making the quick start instructions clearer. by @Onlyartist9 in Making the quick start instructions clearer. #716
• Add lrc output support by @WhichWho in Add lrc output support #718
• Corrects default speak.sh path in talk-llama by @mab122 in Corrects default speak.sh path in talk-llama #720
• Add msvc compiler args /utf-8 fix error C3688 by @WhichWho in Add msvc compiler args /utf-8 fix error C3688 #721
• Changed convert-pt-to-ggml.py to use .tiktoken tokenizer files by @ivan-gorin in Changed convert-pt-to-ggml.py to use .tiktoken tokenizer files #725
• talk/talk-llama: add basic example script for eleven-labs tts by @DGdev91 in talk/talk-llama: add basic example script for eleven-labs tts #728
• readme : add Unity3d bindings by @Macoron in readme : add Unity3d bindings #733
• Update stream.cpp by @AliAlameh in Update stream.cpp #501
• Fix typos in whisper.h by @GitAritron in Fix typos in whisper.h #737
• Update LICENSE by @masguit42 in Update LICENSE #739
• fix potential memory leaks by @baderouaich in fix potential memory leaks #740
• readme: Add alternate swift bindings by @exPHAT in readme: Add alternate swift bindings #755
• Fix the bug related to word splitting errors in the "tokenize" function. by @AfryMask in Fix the bug related to word splitting errors in the "tokenize" function. #760
• Do not launch threads for log_mel_spectrogram when singlethreaded by @maxilevi in Do not launch threads for log_mel_spectrogram when singlethreaded #763
• Core ML support by @ggerganov in Core ML support #566
• ggml : fix build on whisper.android (ARM_NEON) by @jhen0409 in ggml : fix build on whisper.android (ARM_NEON) #764

New Contributors

• @hykelvinlee42 made their first contribution in update csv output format to match OpenAI's Whisper dataframe output #552
• @tinoue made their first contribution in whisper.android: Enable fp16 instrinsics (FP16_VA) which is supported by ARMv8.2 or later. #572
• @venkr made their first contribution in Add quality comparison helper #569
• @Kamilake made their first contribution in Change default encoding to UTF-8 #605
• @tuxpoldo made their first contribution in Provide option for creating JSON output #615
• @jhen0409 made their first contribution in readme : add react-native bindings #619
• @LucasZNK made their first contribution in Nodejs Addon blocking main thread. Implemented Napi::AsyncWorker #642
• @jwijffels made their first contribution in Include link to R wrapper in README #626
• @a5huynh made their first contribution in Add a cmake flag to disable F16C #628
• @ejones made their first contribution in Add Alpaca support to talk-llama example #668
• @razodactyl made their first contribution in Update README.md #682
• @clach04 made their first contribution in issue #470 - working 32-bit ARM #486
• @egorFiNE made their first contribution in fix typo in JSON output #648
• @be-next made their first contribution in Fix shell script ./models/download-ggml-model.sh to handle spaces and special characters in paths #677
• @InconsolableCellist made their first contribution in Minor: fixing usage message for talk-llama #687
• @ZiggerZZ made their first contribution in Small typo #688
• @pajowu made their first contribution in feat: add progress callback #600
• @novag made their first contribution in ggml : fix q4_1 dot product types #759
• @bmurray made their first contribution in Exposed various parts to the Go Interface #697
• @bocytko made their first contribution in Adds shell command example for --print-colors #710
• @duthils made their first contribution in Makefile: disable avx in case f16c is not available #706
• @Onlyartist9 made their first contribution in Making the quick start instructions clearer. #716
• @WhichWho made their first contribution in Add lrc output support #718
• @mab122 made their first contribution in Corrects default speak.sh path in talk-llama #720
• @ivan-gorin made their first contribution in Changed convert-pt-to-ggml.py to use .tiktoken tokenizer files #725
• @DGdev91 made their first contribution in talk/talk-llama: add basic example script for eleven-labs tts #728
• @Macoron made their first contribution in readme : add Unity3d bindings #733
• @AliAlameh made their first contribution in Update stream.cpp #501
• @GitAritron made their first contribution in Fix typos in whisper.h #737
• @masguit42 made their first contribution in Update LICENSE #739
• @baderouaich made their first contribution in fix potential memory leaks #740
• @exPHAT made their first contribution in readme: Add alternate swift bindings #755
• @AfryMask made their first contribution in Fix the bug related to word splitting errors in the "tokenize" function. #760
• @maxilevi made their first contribution in Do not launch threads for log_mel_spectrogram when singlethreaded #763

Full Changelog: v1.2.1...v1.3.0

---

This discussion was created from the release v1.3.0.