You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
This is a collection of utilities to help adopt language models in Swift apps. It tries to follow the Python `transformers` API and abstractions whenever possible, but it also aims to provide an idiomatic Swift interface and does not assume prior familiarity with [`transformers`](https://github.com/huggingface/transformers) or [`tokenizers`](https://github.com/huggingface/tokenizers).
16
+
`swift-transformers` is a collection of utilities to help adopt language models in Swift apps.
7
17
18
+
It tries to follow the Python `transformers` API and abstractions whenever possible, but it also aims to provide an idiomatic Swift interface and does not assume prior familiarity with [`transformers`](https://github.com/huggingface/transformers) or [`tokenizers`](https://github.com/huggingface/tokenizers).
Check out [our announcement post](https://huggingface.co/blog/swift-coreml-llm).
14
24
15
-
-`Tokenizers`. Utilities to convert text to tokens and back. Follows the abstractions in [`tokenizers`](https://github.com/huggingface/tokenizers) and [`transformers.js`](https://github.com/xenova/transformers.js). Usage example:
25
+
## Modules
16
26
27
+
-`Tokenizers`: Utilities to convert text to tokens and back, with support for Chat Templates and Tools. Follows the abstractions in [`tokenizers`](https://github.com/huggingface/tokenizers). Usage example:
17
28
```swift
18
29
importTokenizers
19
-
20
30
functestTokenizer() asyncthrows {
21
-
let tokenizer =tryawait AutoTokenizer.from(pretrained: "pcuenq/Llama-2-7b-chat-coreml")
22
-
let inputIds =tokenizer("Today she took a train to the West")
let tokenizer =tryawait AutoTokenizer.from(pretrained: "deepseek-ai/DeepSeek-R1-Distill-Qwen-7B")
32
+
let messages = [["role":"user", "content":"Describe the Swift programming language."]]
33
+
let encoded =try tokenizer.applyChatTemplate(messages: messages)
34
+
let decoded = tokenizer.decode(tokens: encoded)
24
35
}
25
36
```
26
37
27
-
However, you don't usually need to tokenize the input text yourself - the [`Generation` code](https://github.com/huggingface/swift-transformers/blob/17d4bfae3598482fc7ecf1a621aa77ab586d379a/Sources/Generation/Generation.swift#L82) will take care of it.
38
+
-`Hub`: Utilities for interacting with the Hugging Face Hub! Download models, tokenizers and other config files. Usage example:
39
+
```swift
40
+
importHub
41
+
functestHub() asyncthrows {
42
+
let repo = Hub.Repo(id: "mlx-community/Qwen2.5-0.5B-Instruct-2bit-mlx")
43
+
let filesToDownload = ["config.json", "*.safetensors"]
print("Files downloaded to: \(modelDirectory.path)")
52
+
}
53
+
```
28
54
29
-
-`Hub`. Utilities to download configuration files from the Hub, used to instantiate tokenizers and learn about language model characteristics.
55
+
-`Generation`: Algorithms for text generation. Handles tokenization internally. Currently supported ones are: greedy search, top-k sampling, and top-p sampling.
56
+
-`Models`: Language model abstraction over a Core ML package.
30
57
31
-
-`Generation`. Algorithms for text generation. Currently supported ones are greedy search and top-k sampling.
32
58
33
-
-`Models`. Language model abstraction over a Core ML package.
59
+
## Usage via SwiftPM
34
60
61
+
To use `swift-transformers` with SwiftPM, you can add this to your `Package.swift`:
-[WhisperKit](https://github.com/argmaxinc/WhisperKit): A Swift Package for state-of-the-art speech-to-text systems from [Argmax](https://github.com/argmaxinc)
85
+
-[MLX Swift Examples](https://github.com/ml-explore/mlx-swift-examples): A Swift Package for integrating MLX models in Swift apps.
86
+
87
+
Using `swift-transformers` in your project? Let us know and we'll add you to the list!
35
88
36
89
## Supported Models
37
90
91
+
You can run inference on Core ML models with `swift-transformers`. Note that Core ML is not required to use the `Tokenizers` or `Hub` modules.
92
+
38
93
This package has been tested with autoregressive language models such as:
39
94
40
95
- GPT, GPT-Neox, GPT-J.
@@ -43,61 +98,25 @@ This package has been tested with autoregressive language models such as:
43
98
- Falcon.
44
99
- Llama 2.
45
100
46
-
Encoder-decoder models such as T5 and Flan are currently _not supported_. They are high up in our [priority list](#roadmap).
101
+
Encoder-decoder models such as T5 and Flan are currently _not supported_.
47
102
48
103
## Other Tools
49
104
50
105
-[`swift-chat`](https://github.com/huggingface/swift-chat), a simple app demonstrating how to use this package.
51
106
-[`exporters`](https://github.com/huggingface/exporters), a Core ML conversion package for transformers models, based on Apple's [`coremltools`](https://github.com/apple/coremltools).
52
107
-[`transformers-to-coreml`](https://huggingface.co/spaces/coreml-projects/transformers-to-coreml), a no-code Core ML conversion tool built on `exporters`.
53
108
54
-
## SwiftPM
55
-
56
-
To use `swift-transformers` with SwiftPM, you can add this to your `Package.swift`:
Swift Transformers is a community project and we welcome contributions. Please
112
+
check out [Issues](https://github.com/huggingface/swift-transformers/issues)
113
+
tagged with `good first issue` if you are looking for a place to start!
76
114
77
-
## <aname="roadmap"></a> Roadmap / To Do
78
-
79
-
-[ ] Tokenizers: download from the Hub, port from [`tokenizers`](https://github.com/huggingface/tokenizers)
80
-
-[x] BPE family
81
-
-[x] Fix Falcon, broken while porting BPE
82
-
-[x] Improve tests, add edge cases, see https://github.com/xenova/transformers.js/blob/27920d84831e323275b38f0b5186644b7936e1a2/tests/generate_tests.py#L24
83
-
-[x] Include fallback `tokenizer_config.json` for known architectures whose models don't have a configuration in the Hub (GPT2)
84
-
-[ ] Port other tokenizer types: Unigram, WordPiece
85
-
-[ ][`exporters`](https://github.com/huggingface/exporters) – Core ML conversion tool.
86
-
-[x] Allow max sequence length to be specified.
87
-
-[ ] Allow discrete shapes
88
-
-[x] Return `logits` from converted Core ML model
89
-
-[x] Use `coremltools` @ `main` for latest fixes. In particular, [this merged PR](https://github.com/apple/coremltools/pull/1915) makes it easier to use recent versions of transformers.
90
-
-[ ] Generation
91
-
-[ ] Nucleus sampling (we currently have greedy and top-k sampling)
92
-
-[ ] Use [new `top-k` implementation in `Accelerate`](https://developer.apple.com/documentation/accelerate/bnns#4164142).
93
-
-[ ] Support discrete shapes in the underlying Core ML model by selecting the smallest sequence length larger than the input.
/// - request: The URLRequest for the file to download
148
+
/// - resumeSize: The number of bytes already downloaded. If set to 0 (default), the whole file is download. If set to a positive number, the download will resume at the given position
149
+
/// - numRetries: The number of retry attempts remaining for failed downloads
150
+
/// - expectedSize: The expected size of the file to download. If set, the download will raise an error if the size of the received content is different from the expected one.
151
+
/// - Throws: `DownloadError.unexpectedError` if the response is invalid or file size mismatch occurs
152
+
/// `URLError` if the download fails after all retries are exhausted
153
+
privatefunc httpGet(
154
+
request:URLRequest,
155
+
tempFile:FileHandle,
156
+
resumeSize:Int,
157
+
numRetries:Int,
158
+
expectedSize:Int?
159
+
)asyncthrows{
160
+
guardlet session =self.urlSession else{
161
+
throwDownloadError.unexpectedError
162
+
}
163
+
164
+
// Create a new request with Range header for resuming
0 commit comments