Skip to content

Commit 6549915

Browse files
committed
Update Readme.md
1 parent cf9a3d1 commit 6549915

File tree

1 file changed

+19
-18
lines changed

1 file changed

+19
-18
lines changed
Lines changed: 19 additions & 18 deletions
Original file line numberDiff line numberDiff line change
@@ -1,16 +1,17 @@
11
# Local models
2+
# Local models
23
The two websites benchmark are:
34
- Example 1: https://perinim.github.io/projects
45
- Example 2: https://www.wired.com (at 17/4/2024)
56

67
Both are strored locally as txt file in .txt format because in this way we do not have to think about the internet connection
78

8-
| Hardware | Model | Example 1 | Example 2 |
9-
| ------------------ | --------------------------------------- | --------- | --------- |
10-
| Macbook 14' m1 pro | Mistral on Ollama with nomic-embed-text | 11.60s | 26.61s |
11-
| Macbook m2 max | Mistral on Ollama with nomic-embed-text | 8.05s | 12.17s |
12-
| Macbook 14' m1 pro | Llama3 on Ollama with nomic-embed-text | 29.87s | 35.32s |
13-
| Macbook m2 max | Llama3 on Ollama with nomic-embed-text | 18.36s | 78.32s |
9+
| Hardware | Model | Example 1 | Example 2 |
10+
| ---------------------- | --------------------------------------- | --------- | --------- |
11+
| Macbook 14' m1 pro | Mistral on Ollama with nomic-embed-text | 16.291s | 38.74s |
12+
| Macbook m2 max | Mistral on Ollama with nomic-embed-text | | |
13+
| Macbook 14' m1 pro<br> | Llama3 on Ollama with nomic-embed-text | 12.88s | 13.84s |
14+
| Macbook m2 max<br> | Llama3 on Ollama with nomic-embed-text | | |
1415

1516
**Note**: the examples on Docker are not runned on other devices than the Macbook because the performance are to slow (10 times slower than Ollama). Indeed the results are the following:
1617

@@ -22,20 +23,20 @@ Both are strored locally as txt file in .txt format because in this way we do n
2223
**URL**: https://perinim.github.io/projects
2324
**Task**: List me all the projects with their description.
2425

25-
| Name | Execution time (seconds) | total_tokens | prompt_tokens | completion_tokens | successful_requests | total_cost_USD |
26-
| --------------------------- | ------------------------ | ------------ | ------------- | ----------------- | ------------------- | -------------- |
27-
| gpt-3.5-turbo | 25.22 | 445 | 272 | 173 | 1 | 0.000754 |
28-
| gpt-4-turbo-preview | 9.53 | 449 | 272 | 177 | 1 | 0.00803 |
29-
| Grooq with nomic-embed-text | 1.99 | 474 | 284 | 190 | 1 | 0 |
26+
| Name | Execution time (seconds) | total_tokens | prompt_tokens | completion_tokens | successful_requests | total_cost_USD |
27+
| ------------------------------- | ------------------------ | ------------ | ------------- | ----------------- | ------------------- | -------------- |
28+
| gpt-3.5-turbo | 4.132s | 438 | 303 | 135 | 1 | 0.000724 |
29+
| gpt-4-turbo-preview | 6.965s | 442 | 303 | 139 | 1 | 0.0072 |
30+
| gpt-4-o | 4.446s | 444 | 305 | 139 | 1 | 0 |
31+
| Grooq with nomic-embed-text<br> | 1.335s | 648 | 482 | 166 | 1 | 0 |
3032

3133
### Example 2: Wired
3234
**URL**: https://www.wired.com
3335
**Task**: List me all the articles with their description.
3436

35-
| Name | Execution time (seconds) | total_tokens | prompt_tokens | completion_tokens | successful_requests | total_cost_USD |
36-
| --------------------------- | ------------------------ | ------------ | ------------- | ----------------- | ------------------- | -------------- |
37-
| gpt-3.5-turbo | 25.89 | 445 | 272 | 173 | 1 | 0.000754 |
38-
| gpt-4-turbo-preview | 64.70 | 3573 | 2199 | 1374 | 1 | 0.06321 |
39-
| Grooq with nomic-embed-text | 3.82 | 2459 | 2192 | 267 | 1 | 0 |
40-
41-
37+
| Name | Execution time (seconds) | total_tokens | prompt_tokens | completion_tokens | successful_requests | total_cost_USD |
38+
| ------------------------------- | ------------------------ | ------------ | ------------- | ----------------- | ------------------- | -------------- |
39+
| gpt-3.5-turbo | 8.836s | 1167 | 726 | 441 | 1 | 0.001971 |
40+
| gpt-4-turbo-preview | 21.53s | 1205 | 726 | 479 | 1 | 0.02163 |
41+
| gpt-4-o | 15.27s | 1400 | 715 | 685 | 1 | 0 |
42+
| Grooq with nomic-embed-text<br> | 3.82s | 2459 | 2192 | 267 | 1 | 0 |

0 commit comments

Comments
 (0)