Groq: remove model Meta Llama 3.1 70B Versatile

[lumpidu]

Hi,

just tried the Groq integration with the above model: this seems to be deprecated:

But there is now available deepseek-r1-distill-llama-70B, which is extremely powerful. Would that be possible to switch that over ?

[lumpidu]

It seems these are the right settings for models.yml for the new Groq models:

- api_name: deepseek-r1-distill-llama-70B
  name: DeepSeek R1 Distill Llama 70B
  supports_images: false
  supports_tools: false
  input_token_cost_cents: '0.000075'
  output_token_cost_cents: '0.000099'
  best: true
  supports_system_message: true
  api_service_name: Groq
- api_name: llama-3.3-70b-versatile
  name: Llama 3.3 70B Versatile 128k
  supports_images: false
  supports_tools: false
  input_token_cost_cents: '0.000059'
  output_token_cost_cents: '0.000079'
  best: true
  supports_system_message: true
  api_service_name: Groq

[lumpidu]

what is btw. the correct approach to add / update those entries here ?
Since I edited this, my signup for HostedGPT is broken:

...
<snip (some lengthy Rails logs mostly related to AR)>
...
hostedgpt   | web      | [0f4d77c4-14f5-4a22-ab11-5e3ae8b5c583]   ��� app/models/assistant/export.rb:45:in `block (2 levels) in import_from_file'
hostedgpt   | web      | [0f4d77c4-14f5-4a22-ab11-5e3ae8b5c583]   TRANSACTION (0.1ms)  RELEASE SAVEPOINT active_record_1
hostedgpt   | web      | [0f4d77c4-14f5-4a22-ab11-5e3ae8b5c583]   ��� app/models/assistant/export.rb:45:in `block (2 levels) in import_from_file'
hostedgpt   | web      | [0f4d77c4-14f5-4a22-ab11-5e3ae8b5c583]   LanguageModel Load (0.1ms)  SELECT "language_models".* FROM "language_models" WHERE "language_models"."user_id" = $1 AND (language_models.deleted_at is null or language_models.deleted_at > $2) AND "language_models"."api_name" = $3 LIMIT $4  [["user_id", 11], [nil, "2025-02-03 13:22:02.095721"], ["api_name", "claude-3-5-sonnet-20241022"], ["LIMIT", 1]]
hostedgpt   | web      | [0f4d77c4-14f5-4a22-ab11-5e3ae8b5c583]   ��� app/models/assistant.rb:41:in `language_model_api_name='
hostedgpt   | web      | [0f4d77c4-14f5-4a22-ab11-5e3ae8b5c583]   CACHE Assistant Load (0.0ms)  SELECT "assistants".* FROM "assistants" WHERE "assistants"."user_id" = $1 AND "assistants"."deleted_at" IS NOT NULL AND "assistants"."slug" = $2 LIMIT $3  [["user_id", 11], ["slug", "claude-sonnet"], ["LIMIT", 1]]
hostedgpt   | web      | [0f4d77c4-14f5-4a22-ab11-5e3ae8b5c583]   ��� app/models/assistant/slug.rb:16:in `clear_conflicting_deleted_assistant_slug'
hostedgpt   | web      | [0f4d77c4-14f5-4a22-ab11-5e3ae8b5c583]   CACHE Assistant Exists? (0.0ms)  SELECT 1 AS one FROM "assistants" WHERE "assistants"."slug" = $1 AND "assistants"."user_id" = $2 LIMIT $3  [["slug", "claude-sonnet"], ["user_id", 11], ["LIMIT", 1]]
hostedgpt   | web      | [0f4d77c4-14f5-4a22-ab11-5e3ae8b5c583]   ��� app/models/assistant/export.rb:47:in `block (2 levels) in import_from_file'
hostedgpt   | web      | [0f4d77c4-14f5-4a22-ab11-5e3ae8b5c583]   Assistant Create (0.2ms)  INSERT INTO "assistants" ("user_id", "name", "description", "instructions", "tools", "created_at", "updated_at", "language_model_id", "deleted_at", "external_id", "slug") VALUES ($1, $2, $3, $4, $5, $6, $7, $8, $9, $10, $11) RETURNING "id"  [["user_id", 11], ["name", "Claude 3.5 Sonnet"], ["description", nil], ["instructions", nil], ["tools", "[]"], ["created_at", "2025-02-03 13:22:02.096872"], ["updated_at", "2025-02-03 13:22:02.096872"], ["language_model_id", 355], ["deleted_at", nil], ["external_id", nil], ["slug", "claude-sonnet"]]
hostedgpt   | web      | [0f4d77c4-14f5-4a22-ab11-5e3ae8b5c583]   ��� app/models/assistant/export.rb:47:in `block (2 levels) in import_from_file'
hostedgpt   | web      | [0f4d77c4-14f5-4a22-ab11-5e3ae8b5c583]   Assistant Load (0.1ms)  SELECT "assistants".* FROM "assistants" WHERE "assistants"."user_id" = $1 AND (assistants.deleted_at is null or assistants.deleted_at > $2) AND "assistants"."slug" = $3 LIMIT $4  [["user_id", 11], [nil, "2025-02-03 13:22:02.086531"], ["slug", "llama-3-70b"], ["LIMIT", 1]]
hostedgpt   | web      | [0f4d77c4-14f5-4a22-ab11-5e3ae8b5c583]   ��� app/models/assistant/export.rb:45:in `block (2 levels) in import_from_file'
hostedgpt   | web      | [0f4d77c4-14f5-4a22-ab11-5e3ae8b5c583]   TRANSACTION (0.1ms)  SAVEPOINT active_record_1
hostedgpt   | web      | [0f4d77c4-14f5-4a22-ab11-5e3ae8b5c583]   ��� app/models/assistant/slug.rb:16:in `clear_conflicting_deleted_assistant_slug'
hostedgpt   | web      | [0f4d77c4-14f5-4a22-ab11-5e3ae8b5c583]   Assistant Load (0.1ms)  SELECT "assistants".* FROM "assistants" WHERE "assistants"."user_id" = $1 AND "assistants"."deleted_at" IS NOT NULL AND "assistants"."slug" = $2 LIMIT $3  [["user_id", 11], ["slug", "llama-3-70b"], ["LIMIT", 1]]
hostedgpt   | web      | [0f4d77c4-14f5-4a22-ab11-5e3ae8b5c583]   ��� app/models/assistant/slug.rb:16:in `clear_conflicting_deleted_assistant_slug'
hostedgpt   | web      | [0f4d77c4-14f5-4a22-ab11-5e3ae8b5c583]   Assistant Exists? (0.1ms)  SELECT 1 AS one FROM "assistants" WHERE "assistants"."slug" = $1 AND "assistants"."user_id" = $2 LIMIT $3  [["slug", "llama-3-70b"], ["user_id", 11], ["LIMIT", 1]]
hostedgpt   | web      | [0f4d77c4-14f5-4a22-ab11-5e3ae8b5c583]   ��� app/models/assistant/export.rb:45:in `block (2 levels) in import_from_file'
hostedgpt   | web      | [0f4d77c4-14f5-4a22-ab11-5e3ae8b5c583]   TRANSACTION (0.1ms)  RELEASE SAVEPOINT active_record_1
hostedgpt   | web      | [0f4d77c4-14f5-4a22-ab11-5e3ae8b5c583]   ��� app/models/assistant/export.rb:45:in `block (2 levels) in import_from_file'
hostedgpt   | web      | [0f4d77c4-14f5-4a22-ab11-5e3ae8b5c583]   LanguageModel Load (0.1ms)  SELECT "language_models".* FROM "language_models" WHERE "language_models"."user_id" = $1 AND (language_models.deleted_at is null or language_models.deleted_at > $2) AND "language_models"."api_name" = $3 LIMIT $4  [["user_id", 11], [nil, "2025-02-03 13:22:02.099912"], ["api_name", "llama-3.1-70b-versatile"], ["LIMIT", 1]]
hostedgpt   | web      | [0f4d77c4-14f5-4a22-ab11-5e3ae8b5c583]   ��� app/models/assistant.rb:41:in `language_model_api_name='
hostedgpt   | web      | [0f4d77c4-14f5-4a22-ab11-5e3ae8b5c583]   CACHE Assistant Load (0.0ms)  SELECT "assistants".* FROM "assistants" WHERE "assistants"."user_id" = $1 AND "assistants"."deleted_at" IS NOT NULL AND "assistants"."slug" = $2 LIMIT $3  [["user_id", 11], ["slug", "llama-3-70b"], ["LIMIT", 1]]
hostedgpt   | web      | [0f4d77c4-14f5-4a22-ab11-5e3ae8b5c583]   ��� app/models/assistant/slug.rb:16:in `clear_conflicting_deleted_assistant_slug'
hostedgpt   | web      | [0f4d77c4-14f5-4a22-ab11-5e3ae8b5c583]   CACHE Assistant Exists? (0.0ms)  SELECT 1 AS one FROM "assistants" WHERE "assistants"."slug" = $1 AND "assistants"."user_id" = $2 LIMIT $3  [["slug", "llama-3-70b"], ["user_id", 11], ["LIMIT", 1]]
hostedgpt   | web      | [0f4d77c4-14f5-4a22-ab11-5e3ae8b5c583]   ��� app/models/assistant/export.rb:47:in `block (2 levels) in import_from_file'
hostedgpt   | web      | [0f4d77c4-14f5-4a22-ab11-5e3ae8b5c583]   TRANSACTION (0.1ms)  ROLLBACK
hostedgpt   | web      | [0f4d77c4-14f5-4a22-ab11-5e3ae8b5c583]   ��� app/controllers/users_controller.rb:20:in `create'
hostedgpt   | web      | [0f4d77c4-14f5-4a22-ab11-5e3ae8b5c583]   Rendering layout layouts/public.html.erb
hostedgpt   | web      | [0f4d77c4-14f5-4a22-ab11-5e3ae8b5c583]   Rendering users/new.html.erb within layouts/public
hostedgpt   | web      | [0f4d77c4-14f5-4a22-ab11-5e3ae8b5c583]   Rendered users/new.html.erb within layouts/public (Duration: 0.7ms | GC: 0.0ms)
hostedgpt   | web      | [0f4d77c4-14f5-4a22-ab11-5e3ae8b5c583]   Rendered application/head/_favicons.html.erb (Duration: 0.0ms | GC: 0.0ms)
hostedgpt   | web      | [0f4d77c4-14f5-4a22-ab11-5e3ae8b5c583]   Rendered application/head/_meta_tags.html.erb (Duration: 0.0ms | GC: 0.0ms)
hostedgpt   | web      | [0f4d77c4-14f5-4a22-ab11-5e3ae8b5c583]   Rendered application/_head.html.erb (Duration: 4.1ms | GC: 0.0ms)
hostedgpt   | web      | [0f4d77c4-14f5-4a22-ab11-5e3ae8b5c583]   Rendered layouts/_alerts.erb (Duration: 0.0ms | GC: 0.0ms)
hostedgpt   | web      | [0f4d77c4-14f5-4a22-ab11-5e3ae8b5c583]   Rendered layout layouts/public.html.erb (Duration: 5.2ms | GC: 0.0ms)
hostedgpt   | web      | [0f4d77c4-14f5-4a22-ab11-5e3ae8b5c583] Completed 422 Unprocessable Content in 286ms (Views: 5.3ms | ActiveRecord: 22.6ms (176 queries, 6 cached) | GC: 12.4ms)

[krschacht]

In your paste, I can tell what the changed lines are. For this change that you're making, can you push the change to a PR? Then I can pull it down and test it locally myself.

The spirit of what you are doing should be correct. You should simply be able to edit the models.yml file and restart your server. Upon server restart, it should pick up the changes.

Did you restart the server?

I might be able to spot an issue with how you're changing models.yml, or you may be running into a legitimate bug with how the auto-migrations are being done when we read the new models.yml file. I can't quite tell yet.

[lumpidu]

I just made sure I restarted and immediately there is a non-fatal error in the logs:

hostedgpt   |   TRANSACTION (0.1ms)  ROLLBACK
hostedgpt   |   ��� app/models/assistant/export.rb:47:in `block (2 levels) in import_from_file'
hostedgpt   | bin/rails aborted!
hostedgpt   | ActiveRecord::RecordInvalid: Validation failed: Language model must exist (ActiveRecord::RecordInvalid)
hostedgpt   | /rails/app/models/assistant/export.rb:47:in `block (2 levels) in import_from_file'
hostedgpt   | /rails/app/models/assistant/export.rb:44:in `each'
hostedgpt   | /rails/app/models/assistant/export.rb:44:in `block in import_from_file'
hostedgpt   | /rails/app/models/assistant/export.rb:42:in `each'
hostedgpt   | /rails/app/models/assistant/export.rb:42:in `import_from_file'
hostedgpt   | /rails/lib/tasks/assistants.rake:19:in `block (2 levels) in <main>'
hostedgpt   | /rails/lib/tasks/models.rake:25:in `block in <main>'
hostedgpt   | Tasks: TOP => assistants:import
hostedgpt   | (See full trace by running task with --trace)

I'll file a PR with my changes.

[lumpidu]

see #613