Skip to content

Expose model generate parameters by API server #55

@SeanHH86

Description

@SeanHH86
generate_kwargs:
  do_sample: true
  max_new_tokens: 128
  min_new_tokens: 16
  temperature: 0.7
  repetition_penalty: 1.1
  top_p: 0.8
  top_k: 50

Metadata

Metadata

Assignees

Labels

No labels
No labels

Type

No type

Projects

No projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions