Skip to content

Commit c01449a

Browse files
ikawrakowKawrakow
andauthored
Fix KLD precision (ikawrakow#325)
Co-authored-by: Iwan Kawrakow <iwan.kawrakow@gmail.com>
1 parent 3e7be3d commit c01449a

File tree

1 file changed

+1
-1
lines changed

1 file changed

+1
-1
lines changed

examples/perplexity/perplexity.cpp

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -133,7 +133,7 @@ static double log_softmax(int n_vocab, const float * logits, uint16_t * log_prob
133133
max_logit = std::max(max_logit, logits[i]);
134134
min_logit = std::min(min_logit, logits[i]);
135135
}
136-
min_logit = std::max(min_logit, max_logit - 16);
136+
min_logit = std::max(min_logit, max_logit - 24);
137137
double sum_exp = 0.0;
138138
for (int i = 0; i < n_vocab; ++i) {
139139
sum_exp += expf(logits[i] - max_logit);

0 commit comments

Comments
 (0)