from Hacker News

Ask HN: Why is Generative AI non-deterministic

by michaeljx on 4/1/25, 9:06 PM with 7 comments

Given a trained LLM model with fixed weights, why is it that the same prompt yields different responses? Or is it the case that some type of RL takes place?
  • by bigyabai on 4/1/25, 9:10 PM

    Generative AI typically is deterministic, most inference software includes a random seed to yield different results on each repeated entry.
  • by schoen on 4/1/25, 9:10 PM

    It's deliberately made nondeterministic, partly using something called softmax

    https://en.wikipedia.org/wiki/Softmax_function

    I'd say mainly in order to avoid boring its users.