from Hacker News

On the Difficulty of Extrapolation with NN Scaling

by ericjang on 1/25/22, 5:56 PM with 2 comments

  • by nerdponx on 1/26/22, 8:47 PM

    It seems like fancy hyperparameter optimization techniques (e.g. Bayesian black-box optimization) probably don't help here either, because they don't solve the problem of extrapolating outside the range of hyperparameter values have have already been tried. Is that a valid conclusion?