by cubie on 12/19/24, 4:53 PM with 75 comments
by jph00 on 12/19/24, 6:40 PM
In fact, the stuff mentioned in the blog post is only the tip of the iceberg. There's a lot of opportunities to fine tune the model in all kinds ways, which I expect will go far beyond what we've managed to achieve in our limited exploration so far.
Anyhoo, if anyone has any questions, feel free to ask!
by janalsncm on 12/19/24, 9:23 PM
This is partially because people using decoders aren’t using huggingface at all (they would use an API call) but also because encoders are the unsung heroes of most serious ML applications.
If you want to do any ranking, recommendation, RAG, etc it will probably require an encoder. And typically that meant something in the BERT/RoBERTa/ALBERT family. So this is huge.
by shahjaidev on 12/22/24, 8:06 AM
by deepsquirrelnet on 12/20/24, 12:01 AM
I’d love to distill a “ModernTinyBERT”, but it seems a bit more complex with the interleaved layers.
by jbellis on 12/19/24, 5:49 PM
- Can I fine tune it with SentenceTransformers?
- I see ColBERT in the benchmarks, is there an answerai-colbert-small-v2 coming soon?
by mark_l_watson on 12/20/24, 12:44 AM
I am going to wait until Ollama has this in their library, even though consuming HF is straight forward.
The speedup is impressive, but then so are the massive speed improvements for LLMs recently.
Apple has supported BERT models in their SDKs for Apple developers for years, it will be interesting to see how quickly they update to this newer tech.
by wenc on 12/19/24, 10:50 PM
I was given to understand that they are a better alternative to LLM type models for specific tasks like topic classification because they are trained to discriminate rather than to generate (plus they are bidirectional so they can “understand” context better through lookahead). But LLMs are pretty strong so I wonder if the difference is negligible?
by dmezzetti on 12/19/24, 9:32 PM
by pantsforbirds on 12/19/24, 6:16 PM
by readthenotes1 on 12/19/24, 8:01 PM
by carschno on 12/19/24, 7:26 PM
by 303bookworm on 12/24/24, 8:04 AM
by Labo333 on 12/20/24, 10:16 AM
by GaggiX on 12/20/24, 2:51 AM
by neodypsis on 12/20/24, 3:50 AM
by vietvu on 12/20/24, 2:28 AM
by crimsoneer on 12/19/24, 9:41 PM
by zelias on 12/19/24, 5:54 PM
by Arcuru on 12/19/24, 6:05 PM