from Hacker News

Hidet: A Deep Learning Compiler for Efficient Model Serving

by bretthoerner on 4/27/23, 5:14 PM with 1 comments

• by pavelstoev on 4/27/23, 8:26 PM

  Generally, Hidet outperforms other inference compilers - PyTorch Eager, ORT, TRT, TVM. For example, PyTorch Eager - too much framework overhead. ORT -doesn't do operator fusion. TRT - close-sourced and hard to fix if a model can not run. TVM - tuning time is too long, also limited expressiveness in optimization.

  Additionally this comes with Hidet Script, a brand new domain-specific language to write tensor programs in Python with high flexibility to express optimizations that can only be done in C++ CUDA C code. Hidet Script also supports operator tuning and automatic fusion.