by JackHopkins on 11/28/23, 5:52 PM with 1 comments
by JackHopkins on 11/28/23, 5:52 PM
I'm a main contributor of Tanuki (formerly MonkeyPatch).
The purpose of Tanuki is to reduce the time to ship your LLM projects, so you can focus on building what your users want instead of MLOps.
You define patched functions in Python using a decorator, and the execution of the function is delegated to an LLM, with type-coercion on the response.
Automatic distillation is performed in the background, which can reduce the cost and latency of your functions by up to 10x without compromising accuracy.
The real magic feature, however, is how you can implement alignment-as-code, in which you can use Python's `assert` syntax to declare the desired behaviour of your LLM functions. As this is managed in code, and is subject to code-review and the standard software-lifecycle, it becomes much clearer to understand how an LLM feature is meant to behave.
Any feedback is much appreciated! Thanks.