by dhruvdh on 1/2/24, 1:05 PM with 3 comments
by dhruvdh on 1/2/24, 1:05 PM
Description: It is a survey that tries to identify our current progress in designing task-oriented LLM systems - how informed we are when we make decisions about prompting, augmentation, etc. It lists several such design parameters, describes them, and explores varying these parameters through a thought experiment (!). Then we select three parameters (prompting, augmentation, and uncertainty estimation), try to define them, and organize select available research on these topics. Our definition and organization differ slightly from what you'd expect as we try to avoid overlap in each parameter.
Later, we discuss what we find, defining "linear and non-linear contexts", and using it to show how all (?) prompting techniques can be viewed as multi-agent systems, and speak about the implications of that - one of which is on synthetic data generation which the HN community might be interested in. In all, the paper shares seven conjectures to help guide future research efforts.
I will list these conjectures in a comment for those short on time.
Thank you for reading!
by kk58 on 1/4/24, 5:15 PM