by saran945 on 9/17/24, 2:01 AM
"We evaluated REAP using a dataset designed to highlight the limitations of large language models (LLMs), comparing zero-shot prompting with REAP-enhanced prompts across six state-of-the-art models: OpenAI’s o1-preview, o1-mini, GPT-4o, GPT-4o-mini, Google’s Gemini 1.5 Pro, and Claude 3.5 Sonnet. The results show significant performance improvements, with o1-mini increasing by 40.97%, GPT-4o by 66.26%, and GPT-4o-mini by 112.93%. While OpenAI's o1-preview already demonstrated strong baseline performance, it still showed modest gains. In addition to the performance improvements, REAP provides a cost-effective solution. For instance, GPT-4o-mini, which is about 100 times cheaper than o1-preview, delivered competitive results."