from Hacker News

Deep Research is now available on Gemini 2.5 Pro Experimental

by extesy on 4/8/25, 11:07 PM with 24 comments

• by doctoboggan on 4/9/25, 12:22 AM

  > In our testing, raters preferred the reports generated by Gemini Deep Research powered by 2.5 Pro over other leading deep research providers by more than a 2-to-1 margin.

  Are these raters experts in the field the report was written on? Did they rate the reports on factuality, broadness, and insights?

  These sort of tests (and RLHF in general) are the reason that LLMs often respond with "Great question, you are exactly right to wonder..." or "Interesting insight, I agree that...". I do not want this obsequious behavior, I want "correct answers"[0]. We need some better benchmarks when it comes to human preference.

  [0]: I know there is no objective correct answer for some questions.

• by jeffbee on 4/9/25, 12:14 AM

  I stumbled across the feature a few hours ago. I had asked Gemini why there's a hole in the middle of the city of Azusa, topologically speaking. It had given me a useless tautological response: because they never annexed it. Then it offered to create a research report and I agreed. Five minutes later I got a notification on my mobile that the report was ready. It had 120 sources including assessor's maps, historical maps, court cases, and narrative articles. The text that went along with it was too verbose and still contained paragraphs of vague stuff, but it had key information linking the Mexican land grants, the founding of the city, and other events of history. Very impressive.

• by DadBase on 4/9/25, 12:30 AM

  Deep research used to mean spending a weekend with grep and a coffee pot. Now it’s just autocomplete with a confidence interval.

• by pizzly on 4/9/25, 1:24 AM

  Just tested it on a case we were working on for months so we can better validate the output. We found it was really good at finding websites from google searches and can navigate websites. From that it gave a good compressive review of the case. Where it failed is searching online databases i.e. one example is a business register. If the search result does not have the exact same keyword it will not review the result. However, the keyword appeared within the document of the search result and thus it missed out on this key information. Overall very good but still needs some work.

• by infecto on 4/9/25, 12:16 AM

  Has anyone tested googles functionality vs ChatGPT? I have lightly played around with it but felt that generally ChatGPTs implementation was a little more educated sounding and felt like it took whatever necessary persona well.