from Hacker News

Show HN: ArXiv-txt, LLM-friendly ArXiv papers

by jerpint on 2/20/25, 4:23 AM with 11 comments

Just change arxiv.org to arxiv-txt.org in the URL to get the paper info in markdown

Example:

Original URL: https://arxiv.org/abs/1706.03762

Change to: https://arxiv-txt.org/abs/1706.03762

To fetch the raw text directly, use https://arxiv-txt.org/raw/abs/1706.03762, this will be particularly useful for APIs and agents

  • by westurner on 2/20/25, 2:41 PM

    If you train an LLM on only formally verified code, it should not be expected to generate formally verified code.

    Similarly, if you train an LLM on only published ScholarlyArticles ['s abstracts], it should not be expected to generate publishable or true text.

    Traceability for Retraction would be necessary to prevent lossy feedback.

  • by owalerys on 2/26/25, 8:03 PM

    Really clean API design, I'm a fan!

  • by lgas on 2/20/25, 10:05 AM

    It just extracts the abstracts?

  • by sbpost on 2/20/25, 3:58 PM

    The example you give doesn't seem to work - the raw txt does not have authors.

  • by jmartin2683 on 2/20/25, 2:38 PM

    This would be awesome wrapped in an MCP server/tool call :)

  • by cchance on 2/21/25, 8:27 PM

    Was super excited that it was going to be the actual papers, kinda cool but just being abstracts doesn't go very far, good luck getting the papers working thats gonna be pretty cool once working, then to feed it all into a vector db XD