from Hacker News

Text extraction

by theslay on 1/19/14, 1:44 AM with 2 comments

Hi, I'm working on plagiarism detection and I need some help on text extraction from pdfs. I've tried PDFTextStream which really works well for extracting text from pdfs. I need to be able to extract the text into a strutured format where i could query thing like title, chapters,etc. Would appreciate it if I could get pointers to achieving this task. Thanks
  • by pedalpete on 1/19/14, 2:23 AM

    Have you tried posting this to http://stackoverflow.com ? That's a better forum for these kinds of questions.

    If you were to write a blog post about how to structure the extracted text, that's more the HN thing.

  • by mindcrime on 1/19/14, 2:52 AM

    I won't swear to it, but I suspect you're going to have to largely roll your own, and that it will be at least partly heuristic driven. I use Apache Tika[1] to extract text from PDFs and then index it with Lucene, but we don't need to discriminate between various chapters or anything. But I can picture how you could use OpenNLP[2] and some custom code, to break down the chapters.

    [1]: http://tika.apache.org

    [2]: http://opennlp.apache.org