by perryh2 on 11/29/13, 11:35 PM
by viana007 on 11/29/13, 11:32 PM
by kr1m on 11/29/13, 10:48 PM
You don't scrape Facebook, Facebook scrapes you!
by yalogin on 11/30/13, 12:10 AM
So what does it mean by facebook whitelisting a scraping service? Do they actively block scrapers?
by pdfcollect on 11/29/13, 10:52 PM
Is there a way to replace this robots.txt with a null robots.txt? :)
by bibstha on 11/30/13, 6:05 AM
What is a User Agent: Yeti?
by decasteve on 11/30/13, 12:06 AM
Even Facebook's robots.txt has a hatred for my pseudo-anonymous browser settings. Facebook gives me this (for any page): "Sorry, something went wrong. We're working on getting this fixed as soon as we can."