from Hacker News

Google is using AI to censor independent websites like mine

by rapnie on 5/29/25, 11:05 AM with 167 comments

  • by nilirl on 5/29/25, 12:35 PM

    The whole argument hinges on one claim: We were censored not because of content but because of who we are.

    Strongest evidence in support: Drop in traffic coincided with google algorithm update.

    Biggest lack of evidence: Nothing that shows it's because of who they are and not their content. Author defended their content by pointing out the cost and labor that goes into making their content.

    Was the content really authentic and useful? This is hard to prove (and I suspect hard even for an algorithm to discern), and so I had a look:

    - Each article did have that human feel to it with photos of the author in the locations they were talking about. Articles also included small personalized evaluations about topics.

    So, yes, I did think it was authentic and useful. But how did it compare to the 1000s of other sites who can replicate this? Human writers aren't rare, and travel isn't a small interest to our species.

    And that's the crux of it: Even if you're useful and authentic, search engines have to rank you among other sites that are useful and authentic.

    You could be amazing and still end up on page 30 because you're not alone in being amazing on the internet.

    I don't know what a fair algorithm looks like, and if a fair algorithm would make everyone equally happy.

    For now, someone has the power to direct how the algorithm chooses. Who should have that power is a fair question, but I don't think anything will mitigate the problem of being ranked low even with quality content. It's a ranking against the rest of the internet, not just an evaluation of quality.

    At the moment, if two sites are somewhat equally useful, being profitable to Google probably gets you a bump.

  • by fidotron on 5/29/25, 12:05 PM

    > Google plans to use AI to consume and replace the open web.

    They've been boiling the frog here for a long time. The "open web" is a euphemism for the Google Chrome monoculture walled-garden-in-waiting. Google exert so much influence there that building on the web is in practice barely different than building on say iOS. You can do what you want if you don't want to make money, but if you do then you will have to play along with the big G.

    My hunch is Google never psychologically recovered from Facebook absolutely wiping out G+, and they have been on a mission to ensure nothing like that ever happens again.

  • by blinding-streak on 5/29/25, 12:14 PM

    > As one Google executive recently explained: “Organizing information is clearly a trillion-dollar opportunity, but a trillion dollars is not cool anymore. What’s cool is a quadrillion dollars.”

    This uncited, anonymous quote sounds very made up. Cursory search couldn't find anything like this.

    Regardless, while the article makes some good points, it is also dripping with entitlement. Google gave you incredible monetizable traffic for two+ decades. At some point you need to capture your audience and make a real connection with them so they don't need Google to interact with you. Give them a value prop.

  • by gorjusborg on 5/29/25, 12:22 PM

    I can't get past the sloppy and inflammatory use of English here.

    This is not 'censorship'. It probably isn't banning, nor is it 'shadowbanning'. Google tunes its algorithms and lets the chips fall. Some win, some lose.

    While I understand how Google's dominance in search can have outsized effects on Internet commerce, the writer has near-zero credibility with me based on their writing. Honest people making honest statements don't need to exaggerate to make the point.

  • by kayo_20211030 on 5/29/25, 12:39 PM

    I do sympathize with OP's situation, but there's a dissonance also.

    As a site owner they seemed to have been happy to play the SEO game under the old rules, winning a little bit, but unhappy to play the SEO game under the new rules, losing a little bit. Depressingly, it's Google's ball, so it Google's rules.

    At the end of the post, OP's suggests some remedies for web users, but they seem either impractical or ineffective. The recommendations for the commission also tacitly assume that Google is either a) a monopoly, to be regulated as such, or b) a utility, to be regulated as such. Maybe a) will be proven, but b) seems like a real stretch goal. They're a private enterprise and without congressional action or administrative action to change the laws and regulations, they have wide latitude in how they behave for commercial reasons. I don't believe they can be forced to do anything they don't want to do without statutory or regulatory action.

    I do sympathize, but what's lost for a consumer?

  • by thedevilslawyer on 5/29/25, 12:02 PM

    The article treats as axiom that organic traffic is a right. It's not.

    Most people don't want to read an article on travel advice when AI gives us much better and specific advice, with references, when we want it.

  • by supriyo-biswas on 5/29/25, 12:32 PM

    I'd like to look at this from a different angle.

    Since Google is an established search player, and that the company has made many public statements on the symbiotic relationship between them and publishers, this means the FTC complaint is likely to go through. Given the current US admin, I also assume they'd pursue some action, either combined with the antitrust efforts, or through a separate legal action as it nicely dovetails with the various accusations of "censorship" that they have.

    This means Google would be forced to reduce their AI offerings, and the website publishers in question get the thing that they were looking for. Meanwhile, new search entrants such as OpenAI/Perplexity, etc. are "allowed" to implement the same things that said publishers are opposed to, because of their smaller size and different perceptions, and the lack of similar statements.

    Now, because LLMs are rapidly replacing most search engine uses (I've seen this firsthand while travelling on public transport that users in my country first default to ChatGPT etc.), this would mean Google is slowly replaced in an indirect way, not because they could not innovate but because they were not allowed to.

    The implications of this are very interesting to me; it means that a corporation should rarely make any statements that have the tiniest chance of creating obligations (which is somewhat similar to Everything is Securities Fraud[1]) and corporate displacement happens not because companies out-innovate each other because the incumbent can do so too, but because legal obligations constrain their actions which ultimately lead to their death (somewhat similar to Planck's principle[2]).

    [1] https://www.bloomberg.com/opinion/articles/2019-06-26/everyt...

    [2] https://en.wikipedia.org/wiki/Planck%27s_principle

  • by xnx on 5/29/25, 12:30 PM

    Isn't the business model of "travellemming.com" to get free/cheap user generated content and profit from it? This type of aggregation is very low value add and Google does not owe them any amount of free traffic.
  • by redwood on 5/29/25, 12:13 PM

    It seems inevitable that Google would do this or else they themselves to be killed by someone else.

    What concerns me about this general shift is that it leads to groupthink. How do you ensure that new ideas, new innovation, new perspectives are being bootstrapped into the hive mind.

  • by vladyslavfox on 5/29/25, 12:46 PM

    > Sorry, your request has been blocked

    We have to fight bots so hard now that it often prevents real people from accessing websites.

    I suggest the author to check out Anubis. It's much better for fighting off bots while not blocking humans.

  • by tndibona on 5/29/25, 12:27 PM

    My personal opinion. Google sees the writing on the wall with the rise of perplexity. People want trustable summaries of long winding content to make decisions. It’s business of sending people to the relevant content and serving ads has to change to compete. It is simply redefining how it serves up information. The fact that small information servers like us get wiped out is the unfortunate consequence.

    I’m not saying we all have to innovate or perish but how did our rules based order allow Google to get to this point.

  • by basilgohar on 5/29/25, 12:20 PM

    Each step and subsequent revelation makes a stronger and stronger case that companies like Alphabet (nee Google) wield too much power across their products and should be broken up to ensure competitiveness and user choice remain. Otherwise, we'll face a monopolistic monoculture that caters to the powers that be and cements existing power structures more firmly.

    You'd think Google would see this, but instead they're doubling down instead of eking out a sustainable evolution of their technology.

  • by Barrin92 on 5/29/25, 12:54 PM

    The alleged economic incentive suggested in the article, that Google wants to deal with "established big players" is literally nonsensical, something that the article itself arrives at, weirdly enough.

    "But we later realized the shadowban really was about the type of website we are (i.e., small and independent). While Google gives large publishers an appeal and recovery process, small and independent publishers have no path to appeal our shadowbans.[...]"

    That's not an indication of Google liking big players, it's the opposite. Big players have leverage against Google, you the small publisher do not, as you correctly identified in the case of reddit:

    "[...]Unlike independent publishers, though, Reddit actually had leverage over Google. Reddit was the owner of a trove of historical user generated content that Google wanted for its grand AI plans. If Google could secure a deal for Reddit’s content, maybe that would spare Google the expense of negotiating licensing deals with the web’s many disparate publishers and rightsholders."

    Google doesn't need to negotiate with you, they can kill independent publishers, which is why you're writing this blogpost. That's the entire logic of the internet platform economy. Kill traditional distributors, abuse atomized content creators. Youtube and Google hate publishers as much as Uber hates taxi companies and unions.

    Which is why the literal reason given by Google is much more likely, they just don't think your content is good.

  • by seaourfreed on 5/29/25, 12:14 PM

    Replacing Google with open source. Imaging this: 1) An open-source competitor to Google Search exists 2) It exists in "Front-end nodes". This is the website user interface. Search query entered here. It then searches cached indexes. 3) Searches fall in two patterns. a) Frequent searches (top 20m search query phrases), b) long-tail rare (beyond top 20m) 4) Here is how "Frequent Searches" results are built. * They are built before. Updated once a month * There are LLM prompts on how to assess web pages against a search query. This is about taking the top 5,000 website contenders, and ranking those exactly * There can be contention on these LLM. This is how that is solved. * Stake holders. Each is a company or an owner of a website with human content. They have the ability to vote. This is towards "fair" LLMs to reason on finding the best match for a term. * They vote away LLM prompts that skew or overly focused on one area. 5) Long-tail searches may be redirected to a DuckDuckGo like solution. Bing / Google / DuckDuckGo compete to be fair to win the traffic 6) The search front-end gets add revenue for Frequent searches. Their job is to do advertising to get customers to switch from Google search 7) The Frequent Search final results of voting are saved in the blockchain. From the once a month LLM compute. 8) LLM compute may happen with different LLM vendors, to weed out bias. 9) The LLM prompt runs (once a month), may happen at ~200 (or 2,000) different open source servers. Run by different people. What wins, is consensus on what most matches. This weeds out bad actors and bias. 10) Big non-tech companies may pay for this, in order to get SEO to work again, for their revevnue. Organic traffic. Ford, GM, Home Depot, AT&T, SalesForce, Oracle, J&J, P&E, GE, Colgate-Palmolive, General Mills, Kraft, etc.
  • by FinnLobsien on 5/29/25, 12:29 PM

    Welcome to the era of massively mediocre online content. It started with mediocre SEO content that just amalgamated 5 other page 1 results until Google results became basically inbred content that meant nothing to anyone.

    Now we have LLMs which amalgamate that type of content and spit out a personalized, but still mediocre response.

    On the visual side, TikTok and Reels are doing their part to make everyone run to the same picture spot, go to the same mediocre (but cool-looking) restaurants and so on.

  • by storus on 5/29/25, 2:26 PM

    How is Google going to survive? Their search quality tanked, the AI is too expensive to run for every single query and they cut off people that made their advertisement model possible. Are the advertisers themselves going to form a consortium to buy Google at some point? What is the end game? I only see them bleeding money in the medium term.
  • by wilde on 5/29/25, 12:26 PM

    > It was hurtful to hear Google thought our site was of “little value” to the web.

    I read some of the guides. Google is right that there’s little unique content here.

    Perhaps this is the right tone for a letter to the FTC. I find it hard to sympathize with the author with this style (even though they are likely right that Google is killing the web)

  • by logicchains on 5/29/25, 12:11 PM

    Penalising small websites is a natural consequence of Google's shift towards becoming more and more of a political actor. Google actively tries to suppress content that doesn't support the Silicon Valley narrative, and this best way to achieve this is prioritising a smaller number of large websites and organisations that share the same political values.
  • by 0x000xca0xfe on 5/29/25, 12:58 PM

    Is it just me or has search become even worse in the past couple months?

    I've used Google and Bing to look up my Geocaches because the Geocaching interface is clunky. Except I can't find them anymore. Exact ASCII matches in the <title> tag, even with a unique identifier, but now it's become impossible to find them via search.

    Just blank results, ads or SEO spam no matter what I try. site:xxx, inurl:, exact quotes, similar words, repetition, nothing works anymore. What is going on?!

    And geocaching.com is a pretty big site. I wonder what is happening to small blogs...

  • by tasuki on 5/29/25, 12:27 PM

    I wonder what "independent" means in the title?
  • by Aldipower on 5/29/25, 12:09 PM

    I see it in a positive light. For me it means back to the roots. People will visit your website by typing in your URL directly into the address bar. Direct marketing and mouth-to-mouth will be a thing again.
  • by rs186 on 5/29/25, 12:25 PM

    Relevant: Sundar Pichai recently gave an interview to The Verge, full transcript here: https://www.theverge.com/decoder-podcast-with-nilay-patel/67...

    > You’re always going to have areas where people are robustly debating value exchanges, etc., like app developers and platforms. That’s not on the web, etc. There’s always going to be — when you’re running a platform — these debates. I would challenge, I think more than any other company, we prioritize sending traffic to the web. No one sends traffic to the web in the way we do. I look at other companies, newer emerging companies, where they openly talk about it as something they’re not going to do. We are the only ones who make it a high priority, agonize, and so on. We’ll engage, and we’ve always engaged.

    > There are going to be debates through it all, but we are committed to, I’ve said this before, everything we do across all... You will see us five years from now sending a lot of traffic out to the web. I think that’s the product direction we are committed to. I think it’s what users look for when they come to Google, and the nature of it will evolve. But I am confident that that’s the direction we’ll continue taking.

    Sounds like he wants to be nice with publishers and direct traffic to them like "for over 25 years" (a phrase he used many times in the interview), but it remains to be see what their action is.

  • by M1ch431 on 5/29/25, 12:38 PM

    I'd be happy to find a website like OP's. Google's output is trash, it is unreliable, and it is an inferior product compared to the Google of the past - hopefully competition will push them to improve/remove the AI slop dominating their results.
  • by hackerbeat on 5/29/25, 5:07 PM

    SEO is a dead horse.
  • by p3rls on 5/29/25, 6:28 PM

    Bro I don't want to sound mean but you've got a blog.

    I built a customized wiki cms, it's the best, most info-dense site in my niche by a mile and I've still been getting fucked by google for a decade.

    Literally, please, check right now and look at kpopping.com and then search for some kpop results on google. Things like "Jimin". The top results are always websites like the hindustantimes .com, and sportskeeda.

    This has been going on for literally a decade in my niche, a decade! Every quality website we've had in our scene is long dead and has been replaced by indian and ai garbage besides me, it's pathetic. No one builds anything but wordpress copy paste slop these days.

  • by Sloowms on 5/29/25, 12:34 PM

    Google and all the other algorithm related content platforms have had this power before AI and have been using it as well. I think AI is a new layer to the SEO war.

    I don't care for the grand conspiracy claims though. It's pretty obvious Google will optimize their search engine and the question answering to benefit their bottom line.

    Maybe there are people actively working towards evil within Google. I think it's more likely some people recognize the implications of changes, some people don't but no one is working against those changes and ultimately the system will be capable to be used for very bad things and will do bad things naturally.

    Anyway a better title would have been: Google's AI is the newest website killing feature

  • by moresea on 5/29/25, 12:03 PM

    .
  • by sebstefan on 5/29/25, 12:11 PM

    >Google will just source information from a handful of sources and partner websites that it controls and selects – effectively creating an information cartel.

    And risk providing worse results for "control"..?

    citation needed

  • by ZephyrBlu on 5/29/25, 12:25 PM

    > As one Google executive recently explained: "Organizing information is clearly a trillion-dollar opportunity, but a trillion dollars is not cool anymore. What’s cool is a quadrillion dollars."

    Holy based.