from Hacker News

Kagi Small Web

by u2077 on 9/7/23, 3:22 PM with 185 comments

  • by freediver on 9/7/23, 3:45 PM

    Vlad from Kagi here, thanks for posting. The RSS feed unexpectedly broke (edit: the feed is back up! [3]) just as we published the blog post, in the true spirt of "small web" :) Should be up in 30 minutes which will enable the site to function too (it uses the same feed).

    This has been a personal pet project of mine and I spent considerable time getting my hands dirty with the code, as the team was busy with other initiatives. When I said the "feed broke" for the launch I meant I broke it. Software is messy especially for an old school dev. I learned in the process I am not a very good coder anymore (if I ever was one?), constantly going back and fixing stuff I previously thought was solid. Check it out in the linked repo [1].

    Most importantly - I found the site replace the need for discovery for me, and getting to know various different humans and their writing felt good! A lot of unexpected stuff surfaced and the web felt close again. I think there is a glimpse of hope in the concept and I hope you see it too. And the improvements to search quality and diversity this brings are real.

    You can check the list of included websites here [2]. And all the recent posts already surface in Kagi results (for relevant queries).

    [1] https://github.com/kagisearch/smallweb

    [2] https://github.com/kagisearch/smallweb/blob/main/smallyt.txt

    [3] https://kagi.com/smallweb

  • by arboles on 9/7/23, 5:20 PM

    > These notes will vanish in about a week as we cycle in new content - emphasizing the fleeting, imperfect nature of the small web.

    Kagi could just admit they don't want to moderate notes or store them permanently. No need to push down the small web, because a lot of small sites preserve their content.

    I get that Kagi probably has data indicating the reality of how often sites down, but it seems from my experience that content in big platforms disappears often as well, even in the cases where the creator hadn't forgotten about it. The "Small web" websites made by a creator that cares have the room to be much more permanent.

  • by kickdaddy on 9/7/23, 3:54 PM

    Kagi is worth every penny. Been using it for 6 months as my primary search engine. I think I have used the Google !g 10 or so times in that period maybe?

  • by unshavedyak on 9/7/23, 5:01 PM

    This sort of stuff makes me really happy to be a Kagi subscriber. Not only do i get value out of Kagi, but this shows me that the money is being used to develop Kagi in a way i agree with. By comparison, Spotify (just picking one of my subs) feels hostile to me. I pay them, but would cancel in a heart beat if i felt i had options.

    I really appreciate Kagi's development matching what i feel like i'm buying. Thanks Kagi Team <3

  • by jefftk on 9/7/23, 5:42 PM

    It looks like you're stripping MathML out of the RSS feed -- is that intentional, or are you using an older sanitizer that doesn't recognize it? For example, my RSS feed [1] for my most recent post [2] has:

        &lt;math display=&quot;block&quot;&gt;
    &lt;msup&gt;&lt;mi&gt;e&lt;/mi&gt;
          &lt;mrow&gt;&lt;mi&gt;k&lt;/mi&gt;&lt;mi&gt;t&lt;/mi&gt;&lt;/mrow&gt;&lt;/msup&gt;
    &lt;/math&gt;
    Which shows up in your RSS feed [3] as:

        e
          kt
    [EDIT: filed an issue: https://github.com/kagisearch/smallweb/issues/10]

    [1] https://www.jefftk.com/news.rss

    [2] https://www.jefftk.com/p/weekly-incidence-vs-cumulative-infe...

    [3] https://kagi.com/api/v1/smallweb/feed

  • by marginalia_nu on 9/7/23, 4:14 PM

    Some Q:s

    #1 What's the rationale behind favoring recent blog updates? In my experience, recent updates make the weakest search results, more prone to updates and link breakage, and overall tend to be of lower quality. I also wonder if promoting recent content might incentivize pumping out low-quality entries to increase the odds of being listed.

    #2 In dabbling in the domain I've always ended up with an almost absurd skew toward technical programmer:y blogs. While there is a strong overlap between the cohort with a blog, and the cohort with programmer interests, I feel it would be more inviting to other groups if other interests were better represented. Is this something you've thought about, and if so, what do you think might be done?

  • by mindwork on 9/7/23, 9:30 PM

    This is amazing! Over the last 2 months I've been on a personal journey to browse through small web links. It all started from the "Ask HN: Could you share your personal blog here?" post on HN, and then I found out that somebody put a website with the links from this post organized(https://dm.hn/). I exported all 1651 blog link in to XLS and now every now and then I open up 5-7 just to read random articles and mark it as "viewed". So far I've been through 250 out of 1651!

    I wish kagi has something similar, one place where I can see all the links to the personal websites collected via all it's sources

  • by benhoyt on 9/7/23, 9:30 PM

    Thanks for the prominent link to my article! ("The Small Web is Beautiful") I consider that my personal software manifesto, so I'm glad to see it being promoted more widely.

    I really like what you're doing with Kagi Small Web -- love that you've taken the initiative to start surfacing all this excellent content. Keep up the good work. I think I'll try out Kagi search...

  • by vladstudio on 9/7/23, 4:43 PM

    Vlad, you're awesome! It's hard to tell to what extent it is actually beneficial for users or your company, but I feel warm and fuzzy inside knowing these kinds of things still happen in modern Internet.

  • by sotix on 9/7/23, 8:30 PM

    I would love to have my website added to the list because I block all crawlers in my robots.txt, but it looks like I’m not allowed to submit my own website. Is there a way to go about having it added? It would be great to be part of kagi search results but don’t want to be scraped by Google, ChatGPT, or the next group, so I found it easiest to block everything.

  • by thrtythreeforty on 9/7/23, 3:59 PM

    This is quite interesting. I feel you should include content from blogs that publish less often than once a week - some of the best blogs post once in a blue moon and are still quite interesting.

  • by Sebguer on 9/7/23, 10:25 PM

    I'm interested in the mental gymnastics around how this kind of content manipulation is okay but the suggestion of inserting a suicide hotline at the top of search results around 'how to kill myself' was too political? https://kagifeedback.org/d/865-suicide-results-should-probab...

  • by rebeccaskinner on 9/8/23, 4:23 AM

    I’ve been using Kagi for over a year and it just keeps getting better. I really appreciate Kagis user and privacy-centric view of the web, and I love how the quality of the product proves that sticking to those ideals is really a better way to build. I hope they continue to gain enough momentum with non-tech audiences to continue to exist for a long time.

  • by blitz_skull on 9/7/23, 4:09 PM

    Things like this are why I keep rooting for Kagi. I haven’t purchased a subscription yet because I’m still on my free 100 queries, but I will as soon as they run out. I really love this project!

  • by jabroni_salad on 9/7/23, 5:11 PM

    the smallwebsite is delightful. It reminds me of Stumbleupon, which I miss dearly.

    And of course it showed me Questionable Content, which I first got to via stumbleupon.

  • by ot1138 on 9/7/23, 4:14 PM

    Nice, but I can't figure out how to search it. The "Small Web" site linked to in the article just shows a number of links to Kagi articles.

  • by ilrwbwrkhv on 9/7/23, 4:03 PM

    This is really good. With more and more of the large web filled with SEO crap and gargantuan articles written because that's what google wants, it is great to read from the people who actually write the interesting things the AIs train on.

  • by jci on 9/7/23, 9:30 PM

    Very happy paying Kagi search and Orion browser user here. Just want to send some general good vibes your way. I really love what you guys are doing, and this just adds to that.

  • by vk7eu on 9/7/23, 4:13 PM

    Great idea!

    https://kagi.com/smallweb displays the homepage for the Kagi Blog at the moment, though.

  • by pavlov on 9/7/23, 5:24 PM

    Great initiative, but it feels like they're not quite eating their own dogfood. This post contains a link titled "highlighting blog posts from HN users" which goes to Twitter — a massive centralized social media app that's pretty much the opposite of "small web".

  • by mbwgh on 9/7/23, 3:55 PM

    Kagi is the ten bucks per month that just keep giving.

  • by ecshafer on 9/7/23, 3:51 PM

    This is cool. The web has become too centralized.

  • by rapnie on 9/7/23, 6:56 PM

    More Small Web in the tools Aral Balkan is building

    https://small-tech.org/research-and-development/

  • by goplayoutside on 9/8/23, 10:07 AM

    Another satisfied Kagi customer here ('early adopter's badge). Just wanted to chime in and say I think KSW is awesome. I'll happily renew my annual subscription.

  • by uconnectlol on 9/7/23, 10:54 PM

    getting tired of hearing about the old web. and still not seeing anything that resembles it. everything still needs js. everything still needs cloudflare bypasses if you view it on tor. this includes whatever cool hacker site you think you have because i almost never see these. i have had a site with just plain html for 10 years and it has no nonsense just content, you can do this too, it doesnt need a movement that just talks about doing it.

    not that it matters since the old web wasn't good. it was as terrible as now. the UI of absolutely every website ever made has been terrible quirky garbage compared to something like windows 98. Even back then there was a massive difference going from windows 98 (sane GUI) to web (garbage hackjob GUI + ads (YES REMEMBER 40 POPUPS? ADS? TOOLBARS? THE OLD WEB WAS NOT GOOD IT WAS A HELL JUST LIKE NOW)).

    the content was never good either. every topic discussed on the web is little cliques who believe some easy to digest nonsense and then if you go skim some books on the subject the meta is completely different. except programming since that just centers around the web [1]. think of anything else like cooking or engineering

    the web is a terrible protocol that should have died 20 years ago and been replaced with something that was modern at the time like freenet (and they should have made an alternative to html etc).

    1. and this is ironic too since programming is the one field that is steered by the web's body of pseudoknowledge and as a result you have people who think C, PHP, and OOP are legitimate programming practices.

  • by endorphine on 9/8/23, 10:12 AM

    As a SWE/SRE, how does Kagi compare in quality of results (i.e. finding you the answer easily) vs. DuckDuckGo and Google?

  • by w0mbat on 9/7/23, 6:23 PM

    I find this kind of domain name reuse unsettling: for years, Kagi.com was the address of a payment service for shareware authors. Man those were the days. Lots of people were perfectly willing to pay me $20 for a useful little utility app and now they won't pay $2 in an App Store.

  • by k0k0r0 on 9/7/23, 7:39 PM

    Kagi appearently had a project "expertGPT" (contrast to FastGPT). Does anybody know what happened to that one?

    On a side note, there is now a - to my best knowledge - completely unrelated product "ExpertGPT" from some totally different company. I am not talking about that one.

  • by jcul on 9/8/23, 8:59 AM

    This is really cool.

    Second link I found: https://ambience.sk/quotes-from-books-the-universe-maker/

  • by caskstrength on 9/7/23, 3:58 PM

    And yet in the article they reference their Discord channel and Twitter sigh

  • by tailspin2019 on 9/7/23, 4:11 PM

    I could already tell I was going to like this from the title alone!

  • by fLaMEd on 9/7/23, 10:51 PM

    This is great to see, what a fantastic addition to Kag,i i.

    Yeah I see similarities to marginalia, but it’s great to have multiple services for the small web.

    I need to get my website on the lists asap!

  • by replwoacause on 9/8/23, 3:18 AM

    This is great. Just wanting to chime in that I am also a happy paying customer, and love seeing this kind of work from Kagi. Keep it up!

  • by Gud on 9/7/23, 8:50 PM

    Thanks a lot for this. I really appreciate kagi and the work you put in to it.

    Now more than ever do we need a user friendly search engine.

  • by 58ok5 on 9/7/23, 8:26 PM

    Okay, it's great, and now I'm existed >> Now how can I raise those results in my personalized results?

  • by bennyp101 on 9/7/23, 4:19 PM

    Pretty cool - just wondering why one of the requirements is "The website can appear in an iframe"?

  • by MatthiasPortzel on 9/7/23, 5:41 PM

    This sort of centralized, hand-curated list makes me nervous. The beauty of the web is that it’s open, you don’t have to “apply” or meet “criteria” for your website to viewable by other people. Obviously the status-quo (of Google Search) isn’t great, and I’m glad Kagi is trying to fix this problem. But this isn’t going to scale.

  • by 58ok5 on 9/7/23, 8:27 PM

    Great initiative, now how can I raise those results above all my already raised domains?

  • by gcoguiec on 9/7/23, 4:12 PM

    Very excited about this!

  • by victorbjorklund on 9/7/23, 8:35 PM

    This is pretty cool!

  • by nsonha on 9/8/23, 12:53 AM

    so a search engine "handpicked" internet content for you? We already have social network for that.

  • by sdm on 9/8/23, 12:22 AM

    > Content must be in English (hard to curate non-English).

    Well this is disappointing. It's no harder to curate other languages. You're just say you don't care.