from Hacker News

Ask HN: What AI assistants are already bundled for Linux?

by foothebar on 3/1/24, 10:38 AM with 60 comments

Are there any AI assistants which come bundled with one of the popular distros (Debian-like, Arch, …)?

Or anything which is free (at least as in beer) and readily bundled in distro-specific installation packages?

  • by yjftsjthsd-h on 3/1/24, 3:09 PM

    NixOS has packages for llama-cpp and ollama: https://search.nixos.org/packages?channel=23.11&from=0&size=...

    Edit: And Arch packages ollama officially - https://archlinux.org/packages/?sort=&q=llama&maintainer=&fl... - and a few things in the AUR - https://aur.archlinux.org/packages?O=0&K=llama

  • by brucethemoose2 on 3/1/24, 2:31 PM

    One thing about LLMs is that they are 6GB+ (and much larger for "smart" ones) just sitting in the background. They suck power and produce heat like nothing else, and they are finicky, especially at smaller sizes.

    Running one as a background desktop assistant is whole different animal than calling a Microsoft API.

  • by gtirloni on 3/1/24, 12:00 PM

    None. Microsoft has Copilot in preview mode in Windows and it's not very integrated apart from a chat window. I doubt GNOME/KDE will be able to dedicate enough resources to adding an assistant that is well integrated with the desktop environment any time soon.

    A search in Fedora yields a single GSoC project[0] limited in scope to NetworkManager and it's not clear if anyone actually is working on that.

    If the use case you're interested in is actually having the LLM doing things for you in SaaS applications, that wouldn't need deep integration but, considering Google is yet to deliver a Google Drive client for Linux, I wouldn't hold my breath waiting for a native Linux AI-assisted assistant.

    Your best option right now is to interface with the assistants through their web interface and hope they have plugins/extensions to interact with things you want.

    Other than that, some people have built prototypes running LLMs locally that talk to things like Home Assistant. But again, no deep desktop integration.

    0 - https://docs.fedoraproject.org/en-US/mentored-projects/gsoc/...

  • by louismerlin on 3/1/24, 2:28 PM

    Not as far as I know, but llamafiles are almost easier to get running than native packages.

    https://github.com/Mozilla-Ocho/llamafile

  • by AlienRobot on 3/1/24, 2:16 PM

    Unrelated, but is there something like Bonzi Buddy for linux? Not the spyware part, just the friendly looking clippy-esque character that can tell you about your new e-mails, weather, or whatever? I kind of wish I had something like that.

  • by TheCoreh on 3/1/24, 2:53 PM

    Not bundled in a distro-specific way, but ollama fairly straightforward to install and use:

    https://github.com/ollama/ollama/blob/main/docs/linux.md

  • by gessha on 3/1/24, 2:44 PM

    Shameless plug: I’m writing one for Ubuntu Touch and I’m planning on adding a desktop version as well.

    My frontend side is very weak so it’s going to be very barebones but contributions are welcome once it’s stable:

    https://github.com/gessha/llmtest

  • by bossyTeacher on 3/1/24, 2:50 PM

    Why? They take lots of space and lots of computing power. Linux has always been about lightweight and a bundle only containing essential things. You can always install one if you need it but as it stands right now LLMs are not useful enough to warrant their bundling in a distro. Just my 2 cents

  • by 29athrowaway on 3/1/24, 2:25 PM

    There is one you can install via Flatpak. That will work in most distros.

    https://flathub.org/apps/io.github.Bavarder.Bavarder

  • by bzmrgonz on 3/2/24, 2:08 AM

    Wouldn't it be ironic if chatgpt assistant on linux gives linux end-user desktop dominance? lol... Microsoft would be like, "you fools, you've doomed us all"

  • by harishpillay on 3/3/24, 4:15 PM

    Perhaps this: https://github.com/yusufcanb/tlm?

    it is not distro bundled (yet), but I have it running on my Fedora Linux 39 running on a NUC with 16GB of RAM. Performance is good enough for me.

  • by quickthrower2 on 3/1/24, 11:45 PM

    Jan.ai has a deb package. https://jan.ai/

  • by throwaway8481 on 3/1/24, 2:24 PM

    Could they just package ollama and build a Python GTK interface around the CLI interface?

  • by mdev23 on 3/1/24, 3:24 PM

    Check out https://gitlab.com/mdev1974008/doda - Pure terminal UI. Its just python.

  • by javvee on 3/1/24, 2:58 PM

    Linux options (possibly not bundled) from another thread

    https://news.ycombinator.com/item?id=39533494

  • by onemoresoop on 3/1/24, 3:49 PM

    I'm wondering if there's any AI assistant that could run on a headless RPI? And is easy to install?

  • by xk_id on 3/1/24, 9:51 PM

    When Linux comes bundled with AI I will lose all my faith in humanity and never use a computer again.

  • by dartharva on 3/1/24, 3:56 PM

    It's a fair assumption ChromeOS will be getting Gemini soon.

  • by adr1an on 3/1/24, 2:19 PM

    pip install llm # among other, to run local or not. Yet, KDE or Gnome are yet to integrate or develop a nice API for/ to any of these.

  • by crimsoneer on 3/1/24, 4:06 PM

    lm studio has a good linux version and gives you everything you'd ever need.

  • by dartos on 3/1/24, 2:20 PM

    Open interpreter?