from Hacker News

Ask HN: PCIe cards with their own NIC/Storage/OS?

by snoopy_telex on 12/5/22, 11:20 PM with 8 comments

I'm looking for a PCIe card that runs its own linux os and can talk with the host system. Ideally, it would have its own NIC and NVME interface for storage. Does anything like this exist?

  • by Avlin67 on 12/5/22, 11:49 PM

    what you want is a DPU. you can even run storage and network on pcie switches without a host CPU !

    https://www.servethehome.com/zfs-without-a-server-using-the-...

  • by LargoLasskhyfv on 12/5/22, 11:45 PM

    Many larger FPGA-development kits. NVME for storage maybe not right now, but as an add-on via 'Mezzanine' card atop the dev-kit.

    Basically some dual- to quad-core arm embedded into the fpga, various amounts of memory, sometimes expandable via SODIMM, and so on.

    NOT cheap!

    https://www.xilinx.com/products/design-tools/embedded-softwa... would be cybertrash from Xilinx as offer to run on the ARM-Cores.

    edit: Former Altera, now Intel-FPGA isn't really better.

    Prepare for a world of hurt, if doing anything with FPGAs, and their proprietary tool chains. Prepare for another world of hurt, if thinking they would be supported by open-source tool chains in a sufficient way, or that the the few alternative solutions which are fully supported by them would be enough.

    I'll show myself out now...(rambling)

  • by johndoe0815 on 12/5/22, 11:24 PM

    Perhaps not exactly what you are looking for, but there are Smart NIC PCIe cards running Linux (usually for data center applications), see e.g. these links:

    https://datatracker.ietf.org/meeting/105/materials/slides-10...

    https://legacy.netdevconf.info/0x14/pub/slides/39/Netdev%200...

    https://smartnicssummit.com/proceeding_files/a0q5f000000lcIn...

  • by LinuxBender on 12/5/22, 11:25 PM

    Does anything like this exist?

    They do but many are closed source. Dell and HP have their own ilo/drac cards that run Linux but they have since locked them down. A while back someone here linked an unrestricted ilo card, I will see if I still have the link.

    [Edit] I believe this [1] is what someone linked here a while ago. In the embeded video, he talks about some other more inclusive options.

    [1] - https://hackaday.com/2022/09/20/raspberry-pi-grants-remote-a...

  • by floatinglotus on 12/6/22, 12:09 AM

    You want a smartNIC/DPU, and you should probably run SONiC on it! (DASH)

  • by btgeekboy on 12/6/22, 12:59 AM

    Have a look at the CCR2004-1G-2XS-PCIe from MikroTik. It’s a router on a PCIe card, with its own OS.

    What’s your usecase? Might be able to get better recommendations if you can tell us what you’re trying to do.

  • by wmf on 12/6/22, 1:50 AM

    I recommend the SolidRun SolidNET or Mellanox Bluefield. FPGAs cost a fortune and Mikrotik looks janky.