from Hacker News

Ask HN: How do you use AI for development in high security environments?

by thesurlydev on 5/19/25, 7:37 PM with 4 comments

I'm curious how this is done effectively assuming no source should be sent to a model hosted remotely. Are there foundational model toggles for using inference and embedding but not using data for training?

  • by daemonologist on 5/19/25, 8:20 PM

    All the big providers offer no-training/retention guarantees (either by default, or as a toggle, or upon request). For many high security environments though I'd expect everything to be hosted on-prem or at minimum on company-controlled instances, which does limit your model options somewhat.

    My employer has such contracts for some use cases, but actually forbids use of code completion/generation due to IP concerns.

  • by scarface_74 on 5/21/25, 12:54 AM

    I’ve worked with companies that would never trust publicly hosted models. But don’t have any issues with hosted models on AWS or Azure. But I work in cloud consulting so they already have to trust the cloud provider.

    Yes this includes GovCloud implementations that have citizenship requirements and you can’t connect outside of the US.

    I have not admittedly worked on any projects in the “secret” regions.

    https://aws.amazon.com/federal/secret-cloud/

  • by sky2224 on 5/20/25, 3:32 AM

    As someone else stated, there are enterprise services that offer solutions that make it so your company data isn't consumed, however, I think pretty soon we're going to see a lot of companies maintaining models locally in-house.

    I think this is especially true given that Intel is shifting its focus toward an affordable in-house solution for training AI models locally with its upcoming GPUs.

  • by 2rsf on 5/21/25, 1:12 PM

    Repeating what others have written based on my experience at the bank i work for- business offering will not use or save you data, and for more sensitive material we simply host it on prem