from Hacker News

Private Cloud Compute: A new frontier for AI privacy in the cloud

by serhack_ on 6/10/24, 9:53 PM with 373 comments

• by hexage1814 on 6/11/24, 5:41 AM

  The thing with cloud and with anything related to it, anything that connects to the internet somehow... is that, unless it's open source and the servers decentralized, you are always trusting SOMEONE. Sure, Apple might make their best to ensure nobody – but them – have access to your data... but Apple controls all the end points. It controls the updates your iPhone receives, it controls the servers where this happens. Like, they are so many opportunities for them to find what you are doing. It reminds me of this article "Web-based cryptography is always snake oil"

  https://www.devever.net/~hl/webcrypto

  And to be fair, this doesn't apply only to this case. Even the data you have stored locally, Apple could access it if they wanted, they sure have power to do it if they so wish or were ordered by the government. They might have done it already and just didn't told anyone for obvious reasons. So, I would argue the best you could say is that it's private in the sense that only Apples knows/can know what you are doing rather than a larger number of entities .

  Which, you could argue it's a win when the alternatives will leak your data to many more parts... But still far away from being this unbreakable cryptography that it's portrayed it to be.

• by loteck on 6/10/24, 11:55 PM

  Some good comments on this from cryptographer Matt Green here: https://x.com/matthew_d_green/status/1800291897245835616?t=C...

  (I wonder if Matt realizes nobody can read his tweets without a X account? Use BlueSky or Masto man)

  Edit: here's his thread combined https://threadreaderapp.com/thread/1800291897245835616.html?...

• by zmmmmm on 6/11/24, 3:56 AM

  Read through it all, it still comes down to "trust us". Apple can sign and authorise an update at any time that will backdoor it, and the government is the stroke of a pen away from forcing them to, all completely silently.

  I get that there's benefit to what they are doing. But the problem of selling a message of trust is you absolutely have to be 100% truthful about it, and them failing to be transparent that people's data is still subject to access like this poisons the larger message they are selling.

• by kfreds on 6/11/24, 8:52 AM

  Wow! This is incredibly exciting.

  Apple's Private Cloud Compute seems to be conceptually equivalent with System Transparency - an open-source software project my colleagues and I started six years ago.

  I'm very much looking forward to more technical details. Should anyone at Apple see this, please feel free to reach out to me at stromberg@mullvad.net. I'd be more than happy to discuss our design, your design, and/or give you feedback.

  Relevant links:

  - https://mullvad.net/en/blog/system-transparency-future

  - http://system-transparency.org (somewhat outdated)

  - http://sigsum.org

• by Shank on 6/10/24, 11:25 PM

  > In a first for any Apple platform, PCC images will include the sepOS firmware and the iBoot bootloader in plaintext, making it easier than ever for researchers to study these critical components.

  Yes!

  > Software will be published within 90 days of inclusion in the log, or after relevant software updates are available, whichever is sooner.

  I think this theoretically leaves a 90-day maximum gap between publishing vulnerable software and potential-for-discovery. I sincerely hope that the actual availability of images is closer to instant than the maximum, though.

• by ein0p on 6/11/24, 1:52 AM

  It is not possible for this to be fully private in the United States because the government not only can force Apple to open up the kimono, it can also forbid it to talk about it. There’s not really anything Apple can do to work around this “limitation”. Thank your “representative” for extending the PATRIOT Act when you get a chance.

• by zer00eyz on 6/11/24, 12:16 AM

  I have a big question here.

  Who is this for? Dont get me wrong I think it's a great effort. This is some A+ nerd stuff right here. It's speaking my languge.

  But Im just going to figure out how to turn off "calls home". Cause I dont want it doing this at all.

  Is this speaking to me so I tell others "apple is the most secure option"? I don't want to tell others "linux" because I don't want to do tech support for that.

  At this point I feel like an old man shouting "Dam you keep your hands off my data".

• by WatchDog on 6/11/24, 4:44 AM

  I'm interested in how this compares to AWS nitro enclaves, which they mention briefly.

  The main difference seems to be verifiability down to the firmware level.

  Nitro enclaves does not provide measurements of the firmware[0], or hypervisor, furthermore they state that the hypervisor code can be updated transparently at any time[1].

  Apple is going to provide images of the secure enclave processor operating system(sepOS), as well as the bootloader.

  It also sounds like they will provide the source code for these components too, although the blog post isn't clear on that.

  [0]: https://docs.aws.amazon.com/enclaves/latest/user/set-up-atte....

  [1]: https://docs.aws.amazon.com/pdfs/whitepapers/latest/security...

• by advael on 6/10/24, 11:30 PM

  I really want to see this OS, and have cautious optimism that this could be the first time we'll see a big tech company actually provide an auditable security guarantee!

  I think depending on how this plays out, Apple might manage to earn some of the trust its users have in it, which would be pretty cool! But even cooler will be if we get full chain-of-custody audits, which I think will have to entail opening up some other bits of their stack

  In particular, the cloud OS being open-source, if they make good on that commitment, will be incredibly valuable. My main concern right now is that if virtualization is employed in their actual deployment, there could be a backdoor that passes keys from secure enclaves in still-proprietary parts of the OSes running on user devices to a hypervisor we didn't audit that can access the containers. Surely people with more security expertise than me will have even better questions.

  Maybe Apple will be responsive to feedback from researchers and this could lead to more of this toolchain being auditable. But even if we can't verify that their sanctioned use case is secure, the cloud OS could be a great step forward in secure inference and secure clouds, which people could independently host or build an independent derivative of

  The worst case is still that they just don't actually do it, but it seems reasonably likely they'll follow through on at least that, and then the worst case becomes "Super informative open-source codebase for secure computing at scale just dropped" which is a great thing no matter how the other stuff goes

• by yla92 on 6/11/24, 12:38 AM

  > And finally, we used Swift on Server to build a new Machine Learning stack specifically for hosting our cloud-based foundation model.

  Interesting to see Swift on Server here!

  https://www.swift.org/documentation/server/

• by nardi on 6/11/24, 7:15 AM

  Many people in this thread are extremely cynical and also ignorant of the actual security guarantees. If you don’t think Apple is doing what they say they’re doing, you can go audit the code and prove it doesn’t work. Apple is open sourcing all of it to prove it’s secure and private. If you don’t believe them, the code is right there.

• by v4dok on 6/11/24, 10:31 AM

  This is Confidential Computing https://en.m.wikipedia.org/wiki/Confidential_computing

  with another name. Intel, AMD and Nvidia have been working for years on this. OpenAI released a blog some time ago where they mentioned this as the "next step". Exciting that Apple went ahead and deployed first, it will motivate the rest as well.

• by ramesh31 on 6/11/24, 2:13 AM

  Here's the answer to the "what's taking Apple so long to get on the LLM train?" folks. Per usual, they lag a bit and then do it better than anyone else.

• by piccirello on 6/11/24, 1:04 AM

  > The Secure Enclave randomizes the data volume’s encryption keys on every reboot and does not persist these random keys, ensuring that data written to the data volume cannot be retained across reboot. In other words, there is an enforceable guarantee that the data volume is cryptographically erased every time the PCC node’s Secure Enclave Processor reboots.

• by tzs on 6/11/24, 11:41 AM

  > The Secure Enclave randomizes the data volume’s encryption keys on every reboot and does not persist these random keys, ensuring that data written to the data volume cannot be retained across reboot. In other words, there is an enforceable guarantee that the data volume is cryptographically erased every time the PCC node’s Secure Enclave Processor reboots.

  I wonder if there is anything that enforces an upper limit on the time between reboots?

  Since they are building their own chips it would be interesting to include a watchdog timer that runs off an internal oscillator, cannot be disabled by software, and forces a reboot when it expires.

• by j0e1 on 6/11/24, 1:00 AM

  > The Apple Security Bounty will reward research findings in the entire Private Cloud Compute software stack — with especially significant payouts for any issues that undermine our privacy claims.

  Let the games begin!

• by bayareabadboy on 6/11/24, 12:46 AM

  What are the longer term implications that Apple is doing this on their own hardware and not Nvidia? This seems like a big thing to me, an idiot.

• by ethbr1 on 6/10/24, 10:32 PM

  This entire platform is the first time I've strategically considered realigning the majority of my use to Apple.

  Airtag anonymity was pretty cool, technically speaking, but a peripheral use case for me.

  To me, PCC is a well-reasoned, surprisingly customer-centric response to the fact that due to (processing, storage, battery) limitations not all useful models can be run on-device.

  And they tried to build a privacy architecture before widely deploying it, instead of post-hoc bolting it on.

  >> 4. Non-targetability. An attacker should not be able to attempt to compromise personal data that belongs to specific, targeted Private Cloud Compute users without attempting a broad compromise of the entire PCC system. This must hold true even for exceptionally sophisticated attackers who can attempt physical attacks on PCC nodes in the supply chain or attempt to obtain malicious access to PCC data centers.

  Oof. That's a pretty damn specific (literally) attacker, and it's impressive that made it into their threat model.

  And neat use of onion-style encryption to expose the bare minimum necessary for routing, before the request reaches its target node. Also [0]

  >> For example, the [PCC node OS] doesn’t even include a general-purpose logging mechanism. Instead, only pre-specified, structured, and audited logs and metrics can leave the node, and multiple independent layers of review help prevent user data from accidentally being exposed through these mechanisms.

  My condolences to Apple SREs, between this and the other privacy guarantees.

  >> Our commitment to verifiable transparency includes: (1) Publishing the measurements of all code running on PCC in an append-only and cryptographically tamper-proof transparency log. (2) Making the log and associated binary software images publicly available for inspection and validation by privacy and security experts. (3) Publishing and maintaining an official set of tools for researchers analyzing PCC node software. (4) Rewarding important research findings through the Apple Security Bounty program.

  So binary-only for majority, except the following:

  >> While we’re publishing the binary images of every production PCC build, to further aid research we will periodically also publish a subset of the security-critical PCC source code.

  >> In a first for any Apple platform, PCC images will include the sepOS firmware and the iBoot bootloader in plaintext, making it easier than ever for researchers to study these critical components.

  [0] Oblivious HTTP, https://www.rfc-editor.org/rfc/rfc9458

• by krosaen on 6/11/24, 12:56 PM

  I wonder if they will ever make this available to developers - I can think of many products that would be nice to have at least part of the cloud infra being hosted in a trusted provider like this, e.g indoor cameras for health metrics: sounds awesome but I would never trust a startup to handle private data this sensitive.

• by paul2paul on 6/11/24, 6:47 AM

  We don't need "a new frontier". I want to be the only one who holds the private key to my encrypted data. I think it's pretty lame to sell privacy when it's not.

• by tiffanyh on 6/11/24, 2:56 AM

  I wonder who Apple will be colocating with for data centers.

  And what the PCC chassis looks like for these compute devices (will it be a display-less iPad)?

• by jaydeegee on 6/11/24, 12:38 AM

  Outside of all the security aspects which look to be handled quite well on the surface I do enjoy that the client mainframe architecture is still a staple of computing.

• by vlovich123 on 6/11/24, 3:27 AM

  What I haven’t heard from the announcement is whether the private cloud has external network access. Presumably it wouldn’t otherwise the guarantees of your request staying in your cloud is meaningless. Conversely, a lot of trivial network stuff can be involved (eg downloading the model). Anyone know which balance Apple is choosing to strike initially?

• by dymk on 6/11/24, 1:48 PM

  I would love to be able to run a PCC node locally on my M2 MacBook or similar for my iPhone to offload to, even if it’s only for doing what 15 Pro iPhones can do on-device.

  There’s precedent for this sort of thing as well, like Apple TVs or iPads acting as HomeKit hubs and processing security can footage on-device.

  Maybe they’ll open that up in the future.

• by gigel82 on 6/11/24, 2:54 AM

  The only way to trust this is them selling "cloud compute" servers that folks can deploy and monitor in their own infrastructure. Nothing else can be guaranteed to not include malicious code to exfiltrate the data.

  Or better yet, make the APIs public and pluggable so that one can choose an off-device AI processor themselves if one is needed.

• by KETpXDDzR on 6/14/24, 2:46 AM

  The only way, besides trusting the cloud provider, is encryption. Homomorphic encryption allows you to run calculations on encrypted data without decrypting it. However, besides the performance penalty, it leaks information.

• by asp_hornet on 6/11/24, 7:24 AM

  This thread reads like a whole bunch of sour grapes. Hopefully this challenges other companies to do better

• by CGamesPlay on 6/11/24, 3:33 AM

  All of this is interesting, but how easy is this to circumvent? When Apple changes their mind for whatever reason, don't they just return a key to a fake PCC node, which would bypass all of their listed protections? Furthermore, what prevents Apple from doing this for specific users?

• by thomasahle on 6/10/24, 10:42 PM

  Did Apple say anything about what training data they used for their generative image models?

• by renegade-otter on 6/11/24, 8:29 PM

  I think the best AI business model is charging you to keep the data away from data collection. It's brilliant.

  Or "Professional" version of software that removes all those annoying "AI" features.

• by cherioo on 6/10/24, 11:56 PM

  Can some ELI5 how remote attestation is supposed to work? It feels like asking a remote endpoint “are you who you say you are”. What’s stopping remote endpoint always responding “yes”

• by croes on 6/11/24, 5:54 AM

  Who pays for the costs of private cloud compute, is it free of charge for the iPhone owner (at least until they turn it into a subscription)?

  What about second hand iPhone users?

• by EternalFury on 6/11/24, 4:48 AM

  Let’s not be too picky. This is a good thing.

• by SirensOfTitan on 6/11/24, 12:31 AM

  What I'm most curious about here is if a state actor comes to Apple with a subpoena and compels them to release information on an individual, what would Apple be able to release?

  ... I suppose this is ultimately a question that will be tested sooner or later in the US.

• by clipjokingly on 6/11/24, 2:10 AM

  Is it possible to have zero knowledge AI?

• by m3kw9 on 6/11/24, 1:19 AM

  I wonder how they will do this in china?

• by solarkraft on 6/11/24, 1:38 AM

  I was sceptical of the announcement, but this actually sounds really well thought out.

  One key part though will be the remote attestation that the servers are actually running what they say they're running. Without any access to the servers, how do we do that? Am I correctly expecting that that part remains a "trust me bro" situation?

• by candiddevmike on 6/10/24, 11:11 PM

  Did Apple need to license the phrase Core OS like iOS?

• by rldjbpin on 6/11/24, 5:58 PM

  not trusting any of the privacy/security mumbo-jumbo when their icloud free tier still allows for a paltry 5 gigs, when even google offers thrice as much for their public service.

  i am happy for those who see the positives here, but for the skeptic a toggle to prevent any online processing would be more satisfactory.

• by rmbyrro on 6/11/24, 12:13 AM

  Most ironic thing is they abbreviate this as "PCC". (reads chinese communist party in many languages)

  The absolute worst acronym for anything even remotely related to personal privacy.

• by system7rocks on 6/11/24, 5:54 AM

  I trust Apple

• by Havoc on 6/10/24, 11:30 PM

  Sounds good. Still won’t send anything sensitive there but I appreciate the effort and direction, especially when current industry trend seems to be fuck you were rewriting our TOS to take your data.

• by nerdright on 6/11/24, 3:18 AM

  Even if you don't like Apple's monopolistic approaches, you have to admire how they go an extra mile to stay true to their mantra of selling privacy.

  This is clearly a company with an identity, unlike Microsoft and Google who are very confused.

• by throwaway369 on 6/11/24, 1:14 AM

  [Deleted]

• by nisten on 6/11/24, 9:51 AM

  Complete horseshit marketing speak.

  Was the cloud non-private before? Was it not secure in the first place? Do my Siri searches no longer end up as google ads metadata now? Are the feds no longer able to get rubber stamp access to my i C L O U D now?

  You are a naive idiot for believing that this is anything but security theater to adress the emotional needs of AI anxiety in and outside the company.

  Just my opinion.

• by goupil on 6/11/24, 4:44 AM

  It's sad to see so many discussions on security and so little on privacy. How about solutions that could combine both, such as homomorphic encryption for AI?

• by whatever1 on 6/11/24, 3:44 PM

  Fyi this is the same company that has been accused of showing people's photos and videos in stranger people's devices by accident.

  https://discussions.apple.com/thread/252459254?sortBy=best

• by jeffbee on 6/10/24, 10:58 PM

  A lot of this sounds like Apple has been 10-20 years behind the state of the art and now wants to tell you that they partially caught up. Verifiable hardware roots of trust and end-to-end software supply chain integrity are things that have existed for a while. The interesting part doesn't come until the end where they promise to publish system images for inspection.