from Hacker News

Cleaning up my 200GB iCloud with some JavaScript

by amin on 1/4/24, 6:36 AM with 186 comments

• by asdaq1312512 on 1/5/24, 12:43 PM

  Photos.app doesn't show file size… so I thought, why not build a little Photos.app extension or a separate app that queries for large files?

  Turns out that the API doesn't expose "file size", at least I didn't find a straight-forward way.

  I think that all "photos" or "videos" are just a view of the underlying "photo or video object". If you crop a video, the full-size video will remain. Only if you export the video, it will be cropped and the smaller file size will manifest.

  I guess that's why the file sizes differ.

  [Edit: someone created an AppleScript to query file sizes - I didn't test it, yet: https://discussions.apple.com/docs/DOC-250000422 ]

• by mft_ on 1/5/24, 11:46 AM

  Okay, so it could be a bug, but it is possible that iCloud is secretly storing more than one version of the file in some cases? (After all, Apple does similar things with other media files.)

  The example given at the end is interesting:

  > So iCloud says the video is 128MB, I download it and the video is actually 48MB, and my free storage increases by ~170MB when I deleted it. Interesting!

  This suggests that iCloud isn't simply misrepresenting the size of the example file, as then you'd expect that deleting the 128MB file would clear ~128MB of iCloud space. Instead, the deletion clears roughly the space it reports (128MB) plus the space of the downloaded version (48MB): 128MB + 48MB = 176 MB - which might be close enough, allowing for rounding errors, as iCloud reports the free space (from the article's example) to the nearest 10 MB.

• by seffignoz on 1/5/24, 2:09 PM

  Thanks for the idea/solution!

  I converted this into a TamperMonkey/Greasemonkey script. Also added a feature to "hide" all elements that do not match the threshold.

  https://github.com/seffignoz/icloudcleanup

• by sandreas on 1/5/24, 12:24 PM

  That's one of the reasons I self-host. I just don't trust the cloud providers regarding clarity and transparency... even if my self-hosted solution was far less reliable, less secure and less performant (I imagine it's not ;), I probably wouldn't change.

  I personally use immich[1], a very complete solution with iOS / Android App, Server-Component and Sync / Backup option.

  [1]: https://immich.app/

• by InsomniacL on 1/5/24, 11:54 AM

  I'm pretty sure that iCloud stores the full version and downloads a lower quality optimised version to your phone.

  If you open iPhone settings and browse to:

    > Apple ID > iCloud  > Photos
  

  There is an option to 'Optimise iPhone Storage' which is enabled by default. This states:

    > If your phone is low on space, full resolution photos and videos are automatically replaced with smaller device sized versions. Full-Resolution versions can be downloaded from iCloud at any time. 
  

  This seems perfectly reasonable to me.

• by cianmm on 1/5/24, 10:53 AM

  Interesting article. I have my own problem, that caused me to have to upgrade my iCloud plan, and thus may not be a high priority fix for Apple.

  If you shoot RAW+JPEG (not a super-rare thing to do, for photo enthusiasts) then Apple Photos links the two images. Which is useful, rather than having a bunch of kinda-duplicates littering your library you can easily toggle between RAW and JPEG.

  But this combining, along with the file system design described in this article, makes it impossible (as far as I can tell, anyway) to easily separate them and delete the RAWs. So years later, I have HUGE RAW files that I'll never touch that I can't delete, because I want to keep the much smaller JPEGS.

  Any method that I've found to clean them up (exporting the originals, deleting them from the library, and then re-importing the JPEGs only seems easiest) will lose all of the years of metadata that I've built up in the library.

  So I have to upgrade.

• by cnicolaou on 1/5/24, 11:17 AM

  Surprisingly, I've received a similar storage notice from Apple before the holidays and I decided to download all photos/videos into my own media server instead of hosting it on iCloud. There is not an easy, straight-forward way to download your archive from iCloud. I am slowly getting there using multiple machines and devices.

  The issue with the recent changes with Apple is that they increase the prices for no good reason. We're always going to take photos/videos and their sizes keep increasing with modern tech and capabilities.

• by captn3m0 on 1/5/24, 11:52 AM

  The thing I detest the most is the weird tier that jumps from 200GB to 2TB. There’s no way to pay for incremental storage for 500GB or 1TB, which would be perfect for many families.

  In today’s age where storage is a commodity, this ought to be priced per GB used.

• by mmikeff on 1/6/24, 9:22 AM

  The thing that is taking up loads of my iCloud storage is the 'live' video version of photos where the tiny snippet of movement has a larger filesize than the photo that it accompanies. The only way I have found so far to deal with this is to download the files locally, delete from iCloud, delete the local videos and upload the remaining still images. This is time consuming, clunky and makes me nervous that I'll delete something or lose something important in the process. Is there anything out the that help to automate that process but where there is some level of editorial control so that I can identify and keep the handful of 'live' images that I might actually want to keep?

• by atlas_hugged on 1/5/24, 10:06 AM

  Am I reading this right?

  If this is widespread, this could be seen as apple bloating figures to push people to upgrade, which could lead to a lawsuit, no?

  IANAL

• by kossTKR on 1/5/24, 5:02 PM

  Tangent but does anyone else find iCloud sync random / slow?

  The smart thing about having both an iPhone and a Macbook should potentially be; snap a bunch of photos, instantly have them available on the computer - but no. Apple apparently chooses a random time depending on 100 factors to upload the photos in the next 30 minutes to 7 days.

  So you often have to airdrop a bunch of photos files completely invalidating the purpose of the sync function.

• by orenlindsey on 1/5/24, 11:23 PM

  Interesting, also, Apple should make a mid-tier of iCloud. If they had a 500gb (or 400 or whatever) they could have gotten this guy to pay more.

• by Alifatisk on 1/5/24, 12:40 PM

  I've tried iCloud, Onedrive, Google Photos. Out of all these three, I stuck with Google Photos on my Iphone.

  iClouds bad pricing and storage issues led me to switch to Google Photos, It's way better. The lack of native support on iOS can be a little cumbersome though.

  The only thing I wish these cloud providers offered was a way to deduplicate photos / videos. That would make my life so much easier.

• by alephnan on 1/5/24, 12:32 PM

  Don’t you remember how painful it was to import/export an mp3 file into an IPod?

  It was especially painful on a Windows PC.

  The biggest selling point of other mp3 players for me was the ability to transparently copy and paste files into the filesystem.

• by baxtr on 1/5/24, 5:59 PM

  Slightly off topic: anyone know how to backup iCloud data (mainly photos and videos, but also notes) to an outside platform?

  I’m an Apple fan, but still don’t like the idea of having all my digital life on iCloud only.

• by sccxy on 1/5/24, 12:18 PM

  iCloud backup is real mess.

  Free 5GB is not enough for backing up iPhone system data any more.

  Most insane is that if you remove some apps from backup, then it iCloud usage goes from 4.5GB to 4.6GB.

• by sliq on 1/5/24, 11:55 AM

  Can confirm, have some ideas: Apple uses different file formats "internally" and also different resolutions, maybe saving them all in parallel as "one" video, which might cause this issue. So the total space per image is actually the sum of original .mov, optimized .mp4 and thumbnail image.

  Also, (automated) version history might have caused this! Maybe (automatic) image optimiztion saved old and new versions of everything.

• by spockz on 1/5/24, 6:40 PM

  I’ve been on a similar hunt. My devices are littered with copies of videos that were shared via iMessage, WhatsApp, and other means. Each of these apps then store the copy locally which then gets backed up in iCloud.

  Unfortunately I haven’t found a way to automatically delete all the videos/photos that I send which still have the original in the photo stream. It would be awesome to be able to automate that.

• by Havoc on 1/5/24, 10:52 AM

  Good timing..I've been getting the same warning mails (though 50gb). Was planning to go through photos but checking vids first makes sense

• by sroussey on 1/5/24, 8:41 PM

  I need a good way to cleanup a gmail account…

• by LUmBULtERA on 1/5/24, 11:49 AM

  That's an interesting bug/problem(?) he ran into.. Though, on the topic of reducing storage usage, in the Photos iOS app, you can remove photo duplicates under "Albums" at the very bottom -- if you have duplicates that it recognizes, there will be a line item for this, otherwise the line item will disappear.

• by shreezus on 1/5/24, 10:39 PM

  On this topic...anyone know the best way to export an entire iCloud Photo Library at full resolution?

• by aledalgrande on 1/5/24, 1:40 PM

  In terms of iCloud cleanup, has anyone found a way to clean up iMessage attachments > x years old, greater than n MB or from certain contacts? It would clean up so much space for me. Apple only clears > 1 year old from the settings.

• by tamimio on 1/5/24, 5:53 PM

  > Videos anonymized by slow internet loading

  I chuckled!

  I believe the size difference has to do with the encoding, can’t tell for a fact since you didn’t show that part, but maybe it gets re-encoded when you download it hence the difference.

• by sneak on 1/5/24, 3:31 PM

  Apple interested in maximizing recurring subscription revenue?

  You don’t say. I’m sure this is just a harmless bug and not the source of $20M extra revenue per year across their hundreds of millions of active iCloud users.

• by Havoc on 1/5/24, 1:05 PM

  Just discovered you can see file sizes for some items under > icloud > recommended for you

  Finds duplicates, screenshots and big vids. Had a random vid that was 1.8gb for reasons unknown

  Doesn't always seem to be a accessible though

• by tugberkk on 1/5/24, 1:55 PM

  Apart from the iCloud fiasco, I very much like these kind of programming. We generally lost the fun of it try to make enterprise/corporate programs, we forgot to use them for personal tasks.

• by SirMaster on 1/5/24, 5:46 PM

  Can't you just use the Apple Shortcuts app to do a lot of this?

  Pretty sure that can enumerate and query things like the filesize of the photos and videos in your iCloud.

• by wouldbecouldbe on 1/5/24, 11:41 AM

  It would make sense if they save the video in a few formats for being able to load them fast in the Apple apps, most video hosts do the same.

• by BigBalli on 1/5/24, 6:10 PM

  Was the video with size mismatch edited? Apple keeps both versions (same goes for photos, they include image and MOV for live photos).

• by znpy on 1/5/24, 1:02 PM

  I'm being sarcastic here, but the solution could be quite simple: iCloud is billing you (both in terms of storage size and money) for both your data and the redundant copies.

  Other companies would build the redundancy costs into the final pricing, but Apple is known to "think different".

• by very_good_man on 1/5/24, 6:29 PM

  Apple needs to face criminal penalties for the dark patterns with which they prey upon their users.

• by tamiral on 1/5/24, 4:25 PM

  there is something weird happening on iCloud, my phone storage is off even though the total size of data isnt as big as it says, i throw a lot of .zip/rar files on icloud and it seems like the data on icloud is more than it is on my HDD....

• by NDizzle on 1/5/24, 2:34 PM

  With the iCloud desktop app for windows I'm able to sort by size...

• by FaridIO on 1/5/24, 2:03 PM

  Additionally, if you take a 4MB photo and add it to a shared album with 5 people, those 5 people all have to use up 4MB from their storage. Instead of reusing the same image and having pointers to it, it looks like it's "copied".

• by godzillabrennus on 1/5/24, 7:33 PM

  It seems like a giant class action lawsuit with a juicy target if Apple is overbilling customers on usage for videos in iCloud. I know lots of people paying for more storage in iCloud would appreciate the reprieve of lower costs.

• by lovegrenoble on 1/5/24, 12:41 PM

  It's better to self-host

• by raz32dust on 1/5/24, 10:10 PM

  I am so tired of these patterns. Google Photos also does not allow any easy or good way to clean up junk. Why would they? They want you to run out of space so you can pay for their cloud storage. They also have low incentive to keep photo/video storage space-efficient for the same reason.

• by dthakur on 1/5/24, 11:16 AM

  Compression?

• by 867-5309 on 1/5/24, 4:55 PM

  some possible reasons to investigate:

  1. older videos have a larger file size because they are encoded less efficiently (h264 vs h265)

  2. gigabytes vs gibibytes (like how a 500GB drive formats to ~465 GiB)

  3. rounding errors

  4. Apple fucking its customers

  5. combo of the above

• by saagarjha on 1/5/24, 11:07 AM

  I kind of hate opaque cloud solutions like this, because their storage usage is inscrutable and their interfaces are basically never optimized to help you clear things out. Whether that’s intentional and malicious or just not a priority, I don’t really care, I just want to be in control of the stuff I’m using.

  I have a very similar issue with iMessage in iCloud, and unlike photos, there’s no web interface. You have to interact with it on your device. Thankfully I have a Mac so I can load up the chat database from there and see what’s using the space, but cleaning it up has been a nightmare: I have a script to find the big attachments, but if I delete them then some agent that manages the database goes into a loop for like five minutes per deletion. Then the change gets synced to iCloud (at least, I assume). So unless I fully reverse what the deletion process does and whether it is possible to do a batch operation I’m basically at the mercy of the front end they provide to clean things up, which as I mentioned earlier is absolutely not designed to make it easy to do this.

• by landswipe on 1/5/24, 11:21 AM

  It's called a bug 'turning a blind eye'.. ie. a scam.

• by crossroadsguy on 1/5/24, 3:26 PM

  There’s will be a time when there will be many open (or relatively more open) devices like FairPhone et cetera; and OSes that do not track users and suffocate you with dark patterns and hostility all around - like Graphene and Lineage; and hopefully they’d all be able to join their hands together and not reimplement the wheels of N types for the 1000th time all incompatible with each other. Hope is that there will be services and softwares on top of that spec and API of these OSes that will be the user’s choice to pick - for contact, for calendar, SMS, IM, call , media, sync, backup, device account management etc.

  And such devices will be widely available, also in non-first world countries, with proper OEM warranties and support. Hell, a local manufacturer can just build for that spec.

  I know it’s like a wishful dream. But if this happens, and when this happens, I hope a lot of us will be able to breathe free and hopefully would be candidly able to shit upon legacies of those so called fucking visionaries, who were barely not subhuman and were just rather pathetic jerks, who ensured such shit-show of walled gardens and opaquely implemented utterly inferior systems where users go and get stuck in one of the two houses of the duopoly to experience a lot of shitty things including, but not limited to, some variation of Stockholm syndrome and clear and conscious apologism.

  Why? Because neither of the two is acceptably good and they have become so big and they have closed it down so much that nobody else can even make a dent even if they try. And they try!

  So yeah, until then I will rant and feel shitty about both my iPhone 14 (as they call it - my “daily driver”) and Pixel 5a (my bread and butter phone; form factor wise less shitty one though).