Photos does a lot of extra work on import (merging RAW+JPEG pairs, generating previews, database indexing, optional deletion), so my guess is a concurrency bug where a buffer gets reused or a file handle is closed before the copy finishes.
Rare, nondeterministic corruption fits the profile.
They constantly ask for an example project, even if it's something that is easily demonstrated, simply by running existing Apple software, and creating a project, would be a huge pain.
They also ignore reports. Very rarely, I may get a ping on one of my reports, asking me to verify that it was fixed in some release. Otherwise, there's no sign that they ever even read it.
I usually end up closing my bug reports and feature requests, after a few months, because I'm tired of looking at them.
It's clear that they consider every bug report to be a burden. That's a very strange stance, but then, they are not a typical company.
I guess you can't argue with the results, as they have a market value North of 3 trillion dollars, but that does not make it any less annoying.
Which means that if that bug has been present since the (now unsupported) Mavericks, tough luck!
I would think the diffs would be telling to the right people.
It's on the front page of HN, so that's a good start!
What's the point of it? It is well known in the industry they ignore bugreports.
Also, this bug doesn't affect the majority of users, so it won't ever be fixed.
Random is random, and random is clumpy, so maybe swapping parts is irrelevant, but... I wanted more detail how often the corruption happened throughout his replacement journey.
edit: also wth i just realized I went to "tenderlovemaking.com" at work. gross. lol.
> I stopped checking the “delete after import” button
Edit: Nevermind, the contents are vastly changed. This is like a different stream of input got used, or a buffer was written over with contents from another image.
”Glad” to see it was an actual bug.
It’s more likely that things will be reversed: the old, battle-tested framework may have bugs, but it’s is less likely to have serious ones.
They should try to hunt down bugs in the existing code. A partial rewrite of parts that historically have many bugs may be in order, but a complete replacement? Unlikely to be an improvement.
A better way to have further narrowed down the problem down to Actually iPhoto would've been to do the same tests with a USB-C card reader plugged directly into the Mac, which would've eliminated cables, hubs, and camera hardware/software/firmware as possibilities.
It's worth noting that searches show that OM-1 USB support is imperfect, the camera manual addresses that "USB transfers aren’t guaranteed in some setups", and user consensus seems to be to use a card reader for reliable file transfers.
That is a... pretty damning thing to have in your user manual. I've owned many cameras over the years, and I don't think I've ever seen a manufacturer hedge basic functionality like this.
> Data transfer is not guaranteed in the following environments, even if your computer is equipped with a USB port.
>> Computers with a USB port added by means of an extension card, etc., computers without a factory-installed OS, or home-built computers
Manual for version 1.7 page 286
https://download.omsystem.com/pages/inst/om1/manual_om1_v1.7...
Never use the camera over USB, the experience is terrible on everything from Canon to Sony to Panasonic to Fuji.
Don't fight it, just buy that $30 USB hub and get on with your day.
But good gravy that troubleshooting path got expensive real fast. Replacing the laptop and the camera? Why not start by trying something other than Photos? It doesn’t even need to be a paid product; the Olympus software is free not to mention a good baseline since it - of all the applications - should be able to import photos without corrupting them.
Edit to add: delete on import seems pretty risky. My workflow is to import and only delete from the camera after 1) the imported photos are backed up 2) I’ve done a first pass culling.
I may be paranoid because I used to handle footage for VFX pipelines and you just do not mess around with those kinds of files. If you lose footage, you are in big trouble.
I had one case where I screwed up a shoot and thought file corruption might have been involved (it wasn't) Even though I had formatted the card with the camera and shot maybe 5 test shots I was able to recover most of the images with Disk Drill
https://www.cleverfiles.com/data-recovery-software.html
which has both Windows and Mac versions and looking at a sample of them confirmed it wasn't corruption, it was user error.
https://www.reddit.com/r/audioengineering/comments/mftc0g/ge...
It doesn't strike me as different from "porn" i.e. "unix porn" "food porn" etc, which are at least somewhat widely accepted. I assumed it was self-aware/deprecating humor, as in the people there recognized they were frequently replacing which gear they used beyond what might be strictly necessary.
It was colorful, in the way a lot of music and art is colorful. It's not like it's a sysadmin forum...
It's like, we collectively prioritize efficiency over fun and then we wonder why life is not fun even though it is efficient.
It’s understandable why they changed their name.
I don’t even want to know what ZScaler thinks of “tender love making”.
Ruby's community has been all sorts of whimsy and quirky over the years.
I very much enjoy Tenderlove being the community figurehead he is as is: kind, empathetic, genuine, open-minded, and generally wholesome.
> it's just awkward with interns to explain.
I've never had any trouble talking about Aaron Patterson a.k.a Tenderlove to coworkers, interns or otherwise.
Also when you die that stuff'll go offline pretty quick I expect...
I wonder if there are any viable alternatives though.
For the future, Graphene OS devs have stated publicly that they're working with an unnamed hardware vendor to develop a phone that will meet their list of hardware requirements. Currently only the Pixel line does. From what I understand, a few Samsung phones come close, but don't support bootloader re-locking... When you unlock Samsung bootloaders it burns out a fuse on the board which in turn completely disables Knox, their architecture for a trusted execution environment.
I suspect banks won't ever be able to take their web portals down and go app-only, though Google is now trying to ram through technologies in the Chrome browser to "verify the computing platform" that will have a similar effect to the Google Play "integrity" checks for apps.
Enduring solutions to these vendor lock-in efforts must ultimately be legislative.
That being said, I mostly receive email, and the privacy benefits of running my own server would still be significant even without the ability to send email at all.
You can host it yourself, and it has quite a range of features.
I am a happy self-hosting user.
I was surprised nextcloud has a whole bunch of ai plugins
https://apps.nextcloud.com/categories/ai
funny, I run nextcloud but don't add all these plugins because they require* you to install from the cloud.
* there's a way to install apps locally, but you had to install the app store and it quickly became very complicated.
Use third party apps/services which usually function on interoperable standards/specs.
It's been years since I have used any service by either of these companies where my personal data stays inside their ecosystem - email, notes, pics, videos et cetera.. nothing.
I have since turned off iCloud Photo Library, downgraded iCloud (no longer needed so much storage), and started using fully open source photo management with flat files on disk.
I’ve used Olympus cameras for over a decade. Well, the same camera to be honest, a PEN E-PM2. This has only appeared in the past couple of years.
I haven’t seen it on photos from my Canon EOS 80D yet, but I guess it’s time to change my workflow. And maybe OS.
Though, considering the macOS 26, it’s likely the Photos app.
Tangential story - 12-13 years ago I was a burgeoning and super eager software dev that moved to Seattle to be closer to "the scene." tenderlove's content was a major reason for me going there and I poured through his posts learning way too much about Nokogiri, Active Record, and much much more.
I went to every Ruby meetup I could get to out there and I remember one in particular, a Seattle RB meetup, in the Substantial office. It was a pretty small group, at most 15-20 people.
I was with a coder buddy but knew nobody else. We were all just drinking pints of Manny's beer and eating pizza from Big Mario's or something. Ryan Davis (the creator of minitest among other things) was doing a presentation on Unicode.
Aaron Patterson (tenderlove) was cracking jokes at every opportunity. At one point I asked a relatively naive question and Aaron _tenderly_ answered in joke-form response. I felt such a _part_ of the scene then. Aja Hammerly was super engaged in the presentation, I think even Ryan Bates and/or Geoffrey Grosenbach were there.
It was quite surreal to be in this dream-like state around giants and heroes just doing what they were doing and being so inclusive. It seemed so normal but became a core memory.
Thanks for everything Aaron, you've truly been an inspiration!
Also a good idea to copy to multiple locations when importing. When I do professional work and import into Lightroom from SD card I have it set to create two copies - import to my external SSD (the "working" copy) and also copies the files to my NAS (which is then backed up to the cloud).
Nowhere in that process do I ever delete anything.
They finally recognized there is an issue, but there is no fix, as of a few weeks ago :(
I never need to import anything when I can simply copy the data from the card.
For the longest time my process while traveling was importing onto my iPad or occasionally my iPhone since I didn't have a personal laptop, just a Mac Mini at home.
All my photos are managed using Digikam and developed using Darktable. They are also visualized via immich, but immich only has access via a read-only mountpoint.
Everything is hosted locally of course.
Ensuring ZFS has at least 2 copies on physically separate disks and using scrub frequently is the way, right?
Please correct me if I am wrong HN!
Sure security is important but integrity is too.
* iTunes/Music app randomly reassign my Album artwork, with different (incorrect) art showing up on different devices!
* Reminders app: shared reminder lists can end up with the name of a different list
* Ghost photos that are deleted from my phone, and come back later.
* Maps, when I say "navigate to $friend" set a route that ended in my own driveway.
To me, these bugs suggest a fundamental design flaw, perhaps they are using a simple Integer as an index rather than a UUID?
Or maybe the database schema are solid, but there's some sort of race condition in their synchronization frameworks and the data is getting scrambled in RAM?
Whatever it is, it's absolutely insane that in 2025 these kinds of bugs are happening.
I still use Macs because data on a physical disk seems perfectly reliable, but I've been bitten by so many of these bugs in their apps. iCloud files completely disappear, then reappear a day later. Highlight a couple chapters of a PDF in Preview, then reopen the file and they're gone because iCloud thinks the older unhighlighted version is newer or something. Madness. I don't touch any of these Apple services/apps anymore.
There's very clearly a fundamental bug in whatever sync framework they seem to share across everything. It's bad enough to have data disappear entirely or deleted data reappear, but then when data shows up in the completely wrong place, and this has been happening for years and years and still isn't fixed... I don't know what to think.
You're right. There's no other word for it but "insane". They can engineer their A-series and M-series microchips, but it's been over a decade now and their sync is still fundamentally broken.
I then subscribed to Apple Music and relied on its matching function. After switching from an Intel Mac to an M2 and redownloading my library from remote, it now believes that each and every song in my library are rented Apple Music copies. Even those it shows as having been added in 2003.
Some songs are missing; some go missing, then inexplicably come back months later. Worse: so far I have found around a dozen which have been replaced by different versions.
It's a real mess.
* prompts in settings for adding an account recovery contact that never go away, even after months and months of successfully setting it up multiple times.
* OS account profile picture can barely stay associated with the most recently picked option. Happens for non-iCloud local accounts on Mac, happens when I change profile pictures on iOS for iCloud… weird.
* OS account update screens on iPad, iOS, and watchOS will forget that they are in the middle of updating if you navigate away from the settings screen. Thankfully, today they at least recover from it (it’s probably still happening in the background), but it takes several long seconds of spinning for the settings page to remember that it was doing an update two seconds ago before I navigated away from it.
* similar to your ghost pictures bug, deleting a large media file from a media player app moves it to recently deleted, but you can sometimes end up in situations where you can’t permanently delete the file, or it doesn’t show up anywhere but still takes up space. (Talking about 20GB-80GB file sizes where it makes a big difference on OS storage space)
Some of these bugs have been around for a VERY long time.
But the weird thing is I don’t see them in 3rd party apps.
Not sure when exactly that changed, but it was probably a few OS releases ago?
The Safari reading list can't even sync properly between devices for me. Image Capture ("Keep Originals"??) or AirDrop is a little minimal for such a keystone part of the phone -> computer if you don't want to use Apple ecosystem after.. Let alone the other more complicated issues.
How do you know? Why do you believe that they're competent on writing security code but not competent enough to write a general purpose app? Is there a different company culture applied to the latter?
Is that a necessary qualifier? I used to get that impression, but on the outside it's gradually become a rarely believable pitch. Without having an iPhone and without having an Apple Watch, and without having already had them years ago, it just seems like I've sort of made the right choice with just mac over the years, and with the latest OS that's becoming just a tiny bit more questionable; their decision making with software seems sus.
Like I've never had to qualify my setup of using a mac for work, Android phone for phone, and I guess Audio Technica for headphones. It's not super nerdy, it's not super integrated, but if I wanted it to be super integrated, "what value would I get out of steeping myself into the Apple ecosystem further" is the question that comes to mind. I also have an old iPad that I tried to make useful, and the iPod nano 3rd gen which was actually amazing, but ultimately was hampered by software limitations that they don't seem to have advanced on much in 10 years. I've always found their discrete hardware products to be amazing in terms of industrial design, but they've never really been compelling in terms of their utility.
The software engineering standard at Apple has clearly tanked in the last decade, which is sad because the exact opposite appears to have happened to their hardware.
Used to be, these were full software engineers embedded with dev teams, with a mission to destroy, document, and harden the apps and frameworks.
During the 2010s in all the FAANG that I’m aware of (have worked at 3), QA as a high paid American profession was completely offshored to India and responsibility for quality removed from developers concern. It’s a blocking item on the Launch Checklist. Automated testing was expected to fill the gap but has mostly been ignored.
So they do a terrible job from a data security point of view;)
Recently when I load new music onto my phone I find that random unrelated album art has been mangled or switched with other albums from other artists. And some music, which exists on my phone's hard drive, is now greyed out and when clicked says "This item is not currently available in your country or region." I am considering switching back to a iPod with an upgraded drive and giving up on keeping music on my phone completely.
Anything important should be kept inside the file. Filesystem metadata gets lost all the time, isn't consistent between operating systems, zipping up a folder and extracting it will probably mess up timestampts too.
Dropbox doesn't seem to keep timestamps properly either.
I like using filesystem timestamps to sort through things in Finder, and thankfully I like A Better Finder Attributes for being able to batch copy EXIF data into timestamps.
[1] https://www.publicspace.net/ABetterFinderAttributes/index.ht...
Something related: exporting originals from Photos used to give the current timestamp back in Ventura, which annoyed me to no end.
They fixed that bug in either Sonoma or Sequoia (I jumped straight from Ventura to Sequoia).
That's because my worry is corruption of the entire Library, which Photos stores as one gigantic opaque file/directory abomination. My .photoslibrary file is currently 70gb in size, and I'm terrified of what would happen if it becomes corrupted. The Photos app crashes not infrequently.
It's a folder that acts like a file.
Right click > Show Package Contents works, and there's an "originals" folder that should have all your photos in normal everyday files.
It writes the metadata into the audio file. Badly.
In particular, it's that "Play Count": iTunes rewrites the audio file every time you play the song.
Usually it just corrupts the metadata enough to forget the album art. But it's perfectly willing to destroy the audio data.
Then for the current files you’ll want to see what happened. Often with this class of problem either the bytes are zeroed or shifted. Since the size is the same, perhaps they’re zeroed or perhaps bytes are LE to BE or dumb shit like that (don’t know why it would be but weird world right).
Just diff and see if you see anything (I wrote off memory but you get the idea)
diff -aui <(xxd -r file1) <(xxd -r file2)
From that bare start you can see what’s up.
That describes 2025 too
When flash fails it returns garbage or zeros instead of (what was) your data. It can be tranient or persistent. And without any error codes from the storage device or the file system.
If storage returns garbage for filesystem metadata, all bets are off how the OS filesystem driver will behave.
Reformat should be done in camera. And that card used only in that camera. And only that camera gets to write to that card. And don't delete individual images.
Recently, importing via image capture has resulted in recurrent crashes. Files appear on Image Capture that do not appear on the iPhone, nor can they be downloaded via image capture, or deleted. I wish I knew wtfudge was going on.
So much wasted time, now I just use image capture to import and organize directly.
Still get errors from time to time.
Apple needs to hire more quality control, their software integration is going from a positive to a negative.
Seamless integration was a large part of Apple's initial hook, and continues to be a part of their drive to push services, it should be a priority.
On day 7 or so the import failed and all files on the pad got corrupted. But also the SD card got corrupted.
I stopped using the device and the card because I knew not all is lost. I had to buy a new card in SF as replacement. Back home I used a recovery software to check if data is still on the card (I used the same software before on a card that got deleted by another person and I was able to get all images back). I was able to get most of the images recovered and also recovered a few from the iPad. All in all I lost maybe 10 out of a few 100. Now I travel with multiple cards and backup already each night while in the hotel. And I don’t delete the images on the SD Card. I format only when I’m sure I have everything copied and secured.
I think he was just looking for an excuse to buy new kit
I've been importing raws for years from Sony and earlier from Pentax and didn't experience it.
In fact searching for "OM image corruption" shows bunch of results not related to Apple Photo.
My guess is that OM has buggy SD driver which starts deletion before actual read finished.
There's also the excellent osxphotos utility which can export / backup / migrate photos in and out of apple photos:
No longer have to bring laptop or external drive along for backups
I learned the hard way to never delete photos from the SD. Just buy a new one it's so cheap anyway.
Great article by the way, sounds like my kind of rabbit hole :)
Not a good idea, you are going to have piles and piles of SD cards that will be hard to manage, and you will burn through $$$.
Last time I looked (pre-COVID) there wasn't a lot of promising options, and some didn't support HEIF images
It's strictly for looking and exploring old photos. It doesn't do photo editing (except metadata editing), nor do I expect it to.
For the 99.999% of us who just use our phone, nothing to worry about.
Scared me for no reason! =P
(sorry not sorry)
Personally, I have seen a row of green pixels on the top or bottom + vertically flipped photos on import.
Good sleuthing!
https://cdfinder.de/blog/files/image_capture_bug.html
(I'm not sure whether this bug has been fixed or not yet, though I think it has been fixed.)
I wouldn't say that out loud. Apple's motto with software is move slow, break backwards compat anyway.
I've stopped buying apple stuff
I got similar symptoms as mentioned, I suspected AFS+, but what do I know. It has happened on at least 3 iphones (pros), now when I think about it, I don’t remember any iphone I haven’t have troubles with. Having 5000+ images (non-raw) where 5-10% are corrupt is infuriating, but I just stupidly buy another iphone every year (the most expensive one).
Re-importing images 10-100 times could sometimes extract a few additional images, but the phone just disconnects after a while when running such scripts.
https://www.businessinsider.com/steve-jobs-mobileme-failure-...
https://gist.github.com/tenderlove/25853f50ab46a58738ff2cc22d682f2b
That said, the article does mention replacing basically all the hardware and still encountering the issue. FWIW, my personal experience with Apple software so far is that the usage expected for Average Joe is well tested and polished. But stepping outside of that, it's "Here be dragons" territory very quickly.
Yes this argument is a bit unconvincing for me. Not saying Apple photos doesn't corrupt his files, but this is not real proper investigating either.
That's a mistake no mater what application you're importing to, else we'll be graced with another blog post, "Darktable app Corrupts Photos".
What's the purpose of RAW+jpg though? Seems rather redundant?
Thanks dad.
You get to keep out of camera jpg files. Some people might like how their camera processes jpg files and might also want the raw file for a scenario when a more complex editing is needed.
Processing RAW can be expensive time wise. If you’re sorting through a session of 10,000 photos, you want the speed that comes with the jpeg variant, which allows you to quickly sort out blurry, smeared, severely mis-exposed, and other various defect photos.
The storage cost is negligible (JPEG75@10MP is cheap) and the workflow benefit is immediate. Additionally, cropping and early white balance corrections (as well as a handful of other things) are much faster to preview with a non-RAW version of the image; since you’ll be processing that detail later anyway from scratch in the RAW later, it’s functionally free to do it on the jpeg version before you dig into the raw.
Additionally, there’s a cheap debugging aspect that you saw here: was it Apple Photos mishandling ORF? Was it something else? When working with both, you have a “reference” that can be used to make sure your digital development pipeline is set up correctly; finer details about the imager can sometimes get mangled by some RAW developers like pixel order and sub pixel blending. Not every CCD is a linear grid, not every LCD looks the same, but if you can get your RAW pipeline producing ≈the same as your camera did, it verifies that you have things mostly set up correctly.
If I'm going to share the photo to an album or something, I process the RAWs selectively.
What's the purpose of RAW+jpg though? Seems rather redundant?
I personally would not let device A to automatically delete files from device B while files are being copied from B to A.
My workflow is quite manual when bringing pictures in from camera to my MacBook.
- I simply take the SD Card from the camera and then use the SD Card reader on MacBook itself to copy the files (RAW + JPEG) into a working directory.
- Move just the JPEGs into Apple Photos library
- The ones which I think I can/should improve using RAW processing, are processed in DxO Photo Lab and exported to JPEG with a *_DXO.JPEG filename
- DXO Processed JPEGs are added to Apple Photos again. This time due to the naming scheme, the DXO processed JPEGs and camera baked JPEGs are next to each other which helps in quickly checking the results.
- Delete the camera baked JPEG once I am happy with DXO's output
Regarding...
What's the purpose of RAW+jpg though? Seems rather redundant?
I only keep JPEGs in Apple Photos as all of my image library is backed up to iCloud and don't want that duplication.
RAW files get backed up to another SSD. Looking into a better backup for RAW files.
Also, since I switched recently to a camera which uses CFeB cards for best experience (but also has a SD card slot), the onboard SD Card reader on my MacBook will become useless for this once I get an external CFeB reader.