A few other examples:
- The Touch Bar was much more than an OLED strip, it was Apple’s first move in the transition to Apple Silicon on macs. The Apple T1 chip in the 2016 Touch Bar MacBooks was the first solely Apple-designed processor to appear in a Mac and took over several responsibilities away from intel chipsets like power management, fans, sleep/wake, access to the camera & mic, and the secure enclave powering touch ID. Then the T2 added encryption of the SSD, audio management, image processing for the camera, and prevented tampering with the boot process
- The iPhone 3G shipped with a Liquidmetal SIM eject tool, which is made from a strong custom metal alloy which is "practically unbendable by hand unless you want to hurt or cut your fingers." Although Apple hasn’t released anything with the alloy since then, now nearly 20 years later Apple is rumored to be using liquid metal in their upcoming foldable iPhone.
- RealityKit had 3D scanning and a lot of other cool AR capabilities for years which didn’t make sense until the Apple Vision Pro was released.
Not that there are particularly many places where this is used - mostly because it really is just very expensive. In the awesome position that Apple is in, economic feasibility is so much easier to achieve, with like tens of millions of guaranteed parts to be preduced.
I read their glasses when taking video or pics the lense will light up and or flash more prominently then Metas. Maybe that will help the whole privacy issue and also it's not Meta (do love my Meta or smart glasses as a whole will ditch Metas for Apple quickly as both pair of Metas broke & there's no store for support).
If the bar had been added on top of those, I don't think there would've been the same kind of hate for it.
What drove me crazy though was the escape key. They later added the physical escape key back but I think at that point it was a bit too late.
2) While "happy path" on macOS pretty much never requires you to use Fkeys, but my workflow does. Blindly using touch buttons is harder than real buttons.
3) I'm not huge media keys users, but I bet #2 applies here as well.
I liked the touchbar in every other sense. If it was just an addition to an existing keyboard, people wouldn't have hate it[];
[
]: At that time it was hard to not be frustrated using mac (butterfly keyboard etc), so touchbar might have gotten more hate than it deserved because of overall frustration.I feel like it was fairly customizable - the Mac system settings let you do a lot of drag and drop of controls, and I recall iTerm having a similar interface for customizing the bar in its own settings.
I do think it should’ve been given a lot more love, but that’s Apple for ya I guess
I honestly think it was mostly a "we have a custom secure coprocessor now, what can we do with it?" sort of thing, which also worked out for Touch ID and disk encryption.
I did hate the butterfly keyboard that was introduced at the same time. Probably Apple's biggest hardware mistake of the past 15 years or so.
Things that stick around, are generally value adding across a large or complete subset of their users. Touch bar was always niche, and thus always doomed. I think a good counter comparison is Apple VR headsets. For me, i have no use and little interest. But i can see them as a hedge at the very least, or as an enthusiast entrant into an emerging market, where future products in that segment may become interesting. And on top, it doesnt impact me - i can ignore their existence until it becomes useful.
If touch bar were launched like VR, i suspect it would have gotten similar level of dismisals, but less hate.
However there was something very human and nice about helping out a stranger with a small random task from time to time. I fondly remember one older lady who spilled a box of blueberries on the kitchen floor and I helped her hunt them all down by guiding her around. It was 10 minutes of connection with a random person doing something fun and which is till remember fondly 4 years later
Not going to lie, it deeply worries me how quickly society is just accepting offloading what used to be meaningful human interactions to AI. I hate to imagine what a society looks like once the people in it solely rely on AI for navigating life.
Sorry for the bleak reply. I was genuinely excited to read about the service as you described it and would love to hear that the human assistance side of it still works even if the website only showcases the AI.
I'm someone who absolutely hates generative AI, but AI that assists people in living happy lives outside is something I'm completely in favor of.
> We are all here on earth to help others; what on earth the others are here for, I don't know.
They always had significantly more people willing to help than in need of help. Which is good, not going to knock that. I signed up for it many years ago but didn’t get a single call, so eventually I just uninstalled it.
Input on the iPhone is so dreadful nowadays. Their palm rejection is definitely worse than before, so mistyping is more frequent. Their text-correction algorithm for typing is worse than before, and it frequently makes incorrect corrections to words that I don't notice, because they change words a few words back from where I typed. And STT hasn't improved. On top of that, my fingers are tired of the phone form factor. Please make the iphone not a chore to use, apple.
I found another dreadful iPhone input "feature" yesterday. If you are browsing around in third party carplay apps, and ready to tap your selection, but instead press the accelerator first, it truncates the list to only a few items, and scrolls to the top.
Way to reduce driving distractions guys! What's next? If the car is moving, maps changes destinations?
I really wish human computer interaction research were more broadly applied, and if you do dumb stuff like all of the automotive / carplay world, then you'd be liable in court.
I once had a car that hid the backup cam behind a legal disclaimer every time you turned it on. I'm sure at least one pedestrian was hit by a car in reverse while that screen was on. The manufacturer should be 100% liable for the poor UI decision.
Yeah, that's unfortunate considering you can have it do nearly all of that (download maps, navigate to business all while offline), except asking siri to do it for you.
> I once had a car that hid the backup cam behind a legal disclaimer every time you turned it on.
My car pops up a dialog telling me (in a paragraph+) to pay attention while in semi-autopilot which I have to click "ok" on to get back to the map. It's very ironic, and extremely dangerous.
Looks like Wispr Flow uses a cloud model [0]:
> Cloud based speech processing infrastructure for 1B users
It gets to be a messy comparison because my iPhone can do STT with no latency pretty well fully on device, but Wispr Flow requires a cloud model, but to be fair, older Apple devices do as well. It's not an apples and oranges comparison, but I think those technical details make this a non direct comparison in a few ways.
For on-device with low system resource usage, Apple's is pretty damn good.
Yes, Wispr Flow is a cloud-based application. All of your voice data, dictation, and contextual AI processing happen on remote external servers. It requires a constant internet connection to function and cannot be used offline. that definitely isn’t the path Apple wants to be on so far.
I’d much rather have “cheap, dependable, and good enough” over oligarch pricing for what used to be a one time software purchase any day.
https://www.theregister.com/on-prem/2023/08/16/those-who-rel...
Would be great if they could at least fix two major bugs:
* input simply fails (seemingly) randomly where it is supported and many apps from major vendors don't support dictation input at all (e.g. OneNote) (there should at least be a fallback (a la Dragon Dictate from decades ago) for those cases * capitalization is still random leaving you with many errors to correct
but Apple mostly seems to see accessibility as something to use to enable performative press releases not actual functionality...
The streaming dictation they also added in that release is also much appreciated although occasionally buggy.
Sometimes it gets to “fever dream where you’re suddenly unable to successfully perform everyday tasks” levels of insanity.
And the worst part is: it used to be fine. I’d type more or less on full keyboard levels of speed and accuracy on my iPhone 4S.
Eg: "Audrey, just swing up the sailors". (My name is not Audrey)
Open your Settings app. Tap on General. Scroll down and select Keyboard. Toggle off Slide to Type
I have a friend named Zi in my contacts. For some reason ios kept autocorrecting “I” to “Zi” and would do it too far back for me to notice.
What’s weird is how this is such a dumb bug that Apple usually irons out.
It drives me nuts the number of times it misinterprets some common word as a name I have in my contacts.
One of my primary methods of interacting with an iPhone is through speech and the state of Apple speech transcription is pretty horrible. It bothers me greatly.
I know some of the workarounds and things but it does feel like it’s in the Stone ages.
I don’t think it’s a microphone issue since iPhone microphones are fairly decent and I don’t think it’s a CPU issue either because Apple Silicon seems to be some of the best on the market. Which leaves us with the software…
Maybe they should put that cash hoard to good use and buy up some of these transcription companies or license their IP so we get truly high-quality transcription.
I wish more companies focused on how they can help humans instead of replacing us or squeezing us as hard as possible in the name of productivity.
My experience is limited to my elderly parents who have trouble seeing. With the text size Apple allows them to set it to, their phones are unreadable. Text runs off the screen in every app, 1st and 3rd party.
In their bill example, the user is told to confirm with the provider. Why not offer to call the number on the bill? Instead of telling them to use text detection, do it for them? Presumably Apple Intelligence would already have that capability. I’m afraid this will be a gimmick at best.
EDIT: Forgot to mention, the grip is good to see. Hopefully they don’t charge the apple tax on it.
I have a problem with astigmatic halation that makes ‘dark mode’ difficult to read. Since iOS 26, multiple aspects of the system have been made dark only, contrary to the system setting. Writing text correctly should be the lowest of low-hanging fruit.
I suspect this is more of a flashy ‘AI’ promotion rather than reflective of any real commitment.
They treat new industry advancements as technology, not products itself.
AI will be a feature to improve the customer experience, not the product itself.
https://blog.google/products-and-platforms/platforms/android...
https://android-developers.googleblog.com/2024/09/talkback-u...
Don't get me wrong, Apple using these technologies to help humans who are in need of help is laudable. But let's not pretend we don't know why most corporations don't look into this kind of thing. I think if we're being honest, we all very much know why they leave this sort of thing to the always nebulous "others".
> “When we work on making our devices accessible by the blind,” he said, “I don’t consider the bloody ROI.” It was the same thing for environmental issues, worker safety, and other areas that don’t have an immediate profit. The company does “a lot of things for reasons besides profit motive. We want to leave the world better than we found it.”
— https://www.forbes.com/sites/stevedenning/2014/03/07/why-tim...
I was just answering the question of why other corporations don't.
Money.
There's relatively little money in helping the visually impaired. You have to do it because you want to do it. Not because you're going to get rich.
I assume almost everyone looks into spending less money than more money for equivalent goods and services.
Increasing their productivity is helping humans.
My one hope is that this eventually becomes widespread enough to stop alt text scolds.
this sort of thing really needs input from someone that uses it before we can judge it
my go to example of this is this talk by Saqib Shaikh (a blind software engineer at Microsoft) giving a talk about Visual Studio. Link is timestamped
I wish more people would watch videos like this just because having a realistic idea of how blind people do certain tasks can help you move from pity or even compassion to a more productive kind of understanding. I think sometimes when you haven't seen it, you can't really even imagine how it can be done.
It's not rare among the blind in general.
Unless you're completely technologically illiterate, the kind of person who has no idea how to install an app or sign up for an online account, you're probably doing something of the sort.
I'm not even sure what to say, but discoveries like this are why I use hackernews, I'd never have known this otherwise.
I can easily understand Eloquence (the speech synthesizer he's using) at that speed, but I struggled a bit with this one.
Whenever I'm watching lectures / talks / podcasts, I tend to watch/listen to them at 2x to 2.5x times speed.
I only need to lower it if someone flubs an important word in a definition, I'll replay that part at 1x speed.
If the person is talking particularly slowly (usually for international audiences) I put the speed up to 3x to 4x speed so it sounds like normal 2x to 2.5x speed.
---
My youtube muscle memory:
(standard video controls used by every video editor ever)
J = back 5s
K = play/pause
L = forwards 5s
(youtube specific controls)
Shift F = toggle fullscreen
Speed controls (this part is muscle memorised as fast as a password input):
1. Cmd/Ctrl Shift K: opens console
2. Up arrow: loads previous command, typically: document.querySelector('video').playbackRate = 2.5
3. Enter: runs command
You have to type in the command for the first time, after that to change the speed, change "2.5" to whatever number you want and console history will remember the change so you can go through the different values with up/down arrows before pressing enter.
You have two modes: "focus mode", where you can edit text in text fields and keys are passed straight to the browser, and "browse mode", where keys move a virtual cursor around the page.
In browse mode, navigating with just arrow keys all the time would be just as slow as you might imagine, so you use single-key keyboard shortcuts to move by role, E.G. to the next heading, button, table or unvisited link.
The keyboard layout is optimized for memorizability and not efficiency, you use the actual arrow keys instead of hjkl for example, but the concepts are eerily similar.
There are a couple of other approaches to solve this problem, Mac OS's Voice Over is much more Emacs-like for example, and each approach has its own pros and cons, but that's definitely one way to do it.
RIP kid https://youtu.be/fnH7AIwhpik
We all do that, I mean unless you’re hearing impaired.
Everyone’s familiar with dropping a coin or such and knowing exactly where it landed without looking.
That’s more passive sonar though.
Do I recall seeing videos of guys mountain biking and making a hissing sound for an active sonar style echo location?
Or am I making that memory up.
Even better, fire up Orca (or whatever screenreader application your OS comes with) yourself and try to use your computer while shutting your eyes, kind of eye-opening (no pun intended) what kind of experience these sort of users typically get. And also, you quickly start to understand why they set the speech rate for their voice synthesizer to be so fast, it's almost unbearable navigating applications (and particularly lists) otherwise.
Unfortunately it seems impossible to get all that much funding for accessibility work :/ I wonder what ever happened to the Newton accessibility bus intended to supplement Wayland...
Hm, never heard about it, but now I'm wondering too. I just finished implementing proper accessibility support for my native app toolkit for Linux, macOS and Windows, but only done it for X11 so far, I was just gonna get started with Wayland. What is the accessibility story on Wayland, couldn't people rely on the same protocols as with X11? That was my impression, but haven't really dig into yet.
There are apps I use semi-regularly that less-experienced screen reader users thought were inaccessible, and I couldn't even explain what they were doing wrong from memory. The ways of working around accessibility issues are just so ingrained in me that all I can usually remember is "yeah I did this somehow, but it was six months ago and I have absolutely no idea which specific tricks I needed for this one."
I imagine that for coding it also helps deal with the fundamental problem of an ephemeral stream rather than a persistent document that you can navigate visually in multiple dimensions. Working memory is limited, and getting more text in in a short period of time probably helps you work within that better. I also imagine that working with text via audio all the time gradually stretches and improves memory.
You can show a lot more info on a screen than you can transmit through speech in a short period of time. That doesn't mean you read faster than you listen, just that sighted people essentially use their eyeballs as an "input device" to decide what information to look at.
If there's an object on the screen that you want to examine but that you don't need to click, you can just "navigate to it" with your eyeballs, without ever touching a mouse or keyboard. We don't have that luxury.
This means we need a much more efficient system for navigating what's on the screen, but that only gets you so far. Eventually, the easiest way to deal with this problem is just to increase the bandwidth of your channel, and you do that by increasing the speech rate.
Wouldn’t opposite mean you listen at sub 1x speed.
Whereas as your definition seems to be ”I’m the same, but less so.”
I'm not getting my hopes up though given apple's history with Siri, which is truly awful.
This has been the typical pattern for Apple for the last few years. The flashy features are announced at WWDC, accessibility has a dedicated, earlier press release. Before this practice, accessibility announcements would usually be tucked in some WWDC slide that most people wouldn't even notice.
I don’t think the Google's tech has anything to do with these features.
This would had to have been in the works long before the Google announcement. Also, these are enhancements of existing iOS and macOS features. They don’t require an LLM anyway; these features use Apple’s Machine Learning models.
For example, creating subtitles for videos? iOS 16 introduced Live Captions for FaceTime calls in 2022 [1].
[1]: https://www.apple.com/newsroom/2022/05/apple-previews-innova...
Twenty years and text input & manipulation on iPhone sucks a big fat hair pair of dogs balls still.
The last time I daily drove Android was over two years ago and it was immeasurably less God-damn-I-wanna-dig-Jobs-corpse-up-n-give-the-guy-a-piece-of-my-mind, only problem is his grave is unmarked. Arsehole!
After a few more years of Thanksgivings and Christmases and Mothers' Days, we'll finally train her up to a reasonable speed lmao.
Whether that control you see visually is actually accessible to a blind user is a different matter entirely. Further, it maxes out at 2x, but a blind person would typically screen read at the equivalent of 3-6x.
Related, it seems like YouTube recently paywalled speed increase beyond 2x. Another way in which it's not cheap to lose sight, I guess.
Pretty sure there's enough blind people who don't listen to voice at insane speeds, because they listen in their non-native second language or for whatever other reason. What's wrong in using lowest common denominator that's 100% accessible to those people as well as people who want faster speeds? Unlike "too fast", "too slow" doesn't get entirely inaccessible, it's just boring.
Such a random reason to criticize for.
What does this mean?
Full VR hasn't done well, but it does continue to make me wonder if there's a market for a stripped and slimmed device. I'd maybe be interested in a device that does optical controls if it fit in regular-sized glasses. I'd be super interested if it had a HUD system (even a super basic one that can only show a handful of symbols). Better still if it had some basic audio, but maintaining the "regular glasses" form factor is more important to me than the HUD or audio.
The device is large and makes the user look weird and non-present, which are net negatives.
The only benefit of the AR is showing the directional arrows, but they could get the same thing with much less weird looking non-prescription glasses with arrows sharpied on them. More realistically, anyone really using for mobility probably develops muscle memory for which direction to look to go where and then they don't even need that. At that point it's just a really expensive, really clunky camera.
1. Use AI to determine how much a bill is for
2. Call up the people who billed you and ask them how much they billed you
3. Pay billed amount
(I'm also picturing the poor CSR at the other end of the phone wading through hundreds and hundreds of call logs over the years for simple requests and managers up above screaming 'why is this guy calling us all the damn time costing us money'...)
Through that lens, this all looks a bit performative to me, but again, maybe I'll be pleasantly surprised.
The one thing I'm mildly excited to see is the improvement to Voice Control, as guessing what the programmatic name of a button is or having to constantly use a numbers grid to target elements doesn't sound fun.
To respond to what I see in some of the comments:
- On speech rate: It does take quite a bit of practice to crank up the speech rate and there's a degree of retraining you need to do when you switch voices. A lot of more "human" sounding voices are harder to follow at super high speeds which is why a lot of people prefer more robotic but consistent speech and generally aren't convinced by AI-powered TTS yet; they often fall apart if you raise the speech rate past a certain point. - Re: actually waiting for the target audience's verdict: This is so important. I see more and more companies, individuals etc. talk about accessibility, build accessibility solutions and evangelize AI for accessibility without EVER talking to the people they claim to help. This will almost certainly mean mistakes will be made, up to and including doing more harm than good. If you want to do accessibility right, that includes AI products of any kind, hire people with lived experience or you'll get the equivalent of machine-translated text, hackerproof security in one click or an AI-powered coffee bar that orders thousands of rubber gloves. Coincidental note: I have time for new projects right now :P
And it was valuable to me as someone going from "bad but correctable" vision to low vision. I didn't know all those apps existed. I've been looking for exactly that sort of assistive technology.
Funnily enough we're creating a competitor of these third party app that you mention, with the huge experience of my colleague that is son of blind parents.
We have an mvp online but it's not much yet and i really don't want to be the "do you know i have an app?" guy.
One thing confused me though - you felt like the on-device processing is likely a gimmick. I naively assumed this is a big deal because it means it always work, regardless of your cell service. On the subway, on an airplane, in the middle of nowhere, etc.
Unrelated, what app makes the biggest difference to you in your day to day life?
Rock-solid in AppKit/UIKit. Falls over at the embedded-WebView seam where most modern desktop apps actually live.
https://www.youtube.com/watch?v=B3SmsSCvoss
Those made the ad stand out in my opinion.
Maybe just don't wear them in a car?
I use those motion cues on my iPhone even though I don't struggle with motion sickness https://www.youtube.com/shorts/OxbjggMcKrk
Still somewhat odd when a bus drives out from behind your Terminal mind.
Someone using this feature will want motion cues as well.
And in your quote: Dwell Control is a feature set to interact with an Apple Vision Pro using only your eyes. Lingering your gaze on a button will press it. An AVP is now more comfortable to use in more situations because of motion cues.
Maybe just rethink your "maybe just" comment...?
Why not?
We have harsh laws on using phones whilst driving, a Vision Pro (if configured in a specific way) could entirely block your vision with a Movie or Show and this is dangerous.
Also, in what country is it legal to drive around with a VR headset?
I think the trap in creating anything is doing it for a crowd. Art, software, anything... it turns out better when it is made with a specific, named individual in-mind.
Accessibility features are almost always championed and field-tested with one specific loved one in mind and I think that's what keeps the technical solutions personable and grounded.
Without that, there wouldn’t really be great vlm and conversational models.
The AI companies might have paid for the dictation of some videos on their own but voice assistants etc wouldn’t have existed and our ability to have AI that eventually understands the world would be much much harder.
You however…. Maybe need to switch to decaf?
They are there for everyone. You don’t need to have a permanent disability to benefit from accessibility features. A device designed to work one handed is useful to someone without an arm or a person with two arms who is holding a baby. Subtitles are useful to someone who can’t hear or someone lying to a sleeping spouse or in a noisy place.
“Accessibility needs can be permanent, temporary or situational.”
https://www.coursearc.com/accessibility-content-fundamentals...
The form-factor is a significant issue for real-world usage, and it's kind of unclear if there is a plan for a future product line given its (pretty abysmal) initial receiption.
The price and lack of content and developer interest have been the main problems.
And ultimately, people just don’t seem that interested in this product category. Meta ran into the same issue, though at least they targeted gaming where there is a decent niche.
VR/AR tech seems cool and futuristic, but hasn’t quite found its killer app yet.
Apple really screwed themselves by only supporting WebXR for cross-platform VR experiences. Soon Valve will ship the Steam Frame, which will likely cost a fraction of the Vision Pro and support bog-standard PC games like H3VR, flight simulators and flatscreen PC titles. Meanwhile, AVP owners will have paid $3,500 for a more powerful chip/headset with a fraction of the content library and featureset that Valve and Meta offer. Vision Pro's lack of audience is entirely a self-imposed failure, it seems.
It was a strategic mistake for Apple to not focus on gaming. But realistically, the AVP was always going to be way too expensive for basically anything.
Maybe if you could pick one up for like $800 and there was a lot of great 3D immersive content, it could take off. But even then, I feel like it’s just not a product category the average person is that excited about.
My biggest gripe with Netflix is that they only have like 3 languages and no auto translation. And even bigger gripe is that it's because of the union racket. They apparently need to pay hundreds of thousands for something computers do for free. Insanity.
I guarantee you for most (if not literally all) titles you see, they actually have more subtitles, they just don't / won't show them to _you_.
Netflix has gotten pretty good at VPN detection, but you used to be able to see this _very_ clearly by VPNing to a different country in the EU, and the available audio and subtitle languages changed dramatically.
As of macOS 15 (and I don't think they fixed it in 26), you can only increase the font size of first-party apps on macOS.
The global font size setting doesn't apply to third-party apps, even those built using Apple's frameworks.
Edit: was thinking about this feature https://support.apple.com/guide/iphone/get-live-captions-of-...
The above caption for Apple Vision Pro is for a video that to me, as an Apple Vision Pro user, is discomforting.
More questions are raised than are answered by the short video: Is the user able to fit the Apple Vision Pro by him/herself? What happens when dwelling on a directional control misregisters? Can the user recalibrate the "Eyes and Hands" setting? Dwelling on a control displaces focus and there may be impeding objects in the path of the power wheelchair. Is this really a good idea?
To my sensibility, the video is unsettling (at best), especially given how cumbersome Apple Vision Pro is.
[0] https://www.apple.com/newsroom/2026/05/apple-unveils-new-acc...
With all due respect, my concerns are not nonsensical but borne of my daily use with Apple Vision Pro and my awareness of the limitations of dwell control.
Iterating on this idea with a device lighter than Apple Vision Pro and improvements to dwell control would likely be required before this could ship to larger populations of disabled users, but that is not what is depicted in the video.
My sense is that the possibility of an accessibility affordance with people who are severely disabled is driving opinions in this case more than the reality of what’s available.
To my mind, much of these AX announcements are reminiscent of the circumstance that led John Gruber to author “Something Is Rotten in the State of Cupertino”, which is that these are not shipping features but ones slated for “some time later this year”.
I’m a huge AX fan and work directly in the domain space, but something about that video in particular coupled with my near-daily use of Apple Vision Pro doesn’t feel right.
It’s the hardware I designed coupling the power wheelchair to the AVP, and I’ve driven it myself.
"possibility of an accessibility affordance" what do you mean possibility, that is literally the case. Even if it's not perfect (which nothing truly is, obviously), it is undeniably a novel control system for its target audience.
"doesn’t feel right" So your point is simply that your subjective opinion is that it 'doesn't feel right'? What does that even mean? I'm not saying, and the announcement is not saying, that this is some platonic ideal of accessibility controls. Not sure what you are getting at at all.
Thank you, Apple, for taking accessibility seriously and dedicating resources towards it.
I very much appreciate it, and the work of the entire accessibility team.
And what about windows (if you use it) ?
I think that we should all be concerned by the accessibility feature, we never know what is going to happen in life.
I can tell you that the hearing accommodations on the AirPod Pro 2/3 headphones brought literal tears to my eyes because of how fabulous it makes music sound for me.
This is a a LOT more work than just adding an equalizer because you have to do multiband real time compression and expansion, in relation to other frequency bands and respecting band-specific sound energy limits.
I know I might sound like I’m gushing, and I kinda’ am. They didn’t have to put in the time or energy to do that or maintain it and they did ... and for that, like I said, I am extraordinarily grateful.
This should really be the last resort.
https://developer.apple.com/documentation/accessibility/brai...
Probably 80% of "LLM's are below expectation" complaints (from the general population) involves some form of image analyses.
Image tokenization is hard because unlike language tokenization, where every token is extremely dense with meaning, image tokens tends to be meaningless or irrelevant but are processed all the same.
Give an SOTA LLM a picture of toothpicks and ask it to move one to make a square, and it will probably struggle and fumble it. But give a mid-size LLM from 2 years ago the same problem in verbal form, and it will nail it almost every time.
That takeaway is, do everything you can to avoid having the LLM need to rely on images for the answer.
edit: it seems that asking Apple to follow their own accessibility guidelines isn't popular on HN :-(
The other thing is that if you're around others, voice input means you have no privacy. Even if you're not doing anything particularly private, it's a bit awkward and potentially embarrassing. If you use touch input in conjunction with a screen reader, you can be more like a "normal" user in that what you're doing is just between you and your phone.
Did not test it yet, but blind users may be more prone to dominate Command Line Interfaces, which are becoming increasingly popular due to its easy integration with LLM
iOS is just painfully good. I can pause a video, put my finger on text inside the video, and copy it. Until they added it, I didn’t even know how much I needed that.
Selecting text from anything displayed on the screen for more than 7 years. https://www.thurrott.com/mobile/android/165834/android-9-pie...
I guess if you knew people using Android, you would have known you needed it 7 years ago?
People talk a lot about how MacOS has gone downhill but I feel like it would have been a good start if developers could continue to patch over Apple's shortcomings like they used to be able to.
I imagine that we would be a few years into a spectrum of tools like this if they didn't lock it down like they do.
Totally aware that plenty of HN commenters are very glad that Apple keeps this locked down. I'm just the other opinion, that's all.
Of course it's AI.
I have fond memories of an old coworker 10 years ago who is blind. He would use his phone no problem, texting, going about his day, he was even on Tinder (credit to Tinder for making their app so accessible long ago). He would commute on his own, walk to the train station, even transfer to another train during peak rush hour. I’m not saying it was all easy for him, but nothing in this video really stood out to me more than what shirt was on the bed. I know other services/apps have long existed to be the “eyes” for people who need support, but this video feels….uneventful?
I may be cynical about this though, as I often hate how Apple’s marketing makes these emotional bids about how life-critical they are to society - which is fair to a degree..but it just feels cheap to be glamorising “look! we saved this person from pending doom, cool right??”
If this includes improvements to the screen recognition feature in Voice Over, it could provide accessibility for apps where the developer doesn't care about accessibility, which is extremely common.
The vision capabilities could be useful if they are done well, but I suspect that will always be covered better by 3rd party apps.
Additionally I don't believe this is just marketing. This is adaption to a changing market. Apple's customer base is aging and having these kinds of features will allow them to keep using Apple products for a longer.
