Turning a MacBook into a touchscreen with $1 of hardware (2018) (opens in new tab)

“Who wants a stylus? You have to get ’em and put ’em away, and you lose ’em. Yuck. Nobody wants a stylus. So let’s not use a stylus. We’re going to use the best pointing device in the world. We’re going to use a pointing device that we’re all born with—born with ten of them. We’re going to use our fingers.”

— Steve Jobs, 2007

(8 years before the introduction of the Apple Pencil)

The original "gorilla arm" UX research is much older. However, Microsoft surface was something of a niche hit, and spawned a number of clones. PC laptops with touchscreens are quite prevalent even if they're not in the full-hinge form factor. They work a lot better if you can lay the screen flat or at a low angle in your lap.

Re: the stylus sub-thread, I've actually used cheap Android resistive+stylus phones and a Compaq Palm Pilot clone and .. yes, they were really bad compared to modern phone interfaces. The stylus has a niche market for artists, who need a high quality pressure sensitive version.

(edit: attempting to find the original citation for "gorilla arm" takes me to the Jargon File and the early 1980s. Along the way I found the delightful existence of a UX researcher with the name Sebastian Boring, though)

That's only an argument against using it as the primary interface. There have been many, many times for me where it was so much easier to reach for the screen and touch a button, than to reach for the mouse and maneuver the cursor over the button to click it.

It shouldn't be the only way to interact, but as an option, it's awesome.

I wonder what people will say when Apple releases the touch screen MacBook later this year then.

I had several PCs with touch screen and this absolutely true. Even intermittent use is not something I did, it’s just too inconvenient to ever become a habit, so the few times it’d be great, I don’t think about it being there because it’s not in my active list of affordances.

I work for a company which has developed a special purpose drawing (well, entry-level CAD) app, and for a long while, it didn't have keyboard shortcuts, and that was probably because I mostly used it on my Toshiba Encore 2 Write 10/Samsung Galaxy Book 12/Samsung Galaxy Book 3 Pro 360, so would use it w/ touch and stylus, which was quick/natural enough that I never felt the need for keyboard shortcuts.

Such usages pretty much want a very flexible device though --- I'll often use mine fully flat on a lap desk and will rotate it in various ways depending on what I'm doing, and will then further mix in using my Kindle Scribe and Samsung Galaxy Note 10+ (which use the same stylus tech as the GB12/GB3Pro360).

Sometimes I'll add my MacBook into the mix by way of a Wacom One display, but I have a 1st gen unit, so no touch, so every so often I'll find myself dragging at it to scroll or tapping a control with my left hand to no effect.

I'd like to try Apple Sidecar on an iPad (which arguably is Apple's touch interface), but can't justify the expense, esp. for yet another stylus (I couldn't easily count how many I own, and carry a spare Lamy Wacom EMR in my sling bag), esp. a stylus which only works on one device.

Still waiting on Apple to make a replacement for the Newton --- the smallest size iPad which supports the Apple Pencil is close, but I need something daylight viewable, hence the Kindle Scribe (which I'm going to be replacing w/ the KS Colorsoft presently).

I have never owned a touchscreen laptop and I agree with most of the criticism in comments here against it. But after just briefly using one from my father I have to say there is some thing in our brain that makes it kinda more satisfying, if that is the right word, to touch on things that appear on a screen. Even as a power user being used to just using keyboard most of the time, after 10 minutes with a touchscreen my mind prefers to touch on screen instead of touchpad.

Nobody forces you to use touchscreen exclusively?

The reason you want a touchscreen on a laptop isn't so that you can use it for "an extended period of time" but rather so that you can use it to do the few things that are painful with a touchpad, like drag and drop.

I know I don't want, nor do I want anyone else, touching my Mac screen. Steve Jobs quote notwithstanding. They are pretty good at getting dirty on their own.

Yea but now they're selling $400+ Magic Keyboards for iPads which then function exactly in this vertical way. No difference. I've typed this on one.

I have used a couple of laptops with touchscreens, and the experience was awful, even with the latest technology. If Apple gave us an iPhone or iPad-quality touchscreen on MacBooks, I am 100% sure the experience would be perfect.

Yeah, but this was also strategically in Apple's interest to sell the iPads with nerfed up iPad OS as a separate line up. I love Steve Jobs and all, but this did NOT age well. The millions of people using Surface and Surface Pro will absolutely disagree with this take.

did steve jobs ever see an easel?

I feel like the point isn't "there should be a touch screen MacBook" but more "holy shit we simulated a working touch screen by looking at reflections coming off the glass, isn't that cool".

Sometimes, if I’ve been using my iPad for awhile and switch over to my MBP, I might reach out and touch the screen out of habit. I can’t be the only one.

I don’t understand touchscreens on laptops that aren’t designed to fold flat. It’s got the feel of finger painting an unconstrained birthday balloon.

While I'm the same and totally agree with you, the few times I've been using touchscreen I find the habit sticks so hard that for days I keep touching my macbook screen, so there is definitely some subconscious desire for this (or I would have defaulted to using the trackpad even if my brain thought touch was available)

> I don't get why Apple are apparently going to be putting them into the new MBP line later this year.

Apple has apparently being going to put a touchscreen in a laptop every year since the iPad came out, and it's never materialized.

All my Windows laptops have touch screens and I love it. What is the problem with having another input method available? You only use it when it’s appropriate.

It just feels ancient and weird now that I can tap every screen I own, except my Mac. I don't want to replace the Mac's keyboard & mouse with a touchscreen, I would simply like it to support touch.

(This also made me realize the impending obsolescence of the Studio Monitor XDR: no touch support.)

Tech-reviewers keep harping on the MacBook for not having a touch option, but I think it's mostly of check a box.

Something no one seems to address is that it makes no sense to have touch on the laptop screen, because you honestly don't use it much, at least in a professional setting. You'll always dock your laptop anyway, either for comfort, or legal compliance (or both). My 27" monitor doesn't have touch, that's what I use 99% of the time, the laptop screen is a small auxiliary screen on the side. Why I reach out and touch it? That's also why the touch bar made no sense, it was on a keyboard that I almost never use.

I wouldn't want a touch screen to become the primary input device, but I think it would be useful on occasion. Not entirely unlike how we still have touchpads even though we try to use keyboard commands.

> I wouldn't want a touchscreen MBP even if it was free, anyone else feel similar?

I don’t want a touchscreen MBP, but as long as touching the screen is an optional interaction and everything else is the same, I see no reason to reject it if it was free. I can just not touch the screen.

> we already optimize for keyboard commands to avoid living our fingers over to a touchpad.

“We” is a much smaller percentage of people than you’re likely thinking of. We’re outliers, not the norm. Yes, even amongst professionals.

I have 2-3 old touchscreen laptops lying around. The touchscreen is useless to me. Worse than useless. If I ever use it, it’s accidentally, and I end up annoyed.

Why? Better than a track pad.

Use a Surface Pro some time. If you are just casually browsing or reading a website. I find it much nicer to just tap on a link or swipe to scroll.

I very much would want a touchscreen for my use cases.

Well, when I am doing rather thinking work, so not type in commands as fast as possible - I very much do like my laptop to have a touchscreen. It is way more ergonomic and comfortable, but yes, slower. But when the real work happens in my head, I like to be rather comfortable.

(Also I can immediately test touch features of the apps I develope)

Sometimes I feel the urge to do some art, and the bigger surface might allow that. Perhaps in lieu, make a bigger trackpad.

The ergonomic aspects are horrible. I believe there's actual research on this from the 70s/80s/90s.

Seems like it only makes sense if it's a hybrid tablet laptop like the Yoga. Otherwise it's a nice gimmick. I can also see Apple being terrified at someone's dirty fingers smudging the laptop, though they'd have some anti-smudge coating built in at that point

Macs are definitely not optimized for keyboard commands. If you feel the software you use is keyboard optimized, odds are it's not really Mac software.

It would make sense if the screen folded over. In a laptop form factor a touch screen is just annoying because it keeps pushing the screen back.

Maybe they solved the greasy fingers leaving smudges on your screen problem with a new surface. :D

I completely disagree. I've got a laptop with a touch screen, and it's occasionally very useful. It's rarely my primary input method, but when you're not constantly using the mouse, how often do you lose track of where the mouse cursor is? Instead of reaching for the mouse, figuring out where the cursor is, and then carefully maneuvering it over the button I want to click, I can just reach out to the screen.

A touch screen is incredibly useful when you're not currently already holding your mouse. It's easier to switch from keyboard to touch than from keyboard to mouse.

The benefit of a touchscreen MBP is that Apple will be forced to make their screens more protective.

I agree. I've never wanted a touch screen on my laptop. My screen gets smudged enough already.

Yes, I feel like it'll be a degrade in quality if they do this with any of their current line up. If they want to make a Macbook Ultra or whatever with it, that's fine -- I would have no interest in it.

But the Touch Bar was such a resounding success …

as long as it works well, would rather have it than not (but don't want to pay extra, so yeah... leave it out)

It's actually quite pleasant user experience for scrolling. Some interactions are better with a pointer, others are better with touch.

You can try it on an iPad with Magic Keyboard attached, it's very good to be able to do precision through the trackpad and then casually move large things on the screen with your fingers.

You don't have to use it.

This project brings back memories. I worked somewhere over 20 years ago where we were working on something just like this (touch displays using cameras). The biggest challenge was definitely the lighting conditions as you mentioned. We tried to rely on natural light but it was too unreliable. Darker skin tones were harder to pick up, and then you had issues with random reflections, light and shadow being cast on the screen, etc., which would make the system detect spurious fingers and touches.

We also had algorithms to detect finger shape to detect location of the pointer and when you were touching the screen. I saw way too many videos of fingers touching screens back then, so it's funny to see similar video clips here.

Difficult to capture reflections across a large screen while also dealing with outdoor lighting, glare, and moisture. The touchscreen part isn't usually what makes outdoor signage expensive compared to IP65, temperature control, and a secure housing, all of which would still need to apply here.

This looks like a neat option for retrofitting, and I suspect it'd work for some non-screen glass applications too. A combined IR/visible light solution would be interesting too, since I suspect those are complimentary (IR touch has issues with radiant light, while this wouldn't; this would have issues with low/no light, while IR wouldn't).

Internal with a small mirror

I don’t want to encourage people touch my screen and smudge things up.

Oleophobic coating is standard on phones and tablets, which is part of why they don’t pick up fingerprints as easily.

Some brands offer coating you can DIY yourself (eg ProofTech OLEOPEL) but these seem mostly designed for phone screens. I don’t know whether they’d be as effective on laptop screens

I was working on AI in 2018 too :)

At that time, I was quite interested in adversarial examples and ML security.

You could still use the keyboard and track pad

You chose to suffer not just with touchscreens. That said, it would probably work just fine.

> why would I ever choose to put fingers on that beautful screen ... I don't get it!

I'm not sure, but bare in mind that the iPad is almost as large of a market as the Mac at this point, and the iPhone has long surpassed Mac revenue. Touching your computer is a very popular sentiment among the grimy-handed public.

i used to have a bad touch screen laptop like 7 years ago and back then it made the mobile development more pleasant (both native and web based)

So you mean they're developing the iPad, an insanely popular device, and you're not sure why they would make such a device?