Teachable Machine: Teach a machine using your camera, live in the browser (opens in new tab)

Yes please! :)

And some quick questions:

What network topology do you use, and on what model is it based (e.g. "inception")?

What kind of data have you used to pretrain the model?

There's something fantastically entertaining about this. It's stupidly simple (from the outside) but interacting with the computer in such a different way is weirdly fun.

It's like when you turn on a camera and people can see themselves on a TV. A lot of people can't help but make faces at it.

Why does it not work in Edge? Please keep the web open, do not make stuff that does not work in a modern browser. Also always give an option to try it anyway.

A blog post on the technical details would be great, please. Thanks in advance, since I know it'll take a bit of your time to write.

To answer the question at last, yes, I am interested.

I want to teach my computer to recognize when I'm slouching, so I can correct my posture.

You mean a device such as the Leap Motion Controller? http://store-eur.leapmotion.com/products/leap-motion-control... They seem to offer a VR headset add-on these days, but I've only ever seen the 'basic' controller in action, which worked okay-ish.

Surprisingly well!

It's a really well put together demo & tutorial.

I held a pen up next to me and held the green button.

Then did the same with a mouse.

It would flick between the two if I was holding nothing, so I held the orange button for a bit while holding nothing.

Worked pretty much every time.

Training is fast enough with a few hundred images per class that I didn't notice any delay.

It's working great because they're using a state of the art model (SqueezeNet https://github.com/DeepScale/SqueezeNet) and also the samples / experiments you do are often only on yourself, in the same lighting, same clothes, etc. So it gives a nice idealized playground environment that mostly eliminates annoying details like this.

there are 3 default classes, so you train according to each class(e.g. hand waving, sitting still, etc) you take examples of each(using your camera). you map the input data from your camera to some output data(e.g. if i used the green button to take photos of me waving), display a GIF of a cat that's waving. instead of a GIF you can use sound too

But you would need to have:

1 a working phone

2 a valid credit card

To use azure which places a too high bar on students. I mean I've tried to argue for graduated restrictions so basically students with .edu emails should be able to do some things without entering a credit card number but the fact that it is not possible suggests this isn't a priority for azure.

Google says this finds on your browser so there's little infrastructure cost for this demo, right?

Can you clarify on what you did with it? I'd love to start dabbling in solving problems with ML, but am a bit intimidated by getting started. Is it fairly easy for a novice to do the things you did?

I'm pretty sure this is actually pretrained as a few shot learning model under the hood, so that's a bit hard. For the camera part at least: http://opencv-python-tutroals.readthedocs.io/en/latest/py_tu...

and you could do worse then this https://github.com/fchollet/keras/blob/master/examples/mnist...

It says it in the video: https://deeplearnjs.org

Yes, there is! http://www.wekinator.org

(It is not a Python application but a Java application, but still as fun!)

Are you serious? Do you realize that Chrome is also written by Google and they could theoretically already run arbitrary code on your computer? The potential reputation damage and legal risk for Google would be way too high pull off something like that.

If this happened, the Google Chrome tab would show a camera. Many webcams have adjacent LEDs that identify that they are activated.

Google could theoretically release compromised versions of Google Chrome and only use the permission on devices where webcam LEDs are unlikely (e.g. smartphones), but this is going deep into tin-foil-hat territory.

But won’t it be on just that FQDN alone? Google analytics and ads are served from a totally different domain. What’s the actual concern here?

Chrome does give you this option. It's called "incognito mode"

Good to know, but thankfully easy to remove permissions from the settings.

It works on mobile, it's just slow. Every time we read and write from memory we have to pack and unpack 32 bit floats as 4 bytes without bit shifting operators >.>

Claims like these make privacy-focused efforts less valuable, and I wish people wouldn't make them.

What value is there in taking care to store biometric data only locally, in a separate chip inaccessible even to the OS, if people will simply claim it's equivalent to keeping a remote database of millions of faces?

Facebook beat them to it... that's the whole reason for tagged images imo. Then they can relate identities with each other and with exif gps data to track their movements over time.

I am pretty sure that Apple does not save your image data in any database. Apple is really trying to differentiate itself on privacy.

Also, I don't think that this sends any data to Google, since it trains the neural net in the browser. You could even verify this yourself by looking at the source code.

The machine learning is done in the browser with deeplearn.js, so the images aren't being sent to Google's servers.

The faces are unlabeled, and I'm not sure what that data would be good for. If Google really wanted face data, they could look at:

- Gmail / Google Plus / Google Apps profile pictures

- Google Street View

- Google Hangouts

- implementing a primitive Face ID or Snapchat-style camera on Google Android

- the large mass of face pictures that they index with Google Images

What did Apple beat them to? FaceID is said to not upload data off-device.

It's good to be paranoid about it but at the same time it's quite a cool thing to offer people.

Also I think a lot of the processing is done in the browser using deeplearning.js, so I don't know how much is sent back to Google.

Don't need to, they've got Youtube or so. People has been providing free data set to Google for years anyway.

Don't worry some comment on a forum said they'd never do this, so I think we're all good!

kinda, ya