http://www.nvidia.com/object/gpu-accelerated-applications-te...
But it is a little bit out of date when it comes to version numbers.
If you're going to try TensorBox (https://github.com/TensorBox/TensorBox) it will get a bit harder still because of conflicts and build issues with specific versions of TensorFlow.
There has to be an easier way to distribute a package.
That said, all this is super interesting and Google really moved the needle by opensourcing TensorFlow and other ML packages.
See http://reference.wolfram.com/language/guide/NeuralNetworks.h..., also look at Examples > Applications under http://reference.wolfram.com/language/ref/NetTrain.html for some worked examples. Fun example of live visualization during training (very easy to do, will get even easier in future versions): https://twitter.com/taliesinb/status/839013689613254656
The problem in the end is that the customizability of Mathematica ends right where things get interesting. If you want to show people cool little examples, Mathematica is clean and fast, but you can't build anything serious with it. And by "serious", I guess I mean something with few enough constraints to have an identity of its own, rather than being "a thing you can do with Mathematica."
Another limitation is the data input. Someone needs to rethink it.
I could be wrong. I actually want to be wrong, because of the simplicity and power of Mathematica in its scope.
Programming languages/platforms are network goods. IMHO, Mathematica has tried to swim against this fact and has failed.
Guess the only way is to upgrade to 11.1 and find out, but since you're apparently involved -- might as well ask...
The annoying thing for GPU training is handling the cudNN dependency, which Google's guides are annoyingly lacking.
I've found a nice workaround for that one that does not require registration with nvidia so you can automate it:
ML_REPO_PKG=nvidia-machine-learning-repo-ubuntu1404_4.0-2_amd64.deb
wget http://developer.download.nvidia.com/compute/machine-learning/repos/ubuntu1404/x86_64/${ML_REPO_PKG} -O /tmp/${ML_REPO_PKG}
sudo dpkg -i /tmp/${ML_REPO_PKG}
rm -f /tmp/${ML_REPO_PKG}
CUDA_REPO_PKG=cuda-repo-ubuntu1404_7.5-18_amd64.deb
wget http://developer.download.nvidia.com/compute/cuda/repos/ubuntu1404/x86_64/${CUDA_REPO_PKG} -O /tmp/${CUDA_REPO_PKG}
sudo dpkg -i /tmp/${CUDA_REPO_PKG}
rm -f /tmp/${CUDA_REPO_PKG}
apt-get update
apt-get install libcudnn5-dev
apt-get install libcudnn5The standard build uses a "least common denominator" Intel instruction set (SSE4), but the odds are extremely high that the machine on which you're running tensorflow supports 4.2. Building from source allows you to use the most up-to-date instruction set (the default configuration script at https://www.tensorflow.org will do it automatically).
I've seen dramatic (>50%) reductions in processing time on test scripts by building from source. Note that those tests were built primarily for my own education, not for benchmarking. But the speedup was so dramatic that I couldn't help but notice and probe a little deeper. YMMV depending on the particular application, whether you are using GPU computing (I am not), etc.
But it's a bit disappointing to hear that the build system is something of a nightmare, if I ever wanted to contribute myself. There's always plenty of things to help with, I don't care about the cutting edge of machine learning (I'm happy to submit docs, examples, etc)... Then again, the TF people can't just nerd around on their build system, for dorks like me to maybe write some patches every once in a while. Always great to make it easier, though.
See this issue:
https://github.com/TensorBox/TensorBox/issues/100
and
https://github.com/TensorBox/TensorBox/issues/102
So then we're full-circle and installing from pip which doesn't work :(
sigh.
Anyway, I'll get it to work, somehow.
For CuDNN I've found a good solution, see below.
Tensorflow itself worked ok once I figured out what all the dependencies were, even so I have not been able to get it to use CUDA yet (it only works with the CPU), which is strange because other CUDA stuff works fine.
-(Eng) We need to switch to this new NLP framework
- (VP) Ok, why? Which one is it?
- (Eng) Huh, it's called Parsey McParseface, developed by ...
- (VP) WTF? Don't waste my time with jokes, go build your own
- (Eng) But ...
- (VP) Meeting's over.
...and apparently will release a major version update today. Ouch.
>On the time-honoured benchmark for this task, Parsey McParseface achieves over 94% accuracy, at around 600 words per second. On the same task, spaCy achieves 92.4%, at around 15,000 words per second. The extra accuracy might not sound like much, but for applications, it's likely to be pretty significant.
If spaCy is able to increment the accuracy and maintain the large performance gap, it'll still be my go-to NLP framework!
I do wish SyntaxNet were a bit easier to use. A lot of people have asked for SyntaxNet as a backend for spaCy, and I'd love to be using it in a training ensemble. When I tried this last year, I had a lot of trouble getting it to work as a library. I spent days trying to pass it text in memory from Python, and it seemed like I would have to write a new C++ tensorflow op. Has anyone gotten this to work yet?
I wonder if the spaCy update will go deep :)
I've learned a lot while doing the neural network models, though. The 1.7 model takes advantage of this by having a more sophisticated optimizer. Specifically, I use an online L1 penalty and the Adam optimizer with averaged parameters. The L1 penalty allows control of size/accuracy trade-off.
This means we're finally shipping a small model: 50mb in total, compared to the current 1gb. The small model makes about 15-20% more errors.
[1] https://github.com/tensorflow/models/tree/master/syntaxnet
On HN the idea is to rewrite titles only to make them less misleading (or less baity). Please see https://news.ycombinator.com/newsguidelines.html.
https://github.com/tensorflow/models/blob/master/syntaxnet/g...
This new DRAGNN framework is what I thought the folks here would want to know. Perhaps I should have linked to the github page, rather than the blog announcement.
For the same reason, it probably doesn't make sense to change the current thread to point to that Github page now, since that would orphan the existing discussion.
The bit about guessing the part of speech, stem, etc. for previously unseen words should (I think) make it much more useful in contexts that succumb to neologizing, verbing nouns, nouning verbs, and so on (such as business writing, technical writing, academic papers, science fiction & fantasy, slang, etc.).
I wonder how well it would do at parsing something that seems deliberately impenetrable, like TimeCube rants, or postmodern literary criticism.
Other pathological cases are business books trying to coin a term or twist existing words into new meanings (eg. "cloud"), verbing nouns (incentivize), nouning verbs (likes, learnings), and so on.
Another HN thread has good tutorials for simple uses of Tensorflow: https://news.ycombinator.com/item?id=13464496
However, NNs are optimal for text/image data as they can learn the features. If your data features are already known, you don't necessarily need to use Tensorflow/Keras at all, and you'll have a easier time using conventional techniques like linear/logistic regression and xgboost.
I recommend http://course.fast.ai/ to learn more about the applications of neural networks and how to apply neural networks quickly through python.
It's a very interesting research carried out by Google's research team and I believe this will be especially beneficial for future speech translation algorithms that would bring us a whole new, fresh experience with the way we converse with Alexa, Google Home, Siri, and many more.
If you need to install TensorFlow onto your Windows 10 computer then here's a great guide which I have followed quiet a few times. :)
Alexa, turn the lights on in the kitchen.
Alexa, turn on the kitchen light.
Alexa, light up the kitchen.
Should all accomplish the same task using this framework.
Just simply treating the sentence as a bag of words and looking for "on" or "off" or "change" (and their synonyms) and the presence of known smart objects works extremely well. I could say "Hey Marvin, turn on the lights and TV", or "Hey Marvin, turn the lights and TV on", or even "Hey Marvin, on make lights and TV."
(It's named Marvin it after the android from The Hitchhiker's Guide, my eventual goal is to have it reply with snarky/depressed remarks).
Adding 30 seconds of "memory" of the last state requested also made it seem a million times smarter and turns requests into a conversation rather than a string of commands. If it finds a mentioned smart object with no state mentioned, it assume the previous one.
"Hey Marvin, turn on the lights." lights turn on "The TV too." tv turns on
The downside to this approach is I would be showing it off to friends, and it could mis trigger. "Marvin turn off the lights." lights turn off "That's so cool, so it controls your TV, too?" TV turns off But it was mostly not an issue in real usage.
Ultimately I've got the project on hold for now because I can't find a decent, non-commercial way of converting voice to text. I'd really rather not send my audio out to Amazon/Google/MS/IBM. Not just because of privacy, but cost and "coolness" factor (I want as much as possible processed locally and open-source).
CMUSphinx's detection was mostly very bad. I couldn't even do complex NLP if I wanted because it picks up broken/garbled sentences. I currently build a "most likely" sentence by looping through sphinx's 20 best interpretations of the sentence and grabbing all the words that are likely to be commands or smart objects. I tried setting up Kaldi, but didn't get it working after a weekend and haven't tried again since. I don't really know any other options to use aside from CMUSphone, Kaldi, or a butt SaaS.
I've wanted to add a text messaging UI layer to it. Maybe I'll use that as an excuse to try playing with ParseySaurus.
Same concern here... so my voice->text method is via android's google voice - forced to offline mode. The offline mode is surprisingly good.
Re mis triggers... I also have opencv running on the same android. It only activates the voice recognition when I am actually looking directly at the android device (an old phone).
May also be interpreted as:
Alexa, set fire to the kitchen
Alexa turns the gas stove directly to 'high', and waits.
Oh lord, is this the spark that lights the google skynet powder keg
