undefined | Better HN

0 pointsjnbiche12y ago0 comments

Yes, Numba isn't a general purpose compiler of Python -- sorry if I implied that. And to be clear, this isn't a Python runtime you'd compile to JS for the browser, this is something that could compile scientific/numeric Python into JS. However, it is exactly the type of code that Numba targets that would be the most interesting to port to JS. And it's quite simple to port such code in Numba; usually all that's required is some incantation involving 'jit' or 'autojit'.

I mean, you're not going to be using tkinter or running Python networking modules on the browser, anyway. If you could compile some of the scientific and numeric Python that exists into JS, it would allow a lot of scientific computing to be distributed and run on the web, client-side, including most of the machine learning and NLP code that exists now in Python.

From just an hour of research, it doesn't look like it would be very hard. But I only know enough about the two projects to be dangerous.

At the very least, it's an interesting idea.

EDIT: TO clarify, I'm referring to "distributed" as in distribute an asset (in this case a JS script) to a computer to be run, not distribute a work load.

0 comments

sleepingpills12y ago

The problem with your suggestion is that scientific/numeric python tends to rely very heavily on numpy/scipy/pandas, most of which are in turn written in C/Fortran/Cython. Meaning, you could only funnel the glue code to JS, not the code that does the heavy lifting.

You'd need to also port the above libraries to JS, which would be a pretty large undertaking.

Finally, grid computing is not any easier in JS than in python, since python already has lots of bindings for the required tools (e.g. take a look at http://star.mit.edu/cluster/ which even comes preinstalled on EC2 AMIs).

Edit: I just noticed that you suggested using JS for distributed computing in lieu of python. Care to suggest a scenario where JS would be a better suited language? Why would I want to run scientific computing in browsers?

jnbicheOP12y ago

You know, sometimes people just do stuff like this for the hell of it. We call it hacking. There doesn't have to be a well-founded reason for people to wonder if they can make Part A fit Part B. I'm not proposing this to a dissertation committee. I'm not suggesting it as a means of grid computing. I'm simply suggesting that it might be fun/interesting to play around with some scripts in Python and try to port [edit: this should be transpile not port] them to JS using via Numba, LLVM and emscripten. Obviously, a rewrite of pandas in JS is not in the cards.

Usually, hacking like this leads to nothing, but sometimes it leads to everything.

Care to suggest a reason why I shouldn't do it if I feel like it? I'm really gritting my teeth in forbearance here.

PS: I wasn't even thinking of grid computing when I mentioned this. I meant "distributed" in the sense of distributing an asset, not a work load. Nor was I thinking of whether or not JS was a "better language". I was just wondering if it could be done.

Edit: I can't reply to your comment below, so I'll write a few words here. First, apologies if you weren't trying to be snarky. Phrases like "Care to suggest...?" can in certain contexts imply disdain and snark. If I misread your intent, then my apologies.

Second, Numba actually is a kind of a JIT, so it's interesting on several levels. I don't think you'd get any added benefit from the JS JIT in terms of performance, but instead it would be another way to distribute scientific Python scripts. For example, as far-fetched as it is right now, this would allow ipython notebooks to be distributed and run in their "static" form" without needing a local or remote Python server (note that here again I'm not meaning any kind of distributed or grid computing).

Finally, scientific Python may actually not be the least amenable code to bring into JS, since pure computational code has limited I/O and thus we don't have to worry about the DOM. Also, note that the C libraries that are used in pandas and elsewhere could in theory be compiled into JS using Emscripten. A large task, to be sure, but an interesting idea.

Unfortunately, I'm too busy at the moment to start up a project like this, but it's definitely going on my list of things to hack around with.

sleepingpills12y ago

There is no need to be defensive, I wasn't implying that you mustn't do it.

You brought up scientific python specifically, but this tend to involve (in my experience) overwhelmingly native code with small amounts of plumbing written in python. Numba is already a restricted subset of python, and without porting at least parts of numpy, you are further restricted to just a subset of that subset (by giving up vectorized numpy ops).

I totally agree that playing around with it will be fun, I'm merely pointing out the fact that you chose perhaps the least amenable kind of python code for this particular task.

Again, by all means go for it! I agree that there are lessons to take away even if it doesn't lead to anything specific.

Edit: you also seemed to imply that there is some inherent benefit in moving scientific/numeric computation from python to JS (and the browser). I was honestly interested in the reasoning behind this. I know that numeric python could do well with a proper JIT for example and a lot of research went into JS VMs (pypy doesn't count yet).

j / k navigate · click thread line to collapse

0 pointsjnbiche12y ago0 comments

From just an hour of research, it doesn't look like it would be very hard. But I only know enough about the two projects to be dangerous.

At the very least, it's an interesting idea.

EDIT: TO clarify, I'm referring to "distributed" as in distribute an asset (in this case a JS script) to a computer to be run, not distribute a work load.

0 comments

sleepingpills12y ago

You'd need to also port the above libraries to JS, which would be a pretty large undertaking.

jnbicheOP12y ago

Usually, hacking like this leads to nothing, but sometimes it leads to everything.

Care to suggest a reason why I shouldn't do it if I feel like it? I'm really gritting my teeth in forbearance here.

Unfortunately, I'm too busy at the moment to start up a project like this, but it's definitely going on my list of things to hack around with.

sleepingpills12y ago

There is no need to be defensive, I wasn't implying that you mustn't do it.

I totally agree that playing around with it will be fun, I'm merely pointing out the fact that you chose perhaps the least amenable kind of python code for this particular task.

Again, by all means go for it! I agree that there are lessons to take away even if it doesn't lead to anything specific.

j / k navigate · click thread line to collapse