PyTorch: An Imperative Style, High-Performance Deep Learning Library [pdf] (opens in new tab)

(arxiv.org)

113 pointsstablemap6y ago16 comments

16 comments

jspisak6y ago

Finally the solution to all of your PyTorch citation problems! :)

crazypyro6y ago

Relevant Github issue: https://github.com/pytorch/pytorch/issues/4126

godelski6y ago

It is surprising to me that they don't put this in the README.

(I do know it is in the root directory[0], it is just common practice to have it in the README. To be stupidly obvious)

[0] https://github.com/pytorch/pytorch/blob/master/CITATION

scythe6y ago

Man it’s kinda sad as a Lua fan to see so much interest in a project where the main goal is just to not use Lua.

I guess academics like familiarity and Lua insistently refuses to be like other languages (arrays and maps in one type, 1-based arrays, nonstandard builtin patterns, etc).

elcomet6y ago

> the main goal is just to not use Lua.

I don't think that's accurate. People don't really care about Lua, they don't like or dislike it, they just don't know it. The goal of the project is to use python, because people care about Python.

It just happened that porting (lua) torch to Python was chosen, but it could have been another framework.

cfusting6y ago

It's just about package support and the community. If researchers and practitioners were choosing a language based on merit alone it would probably be Julia for native speed and support for scientific computing. It's nice to have a toy language you appreciate but recall the goal is to write math into algorithms; the language is just tool.

etaioinshrdlu6y ago

As someone who's been a software person for 15 years I am so glad deep learning is centralizing on python. It helps so much to share tools and to use a relatively boring language. Lua offered literally nothing but ecosystem problems...

uoaei6y ago

Academics simply don't have the time to learn languages that do not have substantial ecosystems and relative ease-of-use.

scythe6y ago

It’s true - there have been attempts to fix it, but nobody has created something other people want to use. The Torch project largely replaced all of the then-popular Lua packages — wxLua was dropped for Torch’s internal qtLua, the Lua concurrency libraries (Lanes and luaproc) were ignored in favor of zeroMQ, LPeg and Lua patterns were generally less popular than PCRE and Re2 bindings, et cetera. Maybe Torch is to blame (NIH syndrome), maybe the Lua packages weren’t up to the task, maybe communication within the community is too hard (Lua lacks centralized discussion channels where experienced users are regularly active), but in the end, Lua didn’t come away looking good here.

Learning a new language wasn’t too hard when that language was Python, after all.

1 more reply

amrrs6y ago

If you want to read it online - https://www.arxiv-vanity.com/papers/1912.01703/

wjn06y ago

ah, yes, why squint at a PDF when you can squint at LaTeX compile errors instead

zapnuk6y ago

Is there a typo in Listing 1?

The forward function of the conv net should use:

t3 = self.fc(t2)

instead of:

t3 = self.fc(t1)

AFAIK the nn.functional.relu function is NOT inplace by default [1]

https://pytorch.org/docs/stable/nn.functional.html

MiroF6y ago

yes that's a typo

foxes6y ago

It's a bit funny to call it imperative, when really at the end of the day, the objective is to get something where you have very little insight into what the neural net. is doing to the state.

j / k navigate · click thread line to collapse

16 comments

jspisak6y ago

Finally the solution to all of your PyTorch citation problems! :)

crazypyro6y ago

Relevant Github issue: https://github.com/pytorch/pytorch/issues/4126

godelski6y ago

It is surprising to me that they don't put this in the README.

(I do know it is in the root directory[0], it is just common practice to have it in the README. To be stupidly obvious)

[0] https://github.com/pytorch/pytorch/blob/master/CITATION

scythe6y ago

Man it’s kinda sad as a Lua fan to see so much interest in a project where the main goal is just to not use Lua.

I guess academics like familiarity and Lua insistently refuses to be like other languages (arrays and maps in one type, 1-based arrays, nonstandard builtin patterns, etc).

elcomet6y ago

> the main goal is just to not use Lua.

I don't think that's accurate. People don't really care about Lua, they don't like or dislike it, they just don't know it. The goal of the project is to use python, because people care about Python.

It just happened that porting (lua) torch to Python was chosen, but it could have been another framework.

cfusting6y ago

etaioinshrdlu6y ago

uoaei6y ago

Academics simply don't have the time to learn languages that do not have substantial ecosystems and relative ease-of-use.

scythe6y ago

Learning a new language wasn’t too hard when that language was Python, after all.

1 more reply

amrrs6y ago

If you want to read it online - https://www.arxiv-vanity.com/papers/1912.01703/

wjn06y ago

ah, yes, why squint at a PDF when you can squint at LaTeX compile errors instead

zapnuk6y ago

Is there a typo in Listing 1?

The forward function of the conv net should use:

t3 = self.fc(t2)

instead of:

t3 = self.fc(t1)

AFAIK the nn.functional.relu function is NOT inplace by default [1]

https://pytorch.org/docs/stable/nn.functional.html

MiroF6y ago

yes that's a typo

foxes6y ago

It's a bit funny to call it imperative, when really at the end of the day, the objective is to get something where you have very little insight into what the neural net. is doing to the state.

j / k navigate · click thread line to collapse