undefined | Better HN

0 pointsjoshuamorton6y ago0 comments

> It's more elegant firstly because it follow use

No it doesn't. Currently `def foo(self): pass` is called as `instance.foo()`. You're suggesting that `def self.foo(): pass` would be called as `instance.foo()`, except now it looks like self and instance are syntactically related in ways that they aren't.

> Again, "this" is just an identifier here, and doesn't have any special meaning.

But the grammar is no longer LL(1), and you have weird conditionally valid syntax, like `.` in a function name is valid only in a class block.

> "def foo(x)" has the same meaning both inside and outside of a class declaration

This is a stretch, especially since you're now optimizing for the uncommon case. Staticmethods are rare compared to instance methods (I'll go further and claim that static methods are an antipattern in python, modules are valid namespaces, you can stick a function in a module and couple it to a class also defined in that module and nothing bad will happen. Banning staticmethods entirely doesn't reduce expressiveness). Aligning staticmethods with functions, instead of aligning instance methods with functions (as python does currently) encourages you to do the wrong thing.

> classmethod

Your changes don't affect classmethod at all, if anything they'd make classmethod more of a special case. How do you signal that `self.foo()` takes `self` as the class instead of the instance?

> It's especially annoying when you have class attributes that happen to reference a function, because conversion from functions to methods is a runtime thing - so you have to remember to wrap those in staticmethod() as well.

What do you mean? Like

    class Foo:
        a = Foo.func()
        @staticmethod
        def func():
            return 1

I'll say again: staticmethods are an antipattern in python:

    def func()
        return 1

    class Foo:
        a = func()

works just as well, better in fact. Modules are great namespaces. Classes are more than namespaces, and if all you need is a namespace, you shouldn't use a class.

> because conversion from functions to methods is a runtime thing

I'd also quibble with this: it's a binding thing.

    class Foo:
        a = Foo.foo(None)
        def foo(self):
            return 1

will work, and if you check, type(Foo.foo) is still just `function`, its only when you create an instance of Foo that the function `foo` is bound to the instance, and when that is done, the bound `foo` is converted to a method object. This was different in python2, where Foo.foo and instance.foo were both "instancemethod" objects, but in python3, Foo.foo is a plain old function, and instance.foo is a method.

Specifically this means that if you can get your hands on the `method` constructor (like with `type(instance.method)`), you can then do silly things like

    class A():
      def foo(self): pass
    instance = A()
    def f(self): return 5
    assert instance.func() == 5

and this will work. You'll have bound the function to the instance. Of course, if you stick an attribute on `instance` (or `A`), and reference `self.attribute` in the function, this will still work. (this also lets you do things like bind a given instance of a function to a different instance of the class, but that's because the method constructor is essentially just partial with some bookkeeping for class information)

0 comments

int_19h6y ago

Python grammar is not LL(1) in general; just look at set and dict literals. This is really no different than ":" after an expression being legal inside a dict literal (and how you know that it is a dict literal).

But also, there's no reason to make those legal only inside classes. All it needs to do is make "def foo.bar" produce a different type of function, that has the method descriptor-producing behavior that is currently implemented directly on regular functions.

As far as less vs more common case - I think it's more important to optimize for obviousness and consistency. If "def foo" is a function, it should always be a function, and functions should behave the same in all contexts. They currently don't - given class C and its instance I, C.f is not the same object as I.f, and only one of those two is what "def" actually produced.

What I meant by function references inside classes is this:

   class Foo:
      pass

   Foo.bar = lambda: 123
   foo = Foo()
   print(foo.bar())

This blows up with "TypeError: <lambda>() takes 0 positional arguments but 1 was given", because lambda is of type "function", and it gets the magic treatment when it's read as a member of the instance. So you have to do this:

   Foo.bar = staticmethod(lambda: 123)

and even then this is only possible when you know that the value is going to end up as a class attribute. Sometimes, you do not - you pass a value to some public function somewhere, and it ends up stashed away as a class attribute internally. And it all works great, until you pass a value that just happened to be another function or lambda.

On the other hand, this only applies to objects of type "function", not all callables. So e.g. this is okay:

   Foo.bar = functools.partial(lambda x: x, 123)

because what partial() returns is not a function. Conversely, this means that you can't use partial() to define methods, which can be downright annoying at times. Suppose you have:

   class Foo:
      def frob(self, x, y): ...

and you want to define some helper methods for preset combinations of x and y. You'd think this would work:

   class Foo:
      def frob(self, x, y): ...
      frob_xyzzy = functools.partial(frob, x=1, y=2)
      frob_whammo = functools.partial(frob, x=3, y=4)

except it doesn't - while frob_xyzzy() and frob_whammo() both have the explicit "self" argument, they aren't "proper" functions, and thus that argument doesn't get treated as the implicit receiver:

   foo = Foo()
   foo.frob(x=0, y=0)    # okay
   foo.frob_xyzzy()      # TypeError: frob() missing 1 required positional argument: 'self' 
   foo.frob_whammo(foo)  # okay!

Which is to say, this all is a mess of special cases. You can argue that this all isn't really observable in the "common case" - the problem is that, as software grows more complex, the uncommon cases become common enough that you have to deal them regularly, and then those inconsistencies add even more complexity into the mix that you have to deal with - just when you already thought you had your plate full.

joshuamortonOP6y ago

> Python grammar is not LL(1) in general; just look at set and dict literals. This is really no different than ":" after an expression being legal inside a dict literal (and how you know that it is a dict literal).

Yes it is[0]. LL(1) Grammars can still be recursive, they just can't change the parsing rules based on distant context.

> As far as less vs more common case - I think it's more important to optimize for obviousness and consistency

Yes, but having the "easiest" thing you do:

    class Foo:
        def bar():
            pass

silently do a usually unwanted thing (create a staticmethod) instead of an obviously wrong thing (raise an error) isn't obvious. It's building a footgun into the language.

The rest of your comment complains about inconsistencies of how python converts various callables to methods. This is a fairly valid and interesting complaint, but has nothing to do with syntax, it is solely a semantic complaint that would be solved by having class creation treat all attributes that are callables as functions. In fact, you could customize class creation yourself this way using __new__, no syntactic changes required.

> as software grows more complex, the uncommon cases become common enough that you have to deal them regularly

While this is true, I think you vastly overestimate how common these constructs are. Like, you're in the realm of "this doesn't appear on github" levels of uncommon.

Personally, again, I think "you can't use partial() to define methods" is a very good thing: if you're doing this, you're into weird metaprogramming land. Its not any harder to, for example, write out

    class Foo:
      def frob(self, x, y): ...
      def frob_xyzzy(self): return self.frob(x=1, y=2)
      def frob_whammo(self): return self.frob(x=3, y=4)

unless you're doing weird metaprogrammy magic and then, as someone who does a lot of weird metaprogrammy magic

1. You deserve what you get

2. You can invoke deeper magic to solve these problems

[0]: https://discuss.python.org/t/switch-pythons-parsing-tech-to-...

int_19h6y ago

Why is it an unwanted thing? By the same logic that demands "self" to be explicit, if you don't specify "self", then you explicitly don't want the method to be an instance method - seems very straightforward to me. And it's still invokable via instance member access, so the user of the class is none the wiser. Where's the footgun?

My complaint is of course more complicated than the syntax alone. I'm just saying that a distinctive syntax for self-as-receiver could be used to drive other changes that would make things more intuitive and self-consistent overall. And, of course, any discussion about "elegance" is going to be inherently subjective.

With respect to commonality of various constructs - this is all from personal experience writing Python code (for a project that is hosted on GitHub, by the way). It doesn't require particularly fancy code to trip that wire in general - it just requires code that tries to be generic, i.e. not make more assumptions that it needs to about the types of values that flow through it. Python classes break that genericity by treating functions, and only functions, in a special way whenever they flow through class attributes. This is particularly egregious in a language where every object is potentially callable, and non-function callables are very common; so functions really aren't all that special in general - except for that one case.

With partial() specifically, of course you can avoid that in this manner. But why would you, if it works with regular functions? I prefer it over defs, not just because it's more concise and avoids repeating things, but because it's also clearer - when you see partial(), it immediately tells you that it's a simple alias, nothing else.

But regardless, it's a function that has a certain documented behavior, and common sense would dictate that this behavior works the same for methods as well as functions. That it doesn't is not an intentional limitation of partial() - it's an unfortunate quirk of the design of methods themselves. And if you don't know exactly how functions become methods in Python, you wouldn't have any reason to expect the behavior that it exhibits. That's why the docs for partial() have to spell it out explicitly: "partial objects defined in classes behave like static methods and do not transform into bound methods during instance attribute look-up" - because that's not a reasonable default assumption!

The bigger problem is that every library that offers a generic wrapper callable has to add the same clause to its docs, because they're all affected in the same way. And if they don't document it, and you use, say, a decorator from a library - how do you know whether the fact that it returns a function and not some other callable is part of the contract that you can rely on, or an implementation detail? Conversely, whenever you implement a decorator, you have to be cognizant that changing the type of the return value from/to plain function can be a breaking change for your clients - and that is even less obvious.

1 more reply

j / k navigate · click thread line to collapse

0 comments

int_19h6y ago

What I meant by function references inside classes is this:

   class Foo:
      pass

   Foo.bar = lambda: 123
   foo = Foo()
   print(foo.bar())

   Foo.bar = staticmethod(lambda: 123)

On the other hand, this only applies to objects of type "function", not all callables. So e.g. this is okay:

   Foo.bar = functools.partial(lambda x: x, 123)

because what partial() returns is not a function. Conversely, this means that you can't use partial() to define methods, which can be downright annoying at times. Suppose you have:

   class Foo:
      def frob(self, x, y): ...

and you want to define some helper methods for preset combinations of x and y. You'd think this would work:

   class Foo:
      def frob(self, x, y): ...
      frob_xyzzy = functools.partial(frob, x=1, y=2)
      frob_whammo = functools.partial(frob, x=3, y=4)

except it doesn't - while frob_xyzzy() and frob_whammo() both have the explicit "self" argument, they aren't "proper" functions, and thus that argument doesn't get treated as the implicit receiver:

   foo = Foo()
   foo.frob(x=0, y=0)    # okay
   foo.frob_xyzzy()      # TypeError: frob() missing 1 required positional argument: 'self' 
   foo.frob_whammo(foo)  # okay!

joshuamortonOP6y ago

Yes it is[0]. LL(1) Grammars can still be recursive, they just can't change the parsing rules based on distant context.

> As far as less vs more common case - I think it's more important to optimize for obviousness and consistency

Yes, but having the "easiest" thing you do:

    class Foo:
        def bar():
            pass

silently do a usually unwanted thing (create a staticmethod) instead of an obviously wrong thing (raise an error) isn't obvious. It's building a footgun into the language.

> as software grows more complex, the uncommon cases become common enough that you have to deal them regularly

While this is true, I think you vastly overestimate how common these constructs are. Like, you're in the realm of "this doesn't appear on github" levels of uncommon.

    class Foo:
      def frob(self, x, y): ...
      def frob_xyzzy(self): return self.frob(x=1, y=2)
      def frob_whammo(self): return self.frob(x=3, y=4)

unless you're doing weird metaprogrammy magic and then, as someone who does a lot of weird metaprogrammy magic

1. You deserve what you get

2. You can invoke deeper magic to solve these problems

[0]: https://discuss.python.org/t/switch-pythons-parsing-tech-to-...

int_19h6y ago

1 more reply

j / k navigate · click thread line to collapse