Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

False positive __init__ method from base class is not called when the constructor raises NotImplementedError without using abc #3975

Closed
cpitclaudel opened this issue Dec 11, 2020 · 20 comments · Fixed by #7227
Assignees
Labels
False Positive 🦟 A message is emitted but nothing is wrong with the code Needs PR This issue is accepted, sufficiently specified and now needs an implementation
Milestone

Comments

@cpitclaudel
Copy link
Contributor

Is your feature request related to a problem? Please describe

In the following example, Pylint flags the Derived's __init__ method because it doesn't call Bases's __init__.

class Base:
    def __init__(self, a: int, b: str):
        raise NotImplementedError()

class Derived(Base):
    def __init__(self, a: int, b: str):
        self.a = a + 1
        self.b = b[::-1]

This is discussed here: https://stackoverflow.com/questions/25072363/error-init-method-from-base-class-is-not-called-for-an-abstract-class

Describe the solution you'd like

Pylint could (should?) detect this case and not report an error.

Additional context

An easy fix is to remove the init in the base class, but it can be useful to have it there to define what signature __init__ should have.

@hippo91
Copy link
Contributor

hippo91 commented Dec 12, 2020

@cpitclaudel thanks for your report. Why not using abc module and defining the base class as an abstract one?

@cpitclaudel
Copy link
Contributor Author

Are you thinking of this?

from abc import ABCMeta, abstractmethod

class Base:
    __metaclass__ = ABCMeta

    @abstractmethod
    def __init__(self, a: int, b: str):
        pass

class Derived(Base):
    def __init__(self, a: int, b: str):
        super().__init__(a, b)
        self.a = a + 1
        self.b = b[::-1]

That's what one of the answers on stackoverflow suggests, but it's a lot of cruft, and it doesn't prevent Base from being instantiated.

@cpitclaudel
Copy link
Contributor Author

cpitclaudel commented Dec 12, 2020

Why not using abc module and defining the base class as an abstract one?

A different perspective: maybe there should be a pylint rule that suggests moving to an ABS instead of using NotImplementedError(); but this issue is about a different problem, namely pylint suggesting to call an __init__ that raises NotImplementedError/

@hippo91
Copy link
Contributor

hippo91 commented Dec 13, 2020

@cpitclaudel indeed if you use metaclass this way:

from abc import ABCMeta, abstractmethod

class Base(metaclass=ABCMeta):
    @abstractmethod
    def __init__(self, a: int, b: str):
        pass

class Derived(Base):
    def __init__(self, a: int, b: str):
        super().__init__(a, b)
        self.a = a + 1
        self.b = b[::-1]


if __name__ == "__main__":
    x = Base(3, "toto")

and execute this program, you will get:

TypeError: Can't instantiate abstract class Base with abstract methods __init__

Thus it is impossible to instantiate Base class.

However i agree that pylint should be able to detect that the __init__ method of the base class raises NotImplementError and thus should invite the user to switch toward use of ABCMeta.

@hippo91 hippo91 added the Enhancement ✨ Improvement to a component label Dec 13, 2020
@cpitclaudel
Copy link
Contributor Author

cpitclaudel commented Dec 13, 2020

However i agree that pylint should be able to detect that the init method of the base class raises NotImplementError and thus should invite the user to switch toward use of ABCMeta.

👍 , though does the added ABCMeta stuff buy anything compared to the version that just throws an exception? Pylint currently recognizes methods that throw NotImplementedErrors just fine, without recommending to move these too.

And having to call __init__ in the derived class is definitely a downside, no?

@Py-GNU-Unix
Copy link

I agree with the need to ignore super-init-not-called if the super init raises NotImplementedError. That's a important feature, but no one fixed this in a long time. I tried to fix this by myself, but that's the first time that i look into pylint code and i failed. I hope that someone will fix this...

@Py-GNU-Unix
Copy link

Py-GNU-Unix commented Jul 31, 2021

may someone add an high priority tag to this issue? This is nearly a bug.

@Pierre-Sassoulas
Copy link
Member

Hi @Py-GNU-Unix The high priority tag is for issue that have more than 10 thumbsup/heart reactions, but we're reviewing every contribution. If you want to contribute to this issue, it's going to get merged before high priority issues that do not get any attention.

@Pierre-Sassoulas Pierre-Sassoulas added the Needs PR This issue is accepted, sufficiently specified and now needs an implementation label Jul 2, 2022
@Pierre-Sassoulas Pierre-Sassoulas changed the title Pylint suggests calling an __init__ method that raises NotImplementedError() False positive __init__ method from base class is not called when the constructor raises NotImplementedError without using abc Jul 2, 2022
@Pierre-Sassoulas Pierre-Sassoulas added False Positive 🦟 A message is emitted but nothing is wrong with the code and removed Enhancement ✨ Improvement to a component labels Jul 2, 2022
@vivi90
Copy link

vivi90 commented Jul 24, 2022

I think it's NOT an False Positive.
The reason, i think so is:

  • If you are NOT USING the abc package: Might be hard to detect abstract methods in a reliable way. So pylint prevents you here from possible logical issues.
  • If you are USING the abc package: Even if you are not able to instantiate the base class without implementing all abstract methods (including __init__) in an concrete way, you ARE INDEED ABLE to implement some stuff in abstract methods of the base class and call this methods with super() in it's subclass(es). For the __init__ method this means, that pylint prevents you here from violating OOP concepts.

An good practice example from me:

#!/usr/bin/env python3
# -*- coding: utf-8 -*-

"""Abstract class example."""

from abc import ABCMeta, abstractmethod


class Abstract(metaclass=ABCMeta):
    """Abstract class."""

    @property
    @abstractmethod
    def message(self) -> str:
        """str: Return message."""

    @abstractmethod
    def __init__(self, message: str) -> None:
        """Abstract constructor.

        Args:
            message (str):
                Message.

        """


class Concrete(Abstract):
    """Concrete class."""

    @property
    def message(self) -> str:
        """str: Return message."""
        return self._message

    def __init__(self, message: str) -> None:
        """Concrete constructor.

        Args:
            message (str):
                Message.

        """
        super().__init__(message)
        self._message = message


print(Concrete("test").message)

(using class Abstract(metaclass=ABCMeta) instead of class Abstract(ABC) here, because it's more explicit and works also well in combination with Protocol [if really needed])

@cpitclaudel
Copy link
Contributor Author

@vivi90 Your example is great, but this issue is about an __init__ that throws a NotImplementedError. pylint suggests adding a call to that __init__, which would throws a runtime exception. That's not good ^^

@vivi90
Copy link

vivi90 commented Jul 25, 2022

@cpitclaudel

[..] pylint suggests adding a call to that __init__, which would throws a runtime exception. [..]

Yes, thats WRONG behaviour but with the RIGHT goal. 😄
So how about adding an new warning message like use 'abc' instead of raising 'NotImplementedError' here in combination with the still existing '__init__' method from base class is not called message instead of silencing pylint in this case? 🤔

@DanielNoord
Copy link
Collaborator

@cpitclaudel

[..] pylint suggests adding a call to that __init__, which would throws a runtime exception. [..]

Yes, thats WRONG behaviour but with the RIGHT goal. 😄

So how about adding an new warning message like use 'abc' instead of raising 'NotImplementedError' here in combination with the still existing '__init__' method from base class is not called message instead of silencing pylint in this case? 🤔

I think it makes sense to remove abstract methods from this check. Due to historic reasons pylint considers a number of patterns to be "abstract". For example, the default setting is to consider pass abstract as well.
We probably shouldn't change this.

That said, I think an argument can be made to create a code style check that suggests to inherit from abc. That should probably be its own issue though.

@vivi90
Copy link

vivi90 commented Jul 25, 2022

@DanielNoord

I think it makes sense to remove abstract methods from this check. [..]

I think that everything might be better than removing this check for abstract __init__ methods, because of the second point of my explanation above.

[..] We probably shouldn't change this.

Yes, i think the same.

[..] an argument can be made to create a code style check that suggests to inherit from abc. [..]

Yes, exactly.

[..] That should probably be its own issue though.

Discussing this in an own issue might be out of context.
Here we have an good case at hand.

@DanielNoord
Copy link
Collaborator

I think that everything might be better than removing this check for abstract __init__ methods, because of the second point of my explanation above.

Whenever you implement anything in the abstract class but also raise UnimplementedError I don't think pylint should be able to recognise that. That's just an anti-pattern.

Pylint's recognition of abstract methods and classes is actually quite old and robust so I don't think that should limit us.

We tend to separate issues and messages from each other: users might not have all messages turned on so dependencies between messages are best avoided.

Because of this we should take this message and ask ourselves: should super-init-not-called be raised on abstract __init__? I think the answer is no as if there is anything in the __init__ it is not actually abstract.

If this then creates a false positive or negative we should fix our abstract recognition, but the premise of not raising is still valid imo.

@vivi90
Copy link

vivi90 commented Jul 27, 2022

@DanielNoord

Whenever you implement anything in the abstract class but also raise UnimplementedError I don't think pylint should be able to recognise that. That's just an anti-pattern.

Oh no, that's not what i wanted to purpose. 😆

We tend to separate issues and messages from each other: users might not have all messages turned on so dependencies between messages are best avoided.

Yes, that's also not what i wanted to say.
The '__init__' method from base class is not called should not be an dependecy for the other message or the other way around.

Because of this we should take this message and ask ourselves: should super-init-not-called be raised on abstract __init__? I think the answer is no as if there is anything in the __init__ it is not actually abstract.

hm okay so you mean if someone write something like this..

    @abstractmethod
    def __init__(self, message: str) -> None:
        """Abstract constructor.

        Args:
            message (str):
                Message.

        """
        this._message = message

then this __init__ method is not abstract anymore? 🤔
I mean, it's possible to do that and using it in the subclass by super().
That's what i pointed to with:

I think that everything might be better than removing this check for abstract init methods, because of the second point of my explanation above.

If this then creates a false positive or negative we should fix our abstract recognition, but the premise of not raising is still valid imo.

Hmm, for abc it seems still abstract, since there is an @abstractmethod decorator. 🤔

@DanielNoord
Copy link
Collaborator

hm okay so you mean if someone write something like this..

    @abstractmethod

    def __init__(self, message: str) -> None:

        """Abstract constructor.



        Args:

            message (str):

                Message.



        """

        this._message = message

then this __init__ method is not abstract anymore? 🤔

No, as long as something "meaningful" happens in the method it is not considered abstract.

Hmm, for abc it seems still abstract, since there is an @abstractmethod decorator. 🤔

Yeah. So in such cases this shouldn't warn.

@vivi90
Copy link

vivi90 commented Jul 27, 2022

@DanielNoord

Yeah. So in such cases this shouldn't warn.

That's exactly the point i am worried about.
If it does NOT warn in this case, it may happen that:

  1. The user overwrites this __init__ method in the subclass, because it's abstract according to abc.
  2. The user doesn't call super().__init__(...) in the overwritten __init__ method.
  3. The logic in the 'abstract' __init__ method of the base class is never called..

So then i prefer the current behaviour of pylint. Please don't change it.

@DanielNoord
Copy link
Collaborator

I have added a test to #7227 to make the new behaviour more explicit. It should be:

class Base:
    def __init__(self, param: int, param_two: str) -> None:
        raise NotImplementedError()


class Derived(Base):
    def __init__(self, param: int, param_two: str) -> None:
        self.param = param + 1
        self.param_two = param_two[::-1]


class AbstractBase(abc.ABC):
    def __init__(self, param: int) -> None:
        self.param = param + 1

    def abstract_method(self) -> str:
        """This needs to be implemented."""
        raise NotImplementedError()


class DerivedFromAbstract(AbstractBase):
    def __init__(self, param: int) -> None:  # [super-init-not-called]
        print("Called")

    def abstract_method(self) -> str:
        return "Implemented"

Whenever __init__ is abstract itself we don't raise when it isn't called. If the class is abstract but the __init__ isn't, we do raise.

@cpitclaudel
Copy link
Contributor Author

Thanks for your work @DanielNoord

@DanielNoord
Copy link
Collaborator

#7227 was merged so this should be available in 2.15 for all those watching this issue 😄

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
False Positive 🦟 A message is emitted but nothing is wrong with the code Needs PR This issue is accepted, sufficiently specified and now needs an implementation
Projects
None yet
Development

Successfully merging a pull request may close this issue.

6 participants