Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Refactor check() #7288

Merged
merged 5 commits into from
Aug 17, 2022
Merged

Refactor check() #7288

merged 5 commits into from
Aug 17, 2022

Conversation

DanielNoord
Copy link
Collaborator

  • Add yourself to CONTRIBUTORS if you are a new contributor.
  • Write a good description on what the PR does.

Type of Changes

Type
βœ“ πŸ”¨ Refactoring

Description

Refs #7263.

Blocked by #7286.

@DanielNoord DanielNoord added Blocked 🚧 Blocked by a particular issue Maintenance Discussion or action around maintaining pylint or the dev workflow Skip news πŸ”‡ This change does not require a changelog entry labels Aug 10, 2022
@coveralls
Copy link

coveralls commented Aug 10, 2022

Pull Request Test Coverage Report for Build 2875868156

  • 47 of 47 (100.0%) changed or added relevant lines in 1 file are covered.
  • 1 unchanged line in 1 file lost coverage.
  • Overall coverage increased (+0.007%) to 95.252%

Files with Coverage Reduction New Missed Lines %
pylint/lint/pylinter.py 1 95.25%
Totals Coverage Status
Change from base Build 2872236273: 0.007%
Covered Lines: 16872
Relevant Lines: 17713

πŸ’› - Coveralls

@github-actions

This comment has been minimized.

Copy link
Member

@Pierre-Sassoulas Pierre-Sassoulas left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks good, I only have time for a quick partial review right now, sorry :)

pylint/lint/pylinter.py Show resolved Hide resolved
Comment on lines 735 to 742
except astroid.AstroidError as ex:
template_path = prepare_crash_report(
ex, fileitem.filepath, self.crash_file_path
)
msg = get_fatal_error_message(fileitem.filepath, template_path)
self.add_message(
"astroid-error", args=(fileitem.filepath, msg), confidence=HIGH
)
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Not sure here, but did we remove the distinction between pylint error and astroid error ? It seems the "fatal" message can't be raised anymore.

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think this still exists? It seems to be covered?

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I made a search for "fatal" and found nothing. I think we'd raise an 'astroid-error' now instead of 'fatal' ? Maybe we don't need the distinction and can simplify (the suggestion of new error was to keep that behaviour)

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

get_fatal_error_message is just a helper message that returns a string that points to the template. I don't think we need to change anything here.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm thinking of self.add_message("astroid-error", , previously there was a self.add_message("fatal", that is not raised anymore. I.e. The new code always raise an astroid-error, even when the error come from pylint or somewhere else like an OSError "no space in device", or something unexpected.

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Oh, my bad. I didn't read this correctly. You're right, this is a mistake. Fix incoming.

pylint/lint/pylinter.py Show resolved Hide resolved
@github-actions

This comment has been minimized.

@github-actions

This comment has been minimized.

assert not err
files = tmpdir.listdir()
assert len(files) == 1
assert "pylint-crash-20" in str(files[0])
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Will python/pylint still be relevant in 2100 ? If so we're creating work for the next generation of maintainers πŸ˜„

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copied from the test above πŸ˜„

with open(files[0], encoding="utf8") as f:
content = f.read()
assert "Failed to import module spam." in content
assert any(m.symbol == "fatal" for m in linter.reporter.messages)
Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Previous test makes no sense as this test tries to lint the current file. Since the crash report includes the current file in the report whatever you put in an assertion will always end up in the crash report as well.

Since the test doesn't matter that much anyway I thought replacing was best.

@github-actions

This comment has been minimized.

@DanielNoord
Copy link
Collaborator Author

@Pierre-Sassoulas Gentle ping as this is very prone to creating merge conflicts and I'd like to merge quickyl

@github-actions
Copy link
Contributor

πŸ€– Effect of this PR on checked open source code: πŸ€–

Effect on pandas:
The following messages are now emitted:

  1. no-member:
    Instance of 'int' has no 'encoding' member
    https://github.com/pandas-dev/pandas/blob/main/pandas/tests/config/test_localization.py#L140
  2. redefined-variable-type:
    Redefinition of i2 type from pandas.core.indexes.datetimes.DatetimeIndex to pandas.core.indexes.period.PeriodIndex
    https://github.com/pandas-dev/pandas/blob/main/pandas/tests/indexes/period/test_indexing.py#L615
  3. redefined-variable-type:
    Redefinition of i2 type from pandas.core.indexes.datetimes.DatetimeIndex to pandas.core.indexes.base.Index
    https://github.com/pandas-dev/pandas/blob/main/pandas/tests/indexes/datetimes/test_indexing.py#L142
  4. redefined-variable-type:
    Redefinition of i2 type from pandas.core.indexes.datetimes.DatetimeIndex to pandas.core.indexes.base.Index
    https://github.com/pandas-dev/pandas/blob/main/pandas/tests/indexes/datetimes/test_indexing.py#L212
  5. redefined-variable-type:
    Redefinition of columns type from pandas.core.indexes.base.Index to pandas.core.indexes.datetimes.DatetimeIndex
    https://github.com/pandas-dev/pandas/blob/main/pandas/tests/frame/methods/test_reset_index.py#L371
  6. redefined-variable-type:
    Redefinition of join_index type from pandas.core.indexes.base.Index to pandas.core.indexes.datetimes.DatetimeIndex
    https://github.com/pandas-dev/pandas/blob/main/pandas/core/reshape/merge.py#L1068
  7. redefined-variable-type:
    Redefinition of index type from pandas.core.indexes.base.Index to pandas.core.indexes.datetimes.DatetimeIndex
    https://github.com/pandas-dev/pandas/blob/main/pandas/core/indexes/api.py#L165
  8. redefined-variable-type:
    Redefinition of new_index type from pandas.core.indexes.base.Index to pandas.core.indexes.datetimes.DatetimeIndex
    https://github.com/pandas-dev/pandas/blob/main/pandas/core/indexes/base.py#L1316

The following messages are no longer emitted:

  1. redefined-variable-type:
    Redefinition of i2 type from pandas.core.indexes.numeric.Float64Index to pandas.core.indexes.period.PeriodIndex
    https://github.com/pandas-dev/pandas/blob/main/pandas/tests/indexes/period/test_indexing.py#L615
  2. redefined-variable-type:
    Redefinition of result type from pandas.core.indexes.numeric.Float64Index to pandas.core.indexes.interval.IntervalIndex
    https://github.com/pandas-dev/pandas/blob/main/pandas/tests/indexes/categorical/test_astype.py#L41
  3. redefined-variable-type:
    Redefinition of i2 type from pandas.core.indexes.numeric.Float64Index to pandas.core.indexes.base.Index
    https://github.com/pandas-dev/pandas/blob/main/pandas/tests/indexes/datetimes/test_indexing.py#L142
  4. redefined-variable-type:
    Redefinition of i2 type from pandas.core.indexes.numeric.Float64Index to pandas.core.indexes.base.Index
    https://github.com/pandas-dev/pandas/blob/main/pandas/tests/indexes/datetimes/test_indexing.py#L212
  5. redefined-variable-type:
    Redefinition of columns type from pandas.core.indexes.base.Index to pandas.core.indexes.numeric.Float64Index
    https://github.com/pandas-dev/pandas/blob/main/pandas/tests/frame/methods/test_reset_index.py#L371
  6. redefined-variable-type:
    Redefinition of idx type from pandas.core.indexes.interval.IntervalIndex to pandas.core.indexes.numeric.Float64Index
    https://github.com/pandas-dev/pandas/blob/main/pandas/tests/dtypes/test_inference.py#L1664
  7. redefined-variable-type:
    Redefinition of join_index type from pandas.core.indexes.base.Index to pandas.core.indexes.numeric.Float64Index
    https://github.com/pandas-dev/pandas/blob/main/pandas/core/reshape/merge.py#L1068
  8. redefined-variable-type:
    Redefinition of new_index type from pandas.core.indexes.base.Index to pandas.core.indexes.numeric.Float64Index
    https://github.com/pandas-dev/pandas/blob/main/pandas/core/indexes/base.py#L1316

Effect on sentry:
The following messages are now emitted:

  1. unneeded-not:
    Consider changing "not settings.SENTRY_EVENTSTREAM == 'sentry.eventstream.kafka.KafkaEventStream'" to "settings.SENTRY_EVENTSTREAM != 'sentry.eventstream.kafka.KafkaEventStream'"
    https://github.com/getsentry/sentry/blob/master/src/sentry/runner/commands/devserver.py#L264
  2. unneeded-not:
    Consider changing "not settings.SENTRY_EVENTSTREAM == 'sentry.eventstream.kafka.KafkaEventStream'" to "settings.SENTRY_EVENTSTREAM != 'sentry.eventstream.kafka.KafkaEventStream'"
    https://github.com/getsentry/sentry/blob/master/src/sentry/runner/commands/devserver.py#L273
  3. too-many-instance-attributes:
    Too many instance attributes (12/11)
    https://github.com/getsentry/sentry/blob/master/src/sentry/eventstore/models.py#L34
  4. too-many-instance-attributes:
    Too many instance attributes (23/11)
    https://github.com/getsentry/sentry/blob/master/src/sentry/mediators/param.py#L7

The following messages are no longer emitted:

  1. import-error:
    Unable to import 'sentry.utils.distutils.commands.build_assets'
    https://github.com/getsentry/sentry/blob/master/src/sentry/utils/distutils/__init__.py#L3
  2. import-error:
    Unable to import 'sentry.utils.distutils.commands.build_integration_docs'
    https://github.com/getsentry/sentry/blob/master/src/sentry/utils/distutils/__init__.py#L4
  3. import-error:
    Unable to import 'sentry.utils.distutils.commands.build_js_sdk_registry'
    https://github.com/getsentry/sentry/blob/master/src/sentry/utils/distutils/__init__.py#L5
  4. import-error:
    Unable to import 'sentry.utils.distutils.commands.base'
    https://github.com/getsentry/sentry/blob/master/src/sentry/utils/distutils/commands/build_js_sdk_registry.py#L13
  5. missing-function-docstring:
    Missing function or method docstring
    https://github.com/getsentry/sentry/blob/master/src/sentry/utils/distutils/commands/build_js_sdk_registry.py#L42
  6. no-self-use:
    Method could be a function
    https://github.com/getsentry/sentry/blob/master/src/sentry/utils/distutils/commands/build_js_sdk_registry.py#L42
  7. too-few-public-methods:
    Too few public methods (1/2)
    https://github.com/getsentry/sentry/blob/master/src/sentry/utils/distutils/commands/build_js_sdk_registry.py#L39
  8. import-error:
    Unable to import 'sentry.utils.distutils.commands.base'
    https://github.com/getsentry/sentry/blob/master/src/sentry/utils/distutils/commands/build_assets.py#L12
  9. missing-function-docstring:
    Missing function or method docstring
    https://github.com/getsentry/sentry/blob/master/src/sentry/utils/distutils/commands/build_assets.py#L38
  10. missing-function-docstring:
    Missing function or method docstring
    https://github.com/getsentry/sentry/blob/master/src/sentry/utils/distutils/commands/build_assets.py#L42
  11. no-self-use:
    Method could be a function
    https://github.com/getsentry/sentry/blob/master/src/sentry/utils/distutils/commands/build_assets.py#L42
  12. missing-function-docstring:
    Missing function or method docstring
    https://github.com/getsentry/sentry/blob/master/src/sentry/utils/distutils/commands/build_assets.py#L45
  13. missing-function-docstring:
    Missing function or method docstring
    https://github.com/getsentry/sentry/blob/master/src/sentry/utils/distutils/commands/build_assets.py#L93
  14. attribute-defined-outside-init:
    Attribute 'asset_json_path' defined outside init
    https://github.com/getsentry/sentry/blob/master/src/sentry/utils/distutils/commands/build_assets.py#L39
  15. no-member:
    Instance of 'Param' has no 'name' member
    https://github.com/getsentry/sentry/blob/master/src/sentry/mediators/external_requests/issue_link_requester.py#L116
  16. no-member:
    Instance of 'Param' has no 'schema' member
    https://github.com/getsentry/sentry/blob/master/src/sentry/mediators/sentry_app_components/preparer.py#L25
  17. no-member:
    Instance of 'Param' has no 'schema' member
    https://github.com/getsentry/sentry/blob/master/src/sentry/mediators/sentry_app_components/preparer.py#L40
  18. no-member:
    Instance of 'Param' has no 'schema' member
    https://github.com/getsentry/sentry/blob/master/src/sentry/mediators/sentry_app_components/preparer.py#L58

This comment was generated for commit 1147c08

Copy link
Member

@Pierre-Sassoulas Pierre-Sassoulas left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM ! Sorry for the delay, it was a little hard to review as it seems the code was moved around but the diff is not very helpful. And sometime it's more than moving around (like the fatal/astroid-error change).

@DanielNoord
Copy link
Collaborator Author

LGTM ! Sorry for the delay, it was a little hard to review as it seems the code was moved around but the diff is not very helpful. And sometime it's more than moving around (like the fatal/astroid-error change).

No worries, just thought you might have missed this as you had reviewed more recent PRs. Thanks for the review πŸ˜„

@DanielNoord DanielNoord merged commit a2c57ec into pylint-dev:main Aug 17, 2022
@DanielNoord DanielNoord deleted the lint-discovery-2 branch August 17, 2022 15:14
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Maintenance Discussion or action around maintaining pylint or the dev workflow Skip news πŸ”‡ This change does not require a changelog entry
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants