Skip to content

Commit

Permalink
Add mypy to pre-commit and fix the errors
Browse files Browse the repository at this point in the history
Limit the Python version to =3.10 because of the
`pandas-stubs` typing dependency, see:
pandas-dev/pandas-stubs#400
Poetry command:
```sh
poetry add -G typing mypy pandas-stubs
```
Add the recommended settings for mypy:
https://blog.wolt.com/engineering/2021/09/30/professional-grade-mypy-configuration/
Fix the errors with type ignore as they originate from
wrong typing in pandas/builtin libs and make sure
mypy passes:
```sh
pre-commit run mypy --all-files
```
  • Loading branch information
jond01 committed Nov 22, 2022
1 parent 8bc3a55 commit 058d6fa
Show file tree
Hide file tree
Showing 6 changed files with 129 additions and 10 deletions.
5 changes: 5 additions & 0 deletions .pre-commit-config.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -25,3 +25,8 @@ repos:
entry: poetry run pylint
language: system
types: [python]
- id: mypy
name: mypy
entry: poetry run mypy
language: system
types: [python]
111 changes: 105 additions & 6 deletions poetry.lock

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

13 changes: 12 additions & 1 deletion pyproject.toml
Original file line number Diff line number Diff line change
Expand Up @@ -26,14 +26,18 @@ classifiers = [
]

[tool.poetry.dependencies]
python = "^3.10"
python = ">= 3.10, < 3.11"
pandas = "^1.5.1"
beautifulsoup4 = {version = "^4.11.1", extras = ["lxml", "html5lib"]}

[tool.poetry.group.dev.dependencies]
pre-commit = "^2.20.0"
pylint = "^2.15.6"

[tool.poetry.group.typing.dependencies]
mypy = "^0.991"
pandas-stubs = "^1.5.1.221024"

[tool.poetry.urls]
"Bug Tracker" = "https://github.com/jond01/xil/issues"

Expand All @@ -43,6 +47,13 @@ profile = "black"
[tool.pylint]
enable = ["I"]

[tool.mypy]
check_untyped_defs = true
disallow_untyped_defs = true
warn_unused_ignores = true
warn_return_any = true
show_error_codes = true

[build-system]
requires = ["poetry-core"]
build-backend = "poetry.core.masonry.api"
2 changes: 1 addition & 1 deletion xil/_headers.py
Original file line number Diff line number Diff line change
Expand Up @@ -24,4 +24,4 @@ def get_url_response(
else:
headers = {}
request = urllib.request.Request(url, headers=headers)
return urllib.request.urlopen(request)
return urllib.request.urlopen(request) # type: ignore[no-any-return]
6 changes: 5 additions & 1 deletion xil/fibi.py
Original file line number Diff line number Diff line change
Expand Up @@ -26,7 +26,11 @@

with get_url_response(_FIBI_URL) as response:
dfs = pd.read_html(
response, match=_MATCH, header=_HEADER, encoding=_ENCODING, attrs=_ATTRS
response, # type: ignore[arg-type]
match=_MATCH,
header=_HEADER,
encoding=_ENCODING,
attrs=_ATTRS,
)

df = dfs[0] # It is guaranteed to have at least one element - otherwise an exception
Expand Down
2 changes: 1 addition & 1 deletion xil/union.py
Original file line number Diff line number Diff line change
Expand Up @@ -24,7 +24,7 @@
_HEADER = [0, 1]

with get_url_response(_UNION_URL, headers=_UNION_HEADERS) as response:
df = pd.read_html(response, header=_HEADER)[0]
df = pd.read_html(response, header=_HEADER)[0] # type: ignore[arg-type]

df = df.drop(("Unnamed: 7_level_0", "מכירה"), axis=1) # empty (NaNs) column

Expand Down

0 comments on commit 058d6fa

Please sign in to comment.