[CodeStyle] Flake8 tracking issue #46039

SigureMo · 2022-09-14T09:01:54Z

项目说明

用于跟踪 Call-for-Contributions: Flake8 引入计划及相关 CodeStyle 升级各个任务的计划和进度

部分针对任意文本的代码风格全量修复需要同样为如文档（.md、.rst）、C++ 相关文件（.cc、.c、.h、.cu、.cpp、.cmake、CMakeLists.txt）一并修复

项目初始状态

Flake8 默认启用的三个工具（pycodestyle、pyflakes、mccabe）共包含了一共 132 个子项。Paddle 中存在问题的子项共 64 个。

错误码参考：
- E、W：PyCodeStyle Error code
- F：Flake8 Error code
根据错误码的子项统计说明见：[CodeStyle] init flake8 configs #45943 (comment) [CodeStyle] add pre-commit hook remove-tabs for python files #46290 (comment)
推进方式：采用在 .flake8 配置文件中 ignore 现存的错误码来推进，修复一类错误就去掉一个错误码

Flake8 阶段性成果总结（54/62）

序号	子任务和完成时间	对应错误码和出现次数	相关 PR	修复行数	增量拦截手段 pre-commit hook	作者
1	fail to compile a file into an AST (2022/08/23)	E999(1)	#45287	1	flake8	SigureMo
2-3	trailing whitespace (2022/09/19)	W291(4146) W293(1696)	#45937, #46042, #45990, #46006, #46080, #44474, #46067, #46081	C++ (602) Python (6000) 文档(67) hook/yaml(63)	`trailing-whitespace`（自动修复） + flake8	SigureMo, betterpig
4-5	Tabs (2022/09/27)	E101(587) W191(587)	#46288, #46357, #46309, #46236, #46290, #46232	C++ (34) Python (246)	`remove-tabs`（自动修复） + flake8	SigureMo
6	import module from line N shadowed by loop variable (2022/10/10)	F402(1)	#46698, #46699	1	flake8	caolonghao
7	.has_key() is deprecated, use ‘in’ (2022/10/10)	W601(3)	#46702, #46703	3	flake8	caolonghao
8	dictionary key `name` repeated with different values (2022/10/10)	F601(7)	#46719, #46720	14	flake8	caolonghao
9	test for membership should be ‘not in’ (2022/10/11)	E713(22)	#46734, #46855	22	flake8	caolonghao
10	test for object identity should be ‘is not’ (2022/10/11)	E714(4)	#46741	4	flake8	caolonghao
11	assertion test is a tuple, which is always `True` (2022/10/11)	F631(2)	#46739	14	flake8	gglin001, caolonghao
12	use `==/!=` to compare `str`, `bytes`, and `int` literals (2022/10/11)	F632(18)	#46708, #46856	18	flake8	caolonghao
13	`.format(...)` missing argument (2022/10/11)	F524(1)	#46745, #46854	1	flake8	caolonghao
14	f-string without any placeholders (2022/10/11)	F541(33)	#46700	36	flake8	caolonghao
15	expected 1 blank line before a nested definition (2022/10/11)	E306(1)	#46895, #46897	1	flake8	SigureMo
16	Module imported but unused (2022/10/12)	F401(6750)	cattidea#4 (33个PR) #47455	4891	autoflake（自动修复） + flake8	SigureMo, Yulv-git, Aurelius84
17	‘from module import *’ used; unable to detect undefined names (2022/10/19)	F403(57)	#46946, #46947	51	flake8	SigureMo
18	invalid escape sequence ‘x’ (2022/10/20)	W605(172)	#46752, #46753	77	flake8	caolonghao
19-44	常见格式问题（black 可解决）(2022/10/24)	E121(8) E122(81) E123(12) E125(168) E126(723) E127(140) E128(207) E129(9) E131(45) E201(29) E202(11) E225(61) E226(93) E228(3) E241(2) E251(109) E261(11) E271(4) E272(1) E301(7) E302(3) E303(7) E305(2) E502(400) E701(108) W504(949)	#46014, #47266, #47268, #47146	-	black（自动修复） + flake8	SigureMo
45	`.format(...)` unused named arguments (2022/10/24)	F522(1)	#46743, #47287	2	flake8	caolonghao
46	missing whitespace after ‘,’, ‘;’, or ‘:’ (2022/10/25)	E231(3)	#47297, #47298	5	flake8	SigureMo
47	comparison to None should be ‘if cond is None:’ (2022/11/01)	E711(168)	#47452, #47465	357	flake8	SigureMo
48	comparison to True should be ‘if cond is True:’ or ‘if cond:’ (2022/11/01)	E712(331)	#47464	403	flake8	SigureMo
49-50	inline/block comment should start with ‘# ‘ (2022/11/07)	E262(17) E265(29)	#47687	50	flake8	SigureMo
51	too many leading ‘#’ for block comment (2022/11/09)	E266(106)	#47772, #47773	159	flake8	caolonghao
52	undefined name `name` (2022/11/15)	F821(75)	#47756, #47869, #47899, #47968	223	flake8	SigureMo
53	multiple imports on one line (2022/12/05)	E401(8)	#48538	94	flake8	SigureMo
54	redefinition of unused `name` from line `N` (2022/12/08)	F811(123)	#48745	65	flake8	SigureMo
-	module level import not at top of file (TODO)	E402(1309)	#50458 暂时 hold			@SigureMo
-	line too long (82 > 79 characters) (TODO)	E501(16517)	同上
-	do not compare types, use ‘isinstance()’ (TODO)	E721(8)	同上
-	do not use bare except, specify exception instead (TODO)	E722(151)	同上
-	do not assign a lambda expression, use a def (TODO)	E731(64)	同上
-	do not use variables named ‘l’, ‘O’, or ‘I’ (TODO)	E741(144)	同上
-	name may be undefined, or defined from star imports: module (TODO)	F405(212)	同上
-	local variable `name` is assigned to but never used (TODO)	F841(2124)	同上

Note

这里仅统计已经完成的任务，具体各个任务的计划和进度见下方评论区。

错误码出现次数和最终修复行数会有差异：一方面两者统计时间不一样，另一方面错误码统计方式是出现次数，有一行同一个错误码出现多次的情况，也有一处错误码占据多行的情况。

统计行数为 diff 中增加行数与减少行数中较大值，因为部分任务需要删除内容，而部分则需要增加内容，较大值更能反映实际修改行数

其他成果总结

子任务和完成时间	相关 PR
修复 cpplint hook 在 CI 不生效的问题（2022/09/19）	#46136
修复 Remove-CRLF / Detect Private Key hook 在 CI 不生效的问题（2022/09/20）	#46154 #46155 #46156
对错误使用 `yapf: disable` 的文件进行调整（2022/09/22）	#46410
引入 isort 工具（2022/12/01）	#46475, #48390, #48401, #48402, #48404, #48522

The text was updated successfully, but these errors were encountered:

SigureMo · 2022-09-20T18:33:21Z

Flake8 各个子任务（错误码）的计划和进度

初始化 Flake8 配置：[CodeStyle] init flake8 configs #45943
Flake8 tasks Tracking board：https://github.com/orgs/cattidea/projects/4/views/2

✅ assertion test is a tuple, which is always `True` [F631]

存量修复
- caolonghao 原修复方案及相关讨论：[CodeStyle][F631] Remove parentheses in assertion #46714
- gglin001 修复方案：[IPU] rm fp16 dtype_check in UTs #46739
配置文件修改同 F632

✅ use `==/!=` to compare `str`, `bytes`, and `int` literals [F632]

存量修复 [CodeStyle][F632] Replace 'is' and 'is not' with == and != respectively #46708
配置文件修改 [CodeStyle][F631][F632] Remove F631,F632 in .flake8 config #46856

✅ `.format(...)` missing argument [F524]

存量修复 [CodeStyle][F524] Add escape symbols to curly braces #46745
配置文件修改 [CodeStyle][F524][F541] Remove F524,F541 in .flake8 config #46854

✅ f-string without any placeholders [F541]

存量修复 [CodeStyle][F541] Convert f-strings without curly braces to normal strings #46700
配置文件修改同 F524

✅ expected 1 blank line before a nested definition [E306]

存量修复 [CodeStyle][E306] add a blank line before nested function #46895
配置文件修改 [CodeStyle][E306] remove error code E306 in flake8 config #46897

✅ Module imported but unused [F401]

存量修复，已由 33 个 PR 完成（由于太多，这里折叠一下，点击展开）

增量修复
- [CodeStyle][F401] fix incremental flake8 F401 and F541 issues #46926
- 一个 revert：[CodeStyle][F401] revert an import in sampcd_processor #46944

✅ ‘from module import *’ used; unable to detect undefined names [F403]

存量修复 [CodeStyle][F403] expand star import #46946
配置文件修改 [CodeStyle][F403] remove error code E403 in flake8 config #46947

✅ invalid escape sequence ‘x’ [W605]

存量修复 [CodeStyle][W605] Add escape symbols to some strings #46752
配置文件修改 [CodeStyle][W605] Update .flake8 config #46753

✅ `.format(...)` unused named arguments [F522]

存量修复 [CodeStyle][F522] Remove unused arguments #46743
配置文件修改 [CodeStyle][F522] Update .flake8 config #47287

✅ missing whitespace after ‘,’, ‘;’, or ‘:’ [E231]

存量修复 [CodeStyle][E231] remove unnecessary , #47297
配置文件修改 [CodeStyle][E231] update flake8 config #47298

✅ comparison to None should be ‘if cond is None:’ [E711]

存量修复 [CodeStyle][E711] use is/is not for comparison with None #47452
配置文件修改 [CodeStyle][E711][E712] update flake8 config #47465

✅ comparison to True should be ‘if cond is True:’ or ‘if cond:’ [E712]

存量修复 [CodeStyle][E712] use if cond/if cond is True for comparison with True #47464
配置文件修改同 E711

✅ inline/block comment should start with ‘# ‘ [E262][E265]

存量修复 & 配置修改 [CodeStyle][E262][E265] make comments start with # #47687

✅ too many leading ‘#’ for block comment [E266]

存量修复 [CodeStyle][E266] remove multiple '#' in comments #47772
配置文件修改 [CodeStyle][E266] update .flake8 config #47773

✅ undefined name `name` [F821]

存量修复
- [CodeStyle][F821] fix test_exception in test_unpool3d_op and test_unpool_op #47756（75 -> 70）
- [CodeStyle][F821] refactor unittests utility function parameterized related code #47869（70 -> 43）
- [CodeStyle][F821] fix undefined variables due to missing imports, misspelled variable names #47899（43 -> 21）
- [CodeStyle][F821] fix remaining F821 issues #47968（20 -> 0）
增量修复
- [CodeStyle][F821] add a missing import #48006

✅ multiple imports on one line [E401]

存量修复（含大量 F811）：[CodeStyle][E401][F811] remove redefined imports and variables #48538

✅ redefinition of unused `name` from line `N` [F811]

存量修复：[CodeStyle][F811] fix some test cases shadowed by the same name #48745

Other Error Code...

TODO...

paddle-bot bot assigned zyfncg Sep 14, 2022

This comment was marked as off-topic.

Sign in to view

luotao1 assigned luotao1 and Ligoml and unassigned zyfncg Sep 14, 2022

luotao1 added the PFCC Paddle Framework Contributor Club，https://github.com/PaddlePaddle/community/tree/master/pfcc label Sep 14, 2022

This was referenced Oct 28, 2022

[CodeStyle][E712] use if cond/if cond is True for comparison with True #47464

Merged

[CodeStyle][E711][E712] update flake8 config #47465

Merged

luotao1 mentioned this issue Oct 31, 2022

[Tools]Add autoflake pre-commit hook to remove unused-imports/var #47455

Merged

This was referenced Nov 4, 2022

[CodeStyle][E262][E265] make comments start with # #47687

Merged

[CodeStyle][F821] fix test_exception in test_unpool3d_op and test_unpool_op #47756

Merged

This was referenced Nov 8, 2022

[CodeStyle][E266] remove multiple '#' in comments #47772

Merged

[CodeStyle][E266] update .flake8 config #47773

Merged

Ligoml mentioned this issue Nov 15, 2022

🎁飞桨快乐开源活动表单 #48019

Closed

SigureMo mentioned this issue Nov 19, 2022

[CodeStyle][py36-][E722] remove import handling for collections.abc in different python versions #48165

Merged

This was referenced Nov 29, 2022

[CodeStyle][E401][F811] remove redefined imports and variables #48538

Merged

[CodeStyle][F811] fix some test cases shadowed by the same name #48745

Merged

luotao1 added this to Call for Contributions Dec 6, 2022

luotao1 moved this to In Progress in Call for Contributions Dec 6, 2022

luotao1 assigned SigureMo and unassigned Ligoml Dec 6, 2022

luotao1 mentioned this issue Dec 8, 2022

为paddle添加type hinting 功能 PaddlePaddle/community#346

Closed

SigureMo mentioned this issue Feb 13, 2023

[CodeStyle] update flake8 config #50458

Merged

luotao1 closed this as completed Feb 14, 2023

github-project-automation bot moved this from In Progress to Done in Call for Contributions Feb 14, 2023

paddle-bot bot added the status/close 已关闭 label Feb 14, 2023

SigureMo mentioned this issue Mar 16, 2023

Introducing Ruff tracking issue #51729

Closed

SigureMo mentioned this issue Jun 28, 2023

CINN 代码风格改进计划 #54953

Closed

gouzil mentioned this issue Sep 9, 2023

[CodeStyle] update ruff v0.0.289 and black 23.9.1 #57145

Merged

SigureMo added this to Nyakku @PaddlePaddle 🐾 Sep 15, 2023

gouzil mentioned this issue Sep 15, 2023

[CodeStyle] Ruff upgrade and addition of paddle.base check #57367

Closed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

[CodeStyle] Flake8 tracking issue #46039

[CodeStyle] Flake8 tracking issue #46039

SigureMo commented Sep 14, 2022 •

edited by luotao1

Loading

This comment was marked as off-topic.

SigureMo commented Sep 20, 2022 •

edited

Loading

[CodeStyle] Flake8 tracking issue #46039

[CodeStyle] Flake8 tracking issue #46039

Comments

SigureMo commented Sep 14, 2022 • edited by luotao1 Loading

项目说明

项目初始状态

Flake8 阶段性成果总结 （54/62）

其他成果总结

This comment was marked as off-topic.

SigureMo commented Sep 20, 2022 • edited Loading

Flake8 各个子任务（错误码）的计划和进度

✅ Trailing Whitespace [W291][W293]

✅ Tabs [E101][W191]

✅ Import module from line N shadowed by loop variable [F402]

✅ .has_key() is deprecated, use ‘in’ [W601]

✅ dictionary key name repeated with different values [F601]

✅ test for membership should be ‘not in’ [E713]

✅ test for object identity should be ‘is not’ [E714]

✅ assertion test is a tuple, which is always True [F631]

✅ use ==/!= to compare str, bytes, and int literals [F632]

✅ .format(...) missing argument [F524]

✅ f-string without any placeholders [F541]

✅ expected 1 blank line before a nested definition [E306]

✅ Module imported but unused [F401]

✅ ‘from module import *’ used; unable to detect undefined names [F403]

✅ invalid escape sequence ‘x’ [W605]

✅ .format(...) unused named arguments [F522]

✅ missing whitespace after ‘,’, ‘;’, or ‘:’ [E231]

✅ comparison to None should be ‘if cond is None:’ [E711]

✅ comparison to True should be ‘if cond is True:’ or ‘if cond:’ [E712]

✅ inline/block comment should start with ‘# ‘ [E262][E265]

✅ too many leading ‘#’ for block comment [E266]

✅ undefined name name [F821]

✅ multiple imports on one line [E401]

✅ redefinition of unused name from line N [F811]

Other Error Code...

SigureMo commented Sep 14, 2022 •

edited by luotao1

Loading

Flake8 阶段性成果总结（54/62）

SigureMo commented Sep 20, 2022 •

edited

Loading

✅ Import `module` from line `N` shadowed by loop variable [F402]

✅ dictionary key `name` repeated with different values [F601]

✅ assertion test is a tuple, which is always `True` [F631]

✅ use `==/!=` to compare `str`, `bytes`, and `int` literals [F632]

✅ `.format(...)` missing argument [F524]

✅ `.format(...)` unused named arguments [F522]

✅ undefined name `name` [F821]

✅ redefinition of unused `name` from line `N` [F811]