Flake8 --config option not supported

[jamesbraza]

Firstly, this is an awesome project, I have been waiting for this for years! So before I begin, keep it up, you're helping make the Python community amazing!

---

When I go to the directory with pyproject.toml and invoke flake8, everything works fine. 👍

However, when I try flake8 --config pyproject.toml, I get the following:

Traceback (most recent call last):
  File "/path/to/venv/bin/flake8", line 8, in <module>
    sys.exit(main())
  File "/path/to/venv/lib/python3.10/site-packages/flake8/main/cli.py", line 22, in main
    app.run(argv)
  File "/path/to/venv/lib/python3.10/site-packages/flake8/main/application.py", line 336, in run
    self._run(argv)
  File "/path/to/venv/lib/python3.10/site-packages/flake8/main/application.py", line 324, in _run
    self.initialize(argv)
  File "/path/to/venv/lib/python3.10/site-packages/flake8/main/application.py", line 296, in initialize
    cfg, cfg_dir = config.load_config(
  File "/path/to/venv/lib/python3.10/site-packages/flake8/options/config.py", line 80, in load_config
    if not cfg.read(config, encoding="UTF-8"):
  File "/path/to/.pyenv/versions/3.10.3/lib/python3.10/configparser.py", line 698, in read
    self._read(fp, filename)
  File "/path/to/.pyenv/versions/3.10.3/lib/python3.10/configparser.py", line 1117, in _read
    raise e
configparser.ParsingError: Source contains parsing errors: 'pyproject.toml'
	[line 60]: ']\n'
	[line 65]: ']\n'
	[line 102]: ']\n'
	[line 111]: ']\n'
	[line 168]: ']\n'

All of the lines reported (60, 65, 102, 111, 168) correspond with the ending ] from a list. Also, some of them are for configs for other tools (e.g. pylint).

For example, line 60 and 65 are shown below (I added a comment to make it clear):

[tool.pylint.messages_control]

# Disable the message, report, category or checker with the given id(s).
disable = [
    "missing-docstring",  # Let pep257 take care of docstrings
    "empty-docstring",  # Let pep257 take care of docstrings
    "too-few-public-methods",  # Don't care for this level of linting
    "too-many-ancestors",  # XYZ makes heavy use of inheritance
    "fixme",  # codetags are useful
    "too-many-arguments",  # Don't care to enforce this
    "wrong-import-order",  # Rely on isort for this
    "ungrouped-imports",  # Rely on isort for this
    "unused-wildcard-import",  # Wildcard imports are convenient
    "wildcard-import",  # Wildcard imports are convenient
    "unsubscriptable-object",  # Buggy, SEE: https://github.com/PyCQA/pylint/issues/3637
    "unexpected-keyword-arg",  # pylint doesn't understand XYZ
    "logging-fstring-interpolation",  # f-strings are convenient
    "invalid-name",  # Align with equations (e.g. short names, upper case)
]  # Line 60

# Enable the message, report, category or checker with the given id(s).
enable = [
    "useless-suppression",  # Print unused `pylint: disable` comments
]  # Line 65

Is there anything you can do about this? I use flake8 --config in CI, so currently I can't use Flake8-pyproject until this is fixed.

Versions

I am using Python 3.10.3 and the following:

flake8==5.0.4
Flake8-pyproject==1.1.0.post0

[john-hen]

Hi. And thanks. Yeah, I don't like my repos to be cluttered with a variety of config files, one for each tool, and it seems that many people feel the same.

As for this particular case, I'm not surprised at all it crashes. You're giving Flake8 a .toml file where it expects an .ini file, so it spits it out. This plug-in here doesn't patch Flake8's own --config option. I guess, it could, in principle. But then I'd ask myself what the purpose of that is.

According to Flake8's documentation of the --config=<config>option:

Provide a path to a config file that will be the only config file read and used. This will cause Flake8 to ignore all other config files that exist.

Flake8-pyproject already does just that. It ignores "all other config files that exist" if you run it from the folder that contains pyproject.toml:

It then creates a RawConfigParser instance, converting from the TOML input format, and passes it on to Flake8 while discarding configuration options that would otherwise be sourced from elsewhere.

[jamesbraza]

I think the detail you're missing is one can have a monorepo-ish repo, with 2+ subpackages inside one repo. Each subpackage has its own unique pyproject.toml, and lives in a subfolder of the repo root. In other words, in my use case I don't have just one pyproject.toml at the repo root, I have many inside subfolders, and thus am forced to use --config.

To add to this, since a pre-commit gets originated from the repo root, I have to do something like this:

repos:
  - repo: https://gitlab.com/pycqa/flake8
    rev: 3.9.2
    hooks:
      - id: flake8
        name: flake8 subfolder 1
        alias: flake8-subfolder1
        files: ^subfolder1/
        args:
          - "--config=subfolder1/setup.cfg"
        additional_dependencies:
          - flake8-bugbear
          - flake8-docstrings

Ideally I can leverage Flake-pyproject and change to this:

        args:
          - "--config=subfolder1/pyproject.toml"

However, without --config supporting pyproject.toml, this isn't possible at the moment

[john-hen]

I see. But then the problem is that you cannot set the working directory for the pre-commit command. This seems to be a frequent complaint: pre-commit/pre-commit#1417.

There are also a number of other packages that do the same, or almost the same, as this one here. I've mentioned some (probably not all) of them in #2. Since they each patch Flake8 somewhat differently, you might find one that works for your use case.

[jamesbraza]

Thanks for sharing the alternatives! And let's just ignore the pre-commit anecdote, I feel it's beside the point.

I will say, given this project's core goal is integrating a new config file type to flake8, imo it's surprising to not support flake8's non-default path to config option. It feels like --config is directly in the core wheelhouse/responsibility of Flake8-pyproject.

Open to answering any follow up questions, thanks for entertaining this nonetheless!

[john-hen]

You have a point there. Though I also wrote in the ReadMe that users have to run the flake8 command from the same folder that pyproject.toml is in. So this behavior is as documented, if you will.

The reason for this scope limitation is that it simplifies the code quite a bit. Which is always a good thing. I want to spend as little time as possible maintaining this. I just want to use it in CI, for other projects.

But I'm not against adding this, provided it doesn't complicate things too much. Based on your use case, it looks like it would be easy to test with the fixtures that already exist. So the test suite would only need minor changes.

I may look into this at some point. But if you want to give it a shot, please do. The first step would be to find out where in its code Flake8 parses that --config option. Though if that happens before the plug-ins are loaded (which is likely, actually) then... well, then it's not gonna be an easy fix, I'm afraid. (Then again, hard to say how complicated or easy it is before going down that rabbit hole.)

[justcallmelarry]

I went another way for solving the same issue instead: #11

Currently running for myself, but I would also understand if this is not the way that you would choose to solve it.

Let me know if I can adapt it somehow so that it might also help out others. :)

Great plugin, in any case! I really like not having to run a separate command for having my config in pyproject.toml, and now I can finally drop the tox.ini file for good.

[john-hen]

@justcallmelarry That looks pretty good, thanks! Definitely doesn't add too much complexity, and even comes with a test.

I'll just have to find some time to cut a new release, but I'll merge that eventually.

[john-hen]

A custom command-line option --toml-config is supported as of version 1.2.0, released today.

[jamesbraza]

Already using it, love it!! ❤️

[sbor23]

@jamesbraza did this work out for you using pre-commit? I tried to use it but somehow the --toml-config is not applied when run using pre-commit

  - repo: https://github.com/pycqa/flake8
    rev: 6.1.0
    hooks:
      - id: flake8
        additional_dependencies:
          - flake8-pyproject
          - flake8-bugbear
        args:
          - --toml-config web/pyproject.toml
        files: ^web/

However running flake8 --toml-config web/pyproject.toml works just fine with the same (latest) versions.

---

Edit: Just found out what's going on. The following was the output:
--toml-config web/pyproject.toml:0:1: E902 FileNotFoundError: [Errno 2] No such file or directory: '--toml-config web/pyproject.toml'

So I changed the args line to:

          - --toml-config=web/pyproject.toml

Now it works... 👀

I will open a bug report with pre-commit, it seems that it doesn't handle the args correctly.

[jamesbraza]

Glad you figured it out.

I have args: [--toml-config=test/pyproject.toml], which is one line if you wanna be more concise