New html parser

[q0w]

Is your feature request related to a problem? Please describe.

As I mentioned earlier, html5lib will be removed from pip (it already does not switch to html5lib by default). So maybe rewrite to html.parser or add 3th-party lib parser, like html5lib.
Or use a faster parser, like selectolax to improve performance.

Describe the solution you'd like

I think, html.parser is slow, as html5lib too. So adding selectolax can be a solution.

[frostming]

IMO html parser shouldn't be the performance bottleneck unless you can provide some proof.

[frostming]

At present, PDM is reusing the ability of pip for index parsing so there isn't room for customizing the parser. But we are in the process of dropping pip and third-party HTML parsers may be worth considering.

[q0w]

Do you think dropping pip in pdm will be earlier than pip will fully remove html5lib from vendors?

[abersheeran]

https://peps.python.org/pep-0691/

Maybe don’t need new html parser

[q0w]

But existing api would not be deprecated soon

[frostming]

On PDM 2.0 we switched from pip to unearth, which uses html.parser.

[frostming]

Please test it on 2.0.0a1