Backend implementation of "detect bugs with similar domain names"

[hallvors]

This issue is about coding a backend for the feature described in #307
This work can probably start before the design in #307 is ready. It will most likely depend on the issues database created in #788

• Use tldextract to get domain name in webhooks/helpers.py
• Update db/init.py to add domain name field
• Create new endpoint 'domainsearch' in api/endpoints.py
• Dependency on Migration script for database. See Issue database needs a migration script. #968

[dshgna]

Hi @hallvors: as discussed on IRC, I'll start work from here:

• Use tldextract to get domain name in webhooks/helpers.py
• Update db/init.py to add domain name field
• Create new endpoint 'domainsearch' in api/endpoints.py

Afterwards, how should I continue? Perhaps work on the frontend to check realtime for matches on the domain name?

[hallvors]

Sounds good! I'd suggest as a next step (on the backend - to complete this bug) creating a new endpoint in api/endpoints.py called domainsearch or something like that, accepting one argument - let's call it q. So webcompat.com/api/domainsearch?q=goog will do a lookup in the DB and return

[{"domain": "google.com", "issue":2323, "summary": "apps.google.com - wrong standard gradient syntax "}, {"domain":"google.co.uk", "issue":000, "summary": "foo" } ]

etc.

[hallvors]

(maybe add url too..limit the hits to 10 or something and make sure in the clientside JS to only start searching when 3-4 letters are entered to avoid a "download the internet" torrent..)

[karlcow]

In the list above I would add a reference to a separate issue on migrating the database for backups, restart of the server, and updating the schema.

[karlcow]

@dshgna do not forget to commit your code modifications in a branch and reference this issue. I see you have already ticked two boxes but I don't see any commits.

[karlcow]

I added the list of tasks to the first comment so We can keep track of completed tasks in cross bugs.

[dshgna]

@karlcow: Committed the tasks I've done so far :)

[karlcow]

@dshgna Same comments than the previous PR. Could you avoid large commits with very long commit messages. And do atomic commits. You do not have to push them right away but I would recommend to work on your code in small bits addressing specific section. You will see it's a lot clearer in the future.

See for example what I'm doing in
master...karlcow:710/3

[dshgna]

@karlcow Thanks so much. I will commit so in future commits. I agree it's much more clearer.

[dshgna]

@hallvors @karlcow Just wanted to clarify on the columns returned:

[{domain: "wordpress", issue_id: "11", summary: "wordpress.com - mobile site is not usable"}, {domain: "wordpress", issue_id: "12", summary: "in.wordpress.com - layout is messed up"}]

[karlcow]

@dshgna said on IRC:

[15:12] karlcow: I created a new branch for the changes in domain search as you suggested. Okay to close the existing pull request when making the new pulls request for this branch?

I said yes. So hold on for new comments.

[miketaylr]

As mentioned in the PR for this issue, we want to hold off on any more work towards this feature. It's intended to be one of the projects for the Outreachy intern -- thanks.

[miketaylr]

Closing for now, please see: #307 (comment)