Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Optimize software/versions endpoint database query #24236

Open
ksykulev opened this issue Nov 28, 2024 · 0 comments
Open

Optimize software/versions endpoint database query #24236

ksykulev opened this issue Nov 28, 2024 · 0 comments
Labels
~backend Backend-related issue. bug Something isn't working as documented ~engineering-initiated Engineering-initiated story, such as a bug, refactor, or contributor experience improvement. #g-endpoint-ops Endpoint ops product group :incoming New issue in triage process. ~released bug This bug was found in a stable release.

Comments

@ksykulev
Copy link
Contributor

Fleet 4.60.0 on Chrome 131.0.6778.86 running on macOS


💥  Actual behavior

When loading the https://dogfood.fleetdm.com/software/versions endpoint, the database query and post query processing causes a large amount of memory to be consumed in certain cases. See listSoftwareDB https://github.com/fleetdm/fleet/blob/main/server/datastore/mysql/software.go#L885. The query used joins the software table with software_cve and cve_meta. So despite the UI table only having 20 records, the query can pull hundreds of rows from the database.

🧑‍💻  Steps to reproduce

Find a software version that has many CVEs associated with it. You can use this query to identify which softwares have the most CVEs in your database:

SELECT
  software_id,
  count(1)
FROM software_cve
LEFT JOIN software ON software.id = software_cve.software_id
GROUP BY software_id
ORDER BY count(1) DESC

🕯️ More info (optional)

Split from #23679

TBD

@ksykulev ksykulev added #g-endpoint-ops Endpoint ops product group :incoming New issue in triage process. :release Ready to write code. Scheduled in a release. See "Making changes" in handbook. :reproduce Involves documenting reproduction steps in the issue bug Something isn't working as documented ~backend Backend-related issue. ~engineering-initiated Engineering-initiated story, such as a bug, refactor, or contributor experience improvement. labels Nov 28, 2024
@ksykulev ksykulev added this to the 4.61.0-tentative milestone Nov 28, 2024
@sharon-fdm sharon-fdm removed the :release Ready to write code. Scheduled in a release. See "Making changes" in handbook. label Dec 4, 2024
@lukeheath lukeheath added ~released bug This bug was found in a stable release. and removed :reproduce Involves documenting reproduction steps in the issue labels Dec 6, 2024
@lukeheath lukeheath removed this from the 4.61.0 milestone Dec 10, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
~backend Backend-related issue. bug Something isn't working as documented ~engineering-initiated Engineering-initiated story, such as a bug, refactor, or contributor experience improvement. #g-endpoint-ops Endpoint ops product group :incoming New issue in triage process. ~released bug This bug was found in a stable release.
Development

No branches or pull requests

3 participants