Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Not sure about this: configure our submodules to be shallow #79

Draft
wants to merge 1 commit into
base: pyston_main
Choose a base branch
from

Conversation

kmod
Copy link
Contributor

@kmod kmod commented Aug 30, 2021

Fixes #75 issue 1

There are some drawbacks to this so I don't know if we should merge this, but it was easy to change so I'm putting it up.

With the change:

  • the .git folder goes from 3.8GB to 3.2GB
  • the total checkout goes from 6.1GB to 5.5GB
  • the initial clone takes much longer. I didn't time it but I estimate it went from about 1 minute to 5 minutes. I think this is because of the extra load on the server, where shallow clones don't benefit from the same optimizations that a full clone receives.
  • It'd be annoying to change the submodules, though we rarely do that

@kmod kmod requested a review from undingen August 30, 2021 21:40
Copy link
Contributor

@undingen undingen left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I don't know enough about git so really decide if this a good idea or not.
But in general I sounds like a good trade-off to me (less disk usage and network bandwidth).
How will the workflow for updating a submodule for us change? I guess we will have to do a full checkout but is something else necessary?

@kmod
Copy link
Contributor Author

kmod commented Aug 31, 2021

I think I'm -1 on this change mostly because of the clone-time increase and that it's only a 20% improvement in disk space, but I could be convinced

As for workflow, I think you can revert this commit and then when you update your submodules they will convert back to full clones, then you can update them, and then presumably people with shallow clones will get updated to a new shallow clone.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Repo size - Submodules / Parallel Building Memory Usage
2 participants