Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

"too many open files" when using input-dir #928

Closed
Velyks opened this issue Aug 21, 2020 · 3 comments · Fixed by #934
Closed

"too many open files" when using input-dir #928

Velyks opened this issue Aug 21, 2020 · 3 comments · Fixed by #934
Assignees
Labels

Comments

@Velyks
Copy link

Velyks commented Aug 21, 2020

Darwin Kernel Version 19.5.0
gomplate version 3.7.0

I'm running gomplate with --input-dir set to a folder with 600~ files in which is causing the following error:
11:06:57 FTL error="failed to open somedir/somefile.yaml\nopen somedir/somefile.yaml: too many open files"

On OSX the default soft ulimit -n is 256. Is it possible for gomplate to respect the ulimit, or to have a configuration option to set the max concurrent open files?

@hairyhenderson
Copy link
Owner

Hi @Velyks - sorry for the slow response on this.

Yeah, it certainly seems like it should be possible to respect the ulimit. I'll try to fix this.

@hairyhenderson hairyhenderson self-assigned this Aug 26, 2020
@hairyhenderson
Copy link
Owner

Ok, I've found the bug - the input files are read one-by-one and closed right after the read is complete, however the output files are all opened at once. I've got an idea on how I can solve this with a minimum of effort, so I'll do that soon, and the fix will be in gomplate 3.8 🙂

@Velyks
Copy link
Author

Velyks commented Aug 26, 2020

Thanks!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
2 participants