Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

CPU Usage #8

Closed
edsu opened this issue Dec 30, 2016 · 1 comment
Closed

CPU Usage #8

edsu opened this issue Dec 30, 2016 · 1 comment

Comments

@edsu
Copy link
Member

edsu commented Dec 30, 2016 •
edited
Loading

As Hydrator reads deeper into the tweet id file it consumes more of the CPU. After doing 3 million or so it uses 50% of the CPU on my MacBook Air (1.8 GHz Intel Core i5).

I think this is because for every API request readTweetIds reads the first n lines of the file to get the next 100. Ideally there would be an open filehandle that would be read from rather than having to constantly open/re-open the file.
@edsu
Copy link
Member Author

edsu commented May 2, 2020

Fixed (finally) in v0.0.6!

@edsu edsu closed this as completed May 2, 2020
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
1 participant
@edsu