Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

sceua & mpc : writing data in txt error for mpc #304

Open
ZhiqiangD opened this issue Mar 27, 2023 · 3 comments
Open

sceua & mpc : writing data in txt error for mpc #304

ZhiqiangD opened this issue Mar 27, 2023 · 3 comments

Comments

@ZhiqiangD
Copy link

Hello everyone, I'm new to using spotpy and I have a question. I'm currently trying to use the sceua algorithm with mpc to calibrate VIC. The problem is that the parameters I need to write to a txt file for VIC to use are getting overlapped or mixed up during the parallel calibration process. I've tried adding process locks but it didn't work. Can anyone please suggest a solution or offer some advice? Thank you so much!

@ZhiqiangD
Copy link
Author

Hi,
in tutorial_parallel_computing_hymod.py,
If your model in def simulation reads any files, make sure, they are unique for each CPU. Otherwise, things get messed up...
spot_setup = spot_setup(parallel=parallel)

Could you please advise me on how to ensure that each CPU uses a file that is unique to it, for example, by following the instructions in this link (#266 (comment)) or by using joblib (#266 (comment))?

Thanks @thouska

@thouska
Copy link
Owner

thouska commented Mar 28, 2023

Hi @ZhiqiangD,
thank you for your message and for using spotpy for your research. And perfect that you found already the relevant information for you issue. For the multiprocessing spotpy is using pathos.multiprocessing. So, we just need to figure out, how to ask pathos, on which CPU it is currently running. This value can be used to make copies of your VIC model inputs.
This link looks like it covers this point.

@ZhiqiangD
Copy link
Author

Hi @thouska,
I apologize for the delayed response. After conducting some experiments, I have successfully achieved 'mpc' calibration. However, I have encountered two issues with 'mpc'. Firstly, I am unable to control the number of CPU cores utilized by the program, as it automatically utilizes all available CPU cores on my computer, even when I set ngs=4 or ngs=8. Secondly, after completing multi-core runs, the results in the database still show the number of rate determinations rather than the rate determination times the number of CPU cores used. I was wondering if the algorithm automatically selects the best result from each CPU folder and includes it in the final database.
Thank you for your assistance in advance!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants