Project vs model - ways to set up to run on cluster - Forum - Delft3D Flexible Mesh
Forum
- Home
- 3. Archive
- General
- Project vs model - ways to set up to run on cluster
Project vs model - ways to set up to run on cluster
Julius Schlumberger, modified 4 Years ago.
Project vs model - ways to set up to run on cluster
Youngling Posts: 15 Join Date: 12/20/20 Recent Posts
Dear all,
I have a set of 8 (offline) nested models (DFlow only) which I want to run using outputs from a larger model. My plan was to model these nested ones on a Linux cluster simulatenously (for time efficiency reasons, every nested models takes about 3 days. So I could model all sub-models within 3 days instead of 24). However, just opening Delft3D-FM parallel and simulating does not work since they would all access the same .dia file. Also, MPI seems not applicable since all nested models are distinct from each other and have separate boundary forcings.
I looked into the manual to see, if it would be possible to run multiple models "at once" from outside the GUI but could not find a clear answer.
In ch..3.5.1 it is mentioned that a project could consist of different models which can be run "at once". Does "at once" mean that they queue up and the second model starts being processed after the first model of the project is finished? Since the GUI seems to work only on Windows, is there a corresponding way to set up the run.sh file for Linux for this "at once" modelling? And if so, does Delft3D automatically detect the set of available machines/cores and distributes the models accordingly or do they have to be set manually? If so, how?
Further in the text it is mentioned: "There is no model intelligence in the < .dsproj> configuration file, meaning that the models can also be run outside the GUI from the <*.dsproj_data> folder." In my understanding, this means that I have to set up a bat file, which will run the models individually. But this would then definitely be in a queue type process, right?
I also had the idea to export the dimr_config.xml on the project level (containing multiple models), but this seems not possible. Does that mean, it is only possible to run a set of models subsequentially.
The last idea was to install D3DFM in different locations of the cluster and run every single one of them for a specific model. However, this seems to me rather dowdy, overpowered and infeasible (since I would need to get access to up to eight different licenses - under the assumption that the different installations would work fine and not interfere with each other.
Did anybody already faced a similar situation or knows something about it? Any feedback welcome.
Thank you very much and a great Sunday!
All the best,
Julius
I have a set of 8 (offline) nested models (DFlow only) which I want to run using outputs from a larger model. My plan was to model these nested ones on a Linux cluster simulatenously (for time efficiency reasons, every nested models takes about 3 days. So I could model all sub-models within 3 days instead of 24). However, just opening Delft3D-FM parallel and simulating does not work since they would all access the same .dia file. Also, MPI seems not applicable since all nested models are distinct from each other and have separate boundary forcings.
I looked into the manual to see, if it would be possible to run multiple models "at once" from outside the GUI but could not find a clear answer.
In ch..3.5.1 it is mentioned that a project could consist of different models which can be run "at once". Does "at once" mean that they queue up and the second model starts being processed after the first model of the project is finished? Since the GUI seems to work only on Windows, is there a corresponding way to set up the run.sh file for Linux for this "at once" modelling? And if so, does Delft3D automatically detect the set of available machines/cores and distributes the models accordingly or do they have to be set manually? If so, how?
Further in the text it is mentioned: "There is no model intelligence in the < .dsproj> configuration file, meaning that the models can also be run outside the GUI from the <*.dsproj_data> folder." In my understanding, this means that I have to set up a bat file, which will run the models individually. But this would then definitely be in a queue type process, right?
I also had the idea to export the dimr_config.xml on the project level (containing multiple models), but this seems not possible. Does that mean, it is only possible to run a set of models subsequentially.
The last idea was to install D3DFM in different locations of the cluster and run every single one of them for a specific model. However, this seems to me rather dowdy, overpowered and infeasible (since I would need to get access to up to eight different licenses - under the assumption that the different installations would work fine and not interfere with each other.
Did anybody already faced a similar situation or knows something about it? Any feedback welcome.
Thank you very much and a great Sunday!
All the best,
Julius