System get restarted multiple times during running the simulation

Provides a system for patient-specific cardiovascular modeling and simulation.
POST REPLY
User avatar
niloofar borzooei
Posts: 9
Joined: Sun May 19, 2024 6:21 pm

System get restarted multiple times during running the simulation

Post by niloofar borzooei » Wed Jul 17, 2024 8:24 pm

Hi,
I am using a Colossus supercomputer with Windows 10 to run a simulation(the project is downloaded from vascular repository).
-Device Specifications:
(Processor AMD Ryzen Threadripper 1950X 16-Core Processor 3.40 GHz
Installed RAM 64.0 GB (63.9 GB usable)
Device ID 6EC9C4F4-C739-4211-B614-971213890F27
Product ID 00329-00000-00003-AA464
System type 64-bit operating system, x64-based processor)

-SimVascular version: 23.03.27

-SvsSolver :2019.05.28

However, the system get restarted every 10 (more or less) steps. I don't know what the problem is. I would really appreciate it if you could help me with it.

Thank you
Niloofar Borzooie

User avatar
David Parker
Posts: 1653
Joined: Tue Aug 23, 2005 2:43 pm

Re: System get restarted multiple times during running the simulation

Post by David Parker » Thu Jul 18, 2024 5:06 pm

Hi Niloofar,

However, the system get restarted every 10 (more or less) steps. I don't know what the problem is. I would really appreciate it if you could help me with it.


What do you mean the system gets restarted? Do you mean the computer crashes and reboots?

How many cores are you using?

Which model from the VMR?

Cheers,
Dave

User avatar
niloofar borzooei
Posts: 9
Joined: Sun May 19, 2024 6:21 pm

Re: System get restarted multiple times during running the simulation

Post by niloofar borzooei » Fri Jul 19, 2024 10:37 am

Hi David,

Thank you for getting back to me.

Yes, it seems that it gets crashed and rebooted. I am using 32 cores. The model I am using is from this paper Non invasive Estimation of Pressure Drop Across Aortic Coarctations Validation of 0D with the file name of 0228 H AO COA. Unfortunately, the vascular model website has been down for a couple of days now and I cannot access any other models.
The computer suddenly goes to black screen, sometimes showing the message 'your system runs into a problem and needs to be restart' and then it gets rebooted.

Thank you.

User avatar
David Parker
Posts: 1653
Joined: Tue Aug 23, 2005 2:43 pm

Re: System get restarted multiple times during running the simulation

Post by David Parker » Fri Jul 19, 2024 12:15 pm

Hello,

I'm not sure what might be going on here. Some of the VMR models are quite large so maybe the simulation is too much for your computer, although it seems to have a lot of memory, it does have 16 cores though they can be hyper-threading to get you 32 cores I guess. You are not running out of disk space are you?

I would try running using fewer cores, try 8 for a star and see if things work.

The VMR web site is down now, we are replacing the server.

Cheers,
Dave

User avatar
niloofar borzooei
Posts: 9
Joined: Sun May 19, 2024 6:21 pm

Re: System get restarted multiple times during running the simulation

Post by niloofar borzooei » Wed Jul 24, 2024 4:02 pm

Hello,

I tried running it with 8 and 16 but still have the same problem of restarting and rebooting. I would appreciate it if you could let me know about the system requirements for running the SimVascular softaware and simulating one of projects which exists in vascular repository.

Thank you so much.

Regards,
Niloofar

User avatar
David Parker
Posts: 1653
Joined: Tue Aug 23, 2005 2:43 pm

Re: System get restarted multiple times during running the simulation

Post by David Parker » Wed Jul 24, 2024 7:00 pm

Hi Niloofar,

The 0228_H_AO_COA model has 2,491,655 elements which is a bit large but it should run on a system with 64GB of memory, it will just take a long time to finish 7690 time steps.

Are you able to run other applications using mpi?

It is not very common for software to crash a computer. I am thinking that your computer has some sort of hardware or software problem. Try run diagnostics for it.

Cheers,
Dave

User avatar
niloofar borzooei
Posts: 9
Joined: Sun May 19, 2024 6:21 pm

Re: System get restarted multiple times during running the simulation

Post by niloofar borzooei » Sat Jul 27, 2024 8:30 am

Hi David,

Thank you for your explanation. since the supercomputer is not working yet, I decided to use another resources. However, I ran into some questions that I would appreciate you could help me with.
1- is it possible to download the SimVascular software along with its other requirements(like SVsolver) on my cluster at HPRC of my university? it is an Intel x86-64 Linux(CentOS 7) cluster with 940 compute nodes (45,376 total cores) and 5 login nodes. ( Since the SimVascular download file is for Ubuntu)

Furthermore, is it possible to run simulation on my laptop with the following properties?
- it is a Windows Intel(R) Core(TM) i7-10510U CPU @ 1.80GHz 2.30 GHz
- installed RAM 16.0 GB (15.8 GB usable)
- system type 64-bit operating system, x64-based processor

I would really appreciate it if you could tell me which one I can use or any other system requirements the resource should have.

Thank you so much.

Regards,
Niloofar

POST REPLY