Bad performance of CALC_CHAMP

vishwas

Hello,
I want to share some observations regarding CALC_CHAMP operator. It takes enormous amount of RAM and hangs up the system
For context, I performed a STAT_NON_LINE simulation with a few non linear inputs like VMIS_ISOT_TRAC, contact with friction and enforced motion. Overall DOFs ~ 115000 with 46000 elements(20k solid elements). I know VMIS_ISOT_TRAC creates some more internal variables.
I set these parameters in export file:

P memory_limit 20000.0 P mode interactif P mpi_nbcpu 5 P mpi_nbnoeud 1 P ncpus 1
Firstly some question regarding memory required.
I want to know what memory_limit parameter signify. Is it the memory allocated to all cores (specified by mpi_nbcpu (here 5)) or is it memory allocated per core.?
Also, how can one gauge memory_limit parameter? In this run at final time step VMPeak was ~ 6.5Gb. So that means setting memory_limit to around 7 Gb is correct if memory_limit is allocated RAM per core? But how can one know what VmPeak will be at the end of simulation.
The simulation took 17h to complete and had around 150 time steps. I did not use ARCHIVAGE keyword so it stored all time steps. When CALC_CHAMP is called it just hung up.
The code I used is below:

result = CALC_CHAMP( reuse=result, CONTRAINTE=( 'SIGM_NOEU'), CRITERES=('SIEQ_NOEU', ), MODELE=model, PARALLELISME_TEMPS='OUI', RESULTAT=result, )
So I only requested cauchy nodal stresses and SIEQ_NOEU. Yet the system hung up. It occupied all RAM on system(32 Gb) and swap(8 GB) . It was unresponsive for like 10 hours until I forcefully restarted it. I lost all results.
For a comparison similar simulation was done in Prepomax(using Calculix) and it took nearly 2 hrs( with 100 time steps). Its result file is continuously updated even if it's size is like 5 Gb(due to additional requested quantitites like Contact CDIS, CSTR(5 to 6 variables) in addition to cauchy stress, strain, VMIS stress, VMIS strain and some history output too for certain node groups). Results are immediately available in Calculix after the run is finished.
So whats causing code_aster to take such long times and memory requirements for CALC_CHAMP to work. Is there a way to bypass this problem?

Thanks and Regards
Anirudh

AsterO'dactyle

Hello,

Calculating node quantities from ELGA fields requires extrapolation (to ELNO), followed by averaging (NOEU).
ELNO fields can be calculated efficiently using MPI parallelisation (as they remain a local quantity at the elements). However, averaging at the nodes is a non-local and non-parallelised operation, so it is computationally expensive (particularly in terms of memory).
It turns out, then, that looking at ELGA fields at the nodes is a ‘dangerous’ approximation.

vishwas

Hello,
Thank you for responding.
We know SIEQ_NOEU VMIS is a common and standard output of all FEA software and every one wants to see it at first glance. Indeed averaging is bad that's why abaqus put 75% tag near color bar.
But could calculation speed of _NOEU quantities be increased by MPI without RAM overload?

Regards
Anirudh