High Wall Time Consumption in DiagoElpa elpa_solve during HSE Calculation #6662
Description
Details
I recently performed a DFT simulation using the HSE hybrid functional (LCAO basis) which resulted in a significant wall time (7.95 hours).
The performance log indicates that the diagonalisation routine, DiagoElpa elpa_solve, is consuming a very large proportion of the total runtime (96.18% over 600 calls). I would appreciate if I can get some clarification on whether this high consumption is expected for the scale of this HSE calculation, or if it might suggest an area for potential performance tuning within this specific ABACUS workflow.
This is the overall time statistics:
--------------------------------------------------------------------------------------------
CLASS_NAME NAME TIME/s CALLS AVG/s PER/%
--------------------------------------------------------------------------------------------
total 28629.62 13 2202.28 100.00
Driver reading 0.04 1 0.04 0.00
Input_Conv Convert 0.00 1 0.00 0.00
Driver driver_line 28629.58 1 28629.58 100.00
UnitCell check_tau 0.00 1 0.00 0.00
ESolver_KS_LCAO before_all_runners 15.64 1 15.64 0.05
PW_Basis_Sup setuptransform 0.23 1 0.23 0.00
PW_Basis_Sup distributeg 0.02 1 0.02 0.00
mymath heapsort 0.25 98673 0.00 0.00
Charge_Mixing init_mixing 0.00 1 0.00 0.00
Symmetry analy_sys 0.04 1 0.04 0.00
PW_Basis_K setuptransform 0.07 1 0.07 0.00
PW_Basis_K distributeg 0.01 1 0.01 0.00
PW_Basis setup_struc_factor 0.18 1 0.18 0.00
NOrbital_Lm extra_uniform 1.33 2492 0.00 0.00
Mathzone_Add1 SplineD2 0.06 2492 0.00 0.00
Mathzone_Add1 Cubic_Spline_Interpolation 0.31 2492 0.00 0.00
Exx_LRI init 11.49 1 11.49 0.04
Matrix_Orbs21 init 3.69 2 1.85 0.01
ORB_gaunt_table init_Gaunt_CH 0.14 3 0.05 0.00
ORB_gaunt_table Calc_Gaunt_CH 0.07 82305 0.00 0.00
ORB_gaunt_table init_Gaunt 1.92 3 0.64 0.01
ORB_gaunt_table Get_Gaunt_SH 2.32 3635313 0.00 0.01
Matrix_Orbs21 init_radial 0.00 2 0.00 0.00
Matrix_Orbs21 init_radial_table 5.22 2 2.61 0.02
Center2_Orb cal_ST_Phi12_R 3.95 4447 0.00 0.01
LRI_CV set_orbitals 6.02 1 6.02 0.02
Matrix_Orbs11 init 0.22 1 0.22 0.00
Matrix_Orbs11 init_radial 0.00 1 0.00 0.00
Matrix_Orbs11 init_radial_table 1.31 1 1.31 0.00
Symmetry_rotation cal_Ms 0.43 1 0.43 0.00
ppcell_vl init_vloc 0.29 1 0.29 0.00
Ions opt_ions 28613.07 1 28613.07 99.94
ESolver_KS_LCAO runner 28613.07 1 28613.07 99.94
ESolver_KS_LCAO before_scf 42.00 1 42.00 0.15
Vdwd3 cal_energy 0.36 1 0.36 0.00
atom_arrange search 0.00 1 0.00 0.00
atom_arrange grid_d.init 0.00 1 0.00 0.00
Grid Construct_Adjacent_expand 0.00 1 0.00 0.00
Grid Construct_Adjacent_expand_periodic 0.00 108 0.00 0.00
Grid_Technique init 0.23 1 0.23 0.00
Grid_BigCell grid_expansion_index 0.01 2 0.00 0.00
Grid_Driver Find_atom 0.01 650 0.00 0.00
Record_adj for_2d 0.06 1 0.06 0.00
LCAO_domain grid_prepare 0.00 1 0.00 0.00
Veff initialize_HR 0.03 1 0.03 0.00
OverlapNew initialize_SR 0.03 1 0.03 0.00
EkineticNew initialize_HR 0.03 1 0.03 0.00
NonlocalNew initialize_HR 0.06 1 0.06 0.00
Exx_LRI cal_exx_ions 39.37 1 39.37 0.14
LRI_CV cal_datas 5.80 2 2.90 0.02
Charge set_rho_core 0.21 1 0.21 0.00
PW_Basis_Sup recip2real 13.82 422 0.03 0.05
PW_Basis_Sup gathers_scatterp 1.49 422 0.00 0.01
Charge atomic_rho 0.80 2 0.40 0.00
Potential init_pot 0.42 1 0.42 0.00
Potential update_from_charge 178.44 61 2.93 0.62
Potential cal_fixed_v 0.04 1 0.04 0.00
PotLocal cal_fixed_v 0.04 1 0.04 0.00
Potential cal_v_eff 178.27 61 2.92 0.62
H_Hartree_pw v_hartree 4.89 61 0.08 0.02
PW_Basis_Sup real2recip 14.32 443 0.03 0.05
PW_Basis_Sup gatherp_scatters 1.22 443 0.00 0.00
PotXC cal_v_eff 172.97 61 2.84 0.60
XC_Functional v_xc 10344.25 39 265.24 36.13
Potential interpolate_vrs 0.13 61 0.00 0.00
Symmetry rhog_symmetry 2.82 61 0.05 0.01
Symmetry group fft grids 2.41 61 0.04 0.01
H_Ewald_pw compute_ewald 0.04 1 0.04 0.00
HSolverLCAO solve 27658.95 60 460.98 96.61
HamiltLCAO updateHk 80.27 600 0.13 0.28
OperatorLCAO init 57.33 2400 0.02 0.20
Veff contributeHR 50.13 60 0.84 0.18
Gint_interface cal_gint 80.07 120 0.67 0.28
Gint_interface cal_gint_vlocal 41.71 60 0.70 0.15
Gint_Tools cal_psir_ylm 13.25 46975 0.00 0.05
Gint_k transfer_pvpR 1.12 60 0.02 0.00
OverlapNew calculate_SR 0.15 1 0.15 0.00
OverlapNew contributeHk 3.48 600 0.01 0.01
EkineticNew contributeHR 0.35 60 0.01 0.00
EkineticNew calculate_HR 0.18 1 0.18 0.00
NonlocalNew contributeHR 3.25 60 0.05 0.01
NonlocalNew calculate_HR 0.84 1 0.84 0.00
OperatorLCAO contributeHk 1.84 600 0.00 0.01
HSolverLCAO hamiltSolvePsiK 27536.45 600 45.89 96.18
DiagoElpa elpa_solve 27534.60 600 45.89 96.18
elecstate cal_dm 6.68 60 0.11 0.02
psiMulPsiMpi pdgemm 5.34 600 0.01 0.02
DensityMatrix cal_DMR 2.07 60 0.03 0.01
ElecStateLCAO psiToRho 33.31 60 0.56 0.12
Gint transfer_DMR 1.52 60 0.03 0.01
Gint_interface cal_gint_rho 31.06 60 0.52 0.11
Charge_Mixing get_drho 0.04 60 0.00 0.00
Charge mix_rho 4.89 52 0.09 0.02
Charge Broyden_mixing 0.29 52 0.01 0.00
ModuleIO write_rhog 5.95 8 0.74 0.02
Symmetry_rotation restore_dm 1.56 7 0.22 0.01
RI_2D_Comm split_m2D_ktoR 76.48 7 10.93 0.27
Exx_LRI cal_exx_elec 632.30 7 90.33 2.21
Symmetry_rotation restore_HR 5.86 7 0.84 0.02
RI_2D_Comm add_HexxR 17.47 44 0.40 0.06
XC_Functional_Libxc v_xc_libxc 166.42 44 3.78 0.58
ESolver_KS_LCAO after_scf 0.39 1 0.39 0.00
ESolver_KS_LCAO out_deepks_labels 0.00 1 0.00 0.00
LCAO_Deepks_Interface out_deepks_labels 0.00 1 0.00 0.00
ESolver_KS_LCAO after_all_runners 0.02 1 0.02 0.00
ModuleIO write_istate_info 0.02 1 0.02 0.00
--------------------------------------------------------------------------------------------
This is my input parameters, and my structure has 108 atoms:
INPUT_PARAMETERS
calculation scf
symmetry 1
ecutwfc 100
scf_thr 1e-07
scf_nmax 100
smearing_method gauss
smearing_sigma 0.015
mixing_type broyden
mixing_beta 0.8
basis_type lcao
ks_solver genelpa
kspacing 0.14
dft_functional hse
vdw_method d3_0
Thanks for your help :)
Have you read FAQ on the online manual http://abacus.deepmodeling.com/en/latest/community/faq.html
- Yes, I have read the FAQ part on online manual.
Task list for Issue attackers (only for developers)
- Understand the problem or question described by the user.
- Check if the issue is a known problem or has been addressed in the documentation.
- Test the issue or problem on a similar system or environment, if possible.
- Identify the root cause or provide clarification on the user's question.
- Provide a step-by-step guide, including any necessary resources, to resolve the issue or answer the question.
- If the issue is related to documentation, update the documentation to prevent future confusion (optional).
- If the issue is related to code, consider implementing a fix or improvement (optional).
- Review and incorporate any relevant feedback from users or developers.
- Ensure the user's issue is resolved or their question is answered and close the ticket.