Boundary scattering with phonon-electron coupling

[AnishDas-2025]

Dear experts,
I am the new user of this software, and I found Phoebe very amazing and efficient.
I have a few doubts.

1. I read in the manual that Phoebe supports "Boundary scattering" though "boundaryLength" key word. In the example it is shown that "boundaryLength = 10. mum". What is this mum unit? Is it in micrometer?

2. Boundary scattering is working well with only phonon-phonon scattering; however, it is not working when I am calculating lattice thermal conductivity employing phonon-electron scattering. It is showing the error "Wannier H0 file not found at mat_tb.dat"

Below is my input file:

appName = "phononTransport"

sumRuleFC2 = "simple"
phFC2FileName = "espresso.fc"

phFC3FileName = "FORCE_CONSTANTS_3RD"
windowType = "population"
qMesh = [15,15,5]
withIsotopeScattering = true
boundaryLength = 74 nm

electronH0Name = "mat_tb.dat"
elphFileName = "mat.phoebe.elph.hdf5"
kMesh = [15,15,5]

temperatures = [300.]
chemicalPotentials = [11.09]
numOccupiedStates = 14
smearingMethod = "adaptiveGaussian"

useSymmetries = false
scatteringMatrixInMemory=true
enforcePositiveSemiDefinite = true

solverBTE = ["iterative"]
maxIterationsBTE = 200
convergenceThresholdBTE = 1.0e-6

wsVecFileName = "mat_wsvec.dat"

Kindly, help me through this.
Thank you.

[jcoulter12]

Hi @AnishDas-2025,

Glad to hear you are enjoying the code, I'm happy to help resolve any issues.

1. Yes, the mum unit on boundary scattering is micrometers. You're right that I should add a guide on units somewhere in the input file description.
2. For clarification -- this input file works except when you set an input value for boundaryScattering -- as in, if you comment that line out it runs? I wonder if it's an input parsing issue -- could you try for example, boundaryLength = 0.074 mum just to see if this resolves the issue?

Let me know and we'll go from there.
Best,
Jenny Coulter

[AnishDas-2025]

Dear Dr. Coulter,
Thank you for your prompt response. I beg your pardon; it was my fault. I did not copy the "tight binding" file from the wannier90, that's why the error popped up. I have two more doubts.

1. Will the boundary scattering also work for electron transport? I mean to say in the evaluation of the Kappa_e?

2. In the manual of the "input variable descriptions" for "kMesh", it is mentioned that "In electron-phonon transport calculations, qMesh is set to be equal to the kMesh value and does not need to be specified by the user". However, in the example it is shown that el-ph coupling can be performed in the grid "q" not equal to "k" (k>>q). Does it we that we can perform the el-ph calculation on much denser k grid than the q grid? At low T, currently we can not afford to make q=k, beyond our resource actually.

Thank you
Anish

[jcoulter12]

Hi Anish,

No worries, I do that all the time myself. :) For your other questions:

1. Boundary scattering will also work for electron transport, yes. It's just another scattering rate added to the scattering matrix, so it will affect all the electronic transport properties as a result.

2. For the second question -- if one uses the full scattering matrix, the intermediate grid of q points summed over in the calculation of electron-phonon scattering rates is fixed use whichever q points connect the electronic states by conservation of momentum -- this is where this k = q requirement came from originally.

However, in the example it is shown that el-ph coupling can be performed in the grid "q" not equal to "k" (k>>q).

Which example is this, can you point me to it?

Also, if resources are scarce, I can make a few recommendations about efficient use. For electronic calculations, one thing that can help very much to reduce the expense is to check convergence with respect to the size of the Fermi window used to choose which states are relevant to transport. This is controlled by the variable windowpopulationlimit -- it is by default 1e-10, but often one can get good results for cutoffs such as 1e-5.

The other note I would add -- enforcePositiveSemiDefinite = true is typically not recommended (as noted in it's documentation) -- but also typically not needed. Was there a reason you wanted to turn this option on?

Best,
Jenny

[AnishDas-2025]

Dear Dr. Coulter,
Thank you for your prompt response. In this link "https://phoebe.readthedocs.io/en/develop/tutorials/thermalCondOfMetals.html" at "Step 2" and "Step 3" the kMesh = [75,75,75] and qMesh = [15,15,15] are used for Cu. Also, it is mentioned that "These calculations are done with 40x40x40 q-meshes, and 400x400x400 k-meshes" I have also attached the screen shots.

1. Yes, for BTE I can reduce the "windowpopulationlimit". But I also found that "Wigner" contribution is high for my system, specifically for Ke. For that reason, I used large Fermi window.

2. "enforcePositiveSemiDefinite = true" tag I used because initially I thought this will improve the quality of the scattering matrix. But I found no change as such in the energy vs lifetime plot. So, I can remove this tag.

3. Moreover, below I attached my slurm script. May you please check whether I am efficiently parallelizing or not.

For Kapp_e and ph-el coupling calculation

#!/bin/bash
#SBATCH -J q2phoebe
#SBATCH -p deimos
#SBATCH -e ERROR_%j.err
#SBATCH -N 1
#SBATCH -n 64
##SBATCH --time=72:00:00
#SBATCH --mail-user=anishdas39@gmail.com
#SBATCH --mail-type=ALL
#SBATCH --no-requeue
##SBATCH --exclude=cn057,cn002

module load intel/oneapi2023.2_noimpi
module load mpi/mpich/4.1.2-icc-oneapi2023.2-ch4
module load hdf5/1.14.3-icc-oneapi2023.2-mpich-4.1.2-ch4

export PATH="~/PHOEBE/phoebe_OLD/build":$PATH

export OMP_NUM_THREADS=64
OMP_PROC_BIND=spread
OMP_PLACES=thread
srun -n 1 ~/PHOEBE/phoebe_OLD/build/phoebe -in KAPPA_e.in -out KAPPA_e.out

For Ph-Ph calculation
Same as above but making only

export OMP_NUM_THREADS=64
####OMP_PROC_BIND=true
####OMP_PROC_BIND=spread
####OMP_PLACES=threads
srun -n 64 ~/PHOEBE/phoebe_OLD/build/phoebe -in kp.in -out kp.out

Thank you very much
Anish

[jcoulter12]

Hi Anish,

Ah yes, got it. Indeed, this is what I was thinking had caused confusion -- there are a number of different things meant by k and q mesh. In the case of phonon-electron scattering in phonon thermal conductivity, you can use a denser k-mesh than q-mesh. The comment in the input file documentation is referring to the electron-phonon scattering calculation used in the electrical transport calculation (as this was the first scattering rate written into the code, and the docs are just not updated for other use cases, my fault). Do you have trouble with the cost of that calculation as well?

If Wigner contributions are high, indeed, you're in a tough spot with respect to window size. Thanks for being careful to read the documentation on this. One thing to note is -- the Wigner contribution is only available as an addition to the the relaxation time approximation solution to the BTE (as this is currently the level of formalism available in the literature).

This solution is much cheaper than the iterative solution, anyway, because it doesn't require the whole scattering matrix and often is quite a good approximation. Typically, it's good to check that the iterative solution is not too different than the RTA using a narrow window -- then, if you find they are similar, you can run the RTA calculation with a wider window to get the final Wigner result.

Finally, for the job script, the first example seems fine, but the second one may have an issue with oversubscribing -- it seems you're setting both OMP_NUM_THREADS and MPI processes to 64, when you only have 64 CPUs.
You may also find that a mix of OMP and MPI processes can give best performance (something like 4 MPI processes and 16 OMP threads), but I assume if you're using pure OMP, it's because you're having some trouble with memory?

Best,
Jenny

[AnishDas-2025]

Dear Dr. Coulter,
Thank you for your clarification. The Phoebe manual is quite rigorous and almost every aspect is elaborately mentioned. So, sometimes it is readers' responsibility to go though, thus it is not your fault :).

1. Yes, indeed for electron BTE (Kappa_e and sigma calculation with el-ph coupling), I am using the same k and q mesh. For phonon BTE, Kappa_L calculation I will check for k > q grid, specifically at low T. At high T, I find k = q is OK.

2. I did not reproduce the example given in the tutorial, which I think should perform by myself once. :(

3. For final production calculation, actually I am not using "windowType = "population" for Electron BTE. Rather, I converged my calculation based on changing the Fermi window (using windowType = "muCenteredEnergy" and windowEnergyLimit = [-n, n]), say -0.2 to 0.2, -0.3 to 0.3 etc. Above room temperature, I repeated this process for three temperatures case, 300, 400 and 600. Say for 600 K, the converged window is 1 ev then the intermediate points from 400 to 600, I follow that window. Similarly, the Fermi window for 400 K is used for the calculation of 300 to 400 K and so on. For Wigner contribution, I am doing only RTA with scatteringMatrixInMemory = false and taking the full wannier band space. Is this the correct way?

4. Yes, for the ph-el coupling, when I do mpi=1 and omp=64, it runs fine. But for mpi= 64 and omp = 64 its oversubscribing. Amazing, for ph-ph calculation mpi= 64 and omp = 64 running faster some way, I dont know why. Thats why I am using this for ph-ph calculations.

5. Is there any way that I can use 2 nodes? I think in my case the OMP is restricting me to use two nodes. When I am using two nodes it is showing still oversubscribing. Is it because of OMP?

I am using something like this
#SBATCH -N 2
#SBATCH -n 64
export OMP_NUM_THREADS=128
OMP_PROC_BIND=spread
OMP_PLACES=thread
srun -n 1 ~/PHOEBE/phoebe_OLD/build/phoebe -in KAPPA_e.in -out KAPPA_e.out

Thank you
Anish.

[jcoulter12]

Hi Anish,

1. Ok, this is good to hear. I have worked on getting k =/= q for RTA calculations in the electron transport electron-phonon scattering, it's just not my highest todo list item. It helps to know what problems users are having sometimes so that I can prioritize.
2. It sounds like you're doing fine, you might not need to run the example. If you manage to set up your own calculation, then it's all good. :)
3. This works well and is often what I do for Wigner transport as well. Yes, that's a great way to run the Wigner calculation, good work.
4. I wonder if perhaps the node you're using is hyperthreaded. This might mean that while there are 64 physical CPUs, there's another 2 or more threads associated with each physical core. Hard for me to guess without knowing what kind of node it is, but you might consider playing with the MPI/OMP numbers or looking into what type of node it is.
5. Yes definitely you can use two or more nodes! This is why Phoebe is MPI as well as OMP parallel. This is a tricky skill to learn at first -- basically, OMP is only to be used within a single node. When you set OMP_NUM_THREADS, that's the number of threads on a given node. At most, you can set OMP_NUM_THREADS=number of cores on a node.

If each node has 64 cores, and you wanted to run a job with 16 MPI processes, with each MPI process using 8 OMP threads, you would set it up like this (where I am using SLURM environment variables to simplify things, and for me on my current SLURM system, --cpu-bind=cores is recommended). Using -c OMP_NUM_THREADS

#SBATCH -N 2      # number of nodes
#SBATCH -n 16    # total number of CPUs (use this for total number of MPI processes) 
                              # --> n/N = CPUs per node. Here, there would be 8 CPUs per node. 
#SBATCH -c 8      # threads (for OMP) per CPU --> n*c = total number of CPUs used by the job. Here, the job has 16 * 8 = 128 CPUs total. Slurm sets the env variable SLURM_CPUS_PER_TASK to this number. 

export OMP_NUM_THREADS=$SLURM_CPUS_PER_TASK
OMP_PROC_BIND=spread
OMP_PLACES=thread
srun  --cpu-bind=cores -c $SLURM_CPUS_PER_TASK ~/PHOEBE/phoebe_OLD/build/phoebe -in KAPPA_e.in -out KAPPA_e.out

Maybe this works better?
Jenny

[jcoulter12]

I also brought up a documentation page I had prepared a while ago but not yet added publicly on running the code efficiently. I think however, you've already figured most of this out:

https://phoebe.readthedocs.io/en/develop/faq.html

[AnishDas-2025]

Dear Dr. Coulter,
Thank you very much for your effort and time. Your clarifications helped me a lot. I will try the script you provided and also look up the nodes if it is hyperthreaded or not.

Thank you
Anish.

[jcoulter12]

Great! and feel free to write again if anything isn't clear. As I mentioned, it's good to have feedback from users who are engaged with the code features -- I don't have perspective on what is hard to learn or understand anymore, I've looked at the code for too long myself to see it. :)

Thank you,
Jenny