Dealing with 2FA

[Andrew-S-Rosen]

I know this project is still in its early stages, but one troublesome point to perhaps think about early is that some clusters (e.g. Savio at UC Berkeley) require a 2FA key to be entered with each login, such that an SSH key alone isn't sufficient. I find this super annoying, but it is what it is. It's not immediately clear the best way to get around that. One could imagine using a Python wrapper around oathtool (e.g. like here) to generate the OTP, but there are some security questions worth considering too.

[gpetretto]

Hi @arosen93, thanks for raising this point. This is indeed a strong limitation for cluster with 2FA. Unfortunately, I don't think there is a way that would allow to automatically login, respecting the security policies required by the cluster administrators.
I have seen similar discussion for AiiDA and, as far as I know, no standard and safe solution has been found for this problem.
A potential solution that I have seen mentioned to help the use of automation tools is firecrest, but I don't know if cluster administrators will decide to adopt it.

[Andrew-S-Rosen]

Thanks for your input! I was definitely interested to know what your take on this was as well. I ended up coming to basically the same conclusion --- there's probably not a way to handle it in a secure manner. I guess the best solution in such a case is to simply launch jobs from within the network where 2FA isn't needed.

[janosh]

IIUC, NERSC's recommended solution to this problem is https://github.com/NERSC/sfapi_client which allows developers to exchange client credentials for access tokens and then make requests to authenticated cluster endpoints via the Superfacility API. Of course, that means extra work for you guys.

Not sure if Savio has sth similar.

[Andrew-S-Rosen]

Right, forgot about that!

[gpetretto]

Hi Janosh,
thanks for letting us know about this solution for NERSC. I think we can consider supporting this.
It is a bit inconvenient that apparently different clusters are developing their own custom API, which will make it harder to provide a widespread support.

[janosh]

It is a bit inconvenient that apparently different clusters are developing their own custom API, which will make it harder to provide a widespread support.

My thoughts exactly. It would need some kind of open standard for HPC APIs or a lessening of security restrictions. Latter seems very unlikely. Maybe the former exists...