When JAX solves the eigenvalue problem, if there are multiple eigenvalues, can it still obtain accurate gradients?

[DoTulip]

Hello, JAX Team,
I have a question about the automatic differentiation of eigenvalue problems. When JAX solves the eigenvalue problem if there are multiple eigenvalues, can it still obtain accurate gradients?

[mattjj]

Check out the discussion on #669 and @shoyer's comment here on #8732 (also #2311). See also this comment in the code.

I think the short answer is no. Moreover I think that with repeated eigenvalues the gradient for eigendecomposition is not well defined mathematically (though for autodiff of an eig/eigh routine one could define the autodiff gradient with reference to whatever gauge-symmetry-breaking choices the implementation happened to make).

@shoyer @levskaya please correct any mistakes in the above!

[DoTulip]

Thank you so much for your reply, these discussions have helped me a lot!