Initialisation on TPU v3-256 takes 190 seconds. Is it normal?

[ayaka14732]

I ran the following script on all TPU hosts:

import jax
jax.distributed.initialize()

It repeats the following warning for 32 times:

/home/ayaka/nfs_share/venv/lib/python3.11/site-packages/jax/_src/xla_bridge.py:187: UserWarning: TPU backend initialization is taking more than 60.0 seconds. Did you run your code on all TPU hosts? See https://jax.readthedocs.io/en/latest/multi_process.html for more information.
  warnings.warn(

And the script finally finished after 190 seconds.

Is this a normal behaviour?

Environment:

TPU v3-256
tpu-vm-tf-2.13.0-pod