Questions on duet and Federated learning when model is trained in real world

[AbinavRavi]

Hello, I would like to thank the courses team for a beautiful lesson with examples on Federated learning. I have finished with the lesson and had a few questions on how to execute this in the real world.

1. Is there an example of how to use cuda to train the models on GPU?

2. In cross-silo federated learning is there an expectation that the institutions will provide the hardware for training on their huge datasets?

3. Is there an expectation that the data owner also has some kind of data science literacy since they will probably will have to clean the data and provide the data and targets for the data scientists to work on? For example if we are training on x-rays and lung CT scans from different data owners who are huge hospitals generally the data is provided as a dump of image files and the data scientists are expected to clean and work on the data.

Thanks in advance

[znreza]

Hi @AbinavRavi, we are glad that you liked it! I have already answered this question in slack channel, but adding it here again so that others with similar questions can be helped as well.

1. Here's exactly what you are looking for
   https://github.com/OpenMined/courses/blob/foundations-of-private-computation/federated-learning/duet_mnist/MNIST_Syft_Data_Scientist.ipynb

# lets ask to see if our Data Owner has CUDA
has_cuda = False
has_cuda_ptr = remote_torch.cuda.is_available()
has_cuda = bool(has_cuda_ptr.get(
    request_block=True,
    reason="To run test and inference locally",
    timeout_secs=5,  # change to something slower
))
print(has_cuda)
# if we have CUDA lets send our model to the GPU
if has_cuda:
    model.cuda(device)
else:
    model.cpu()

2. Yes, in fact that's the usual expectation since we will be training them on their data in THEIR side. So at the current architecture, we need them to let us use their hardware for training, unless they themselves want to use some cloud-based or other external hardware.

3. That's a great question. If you remember the current duet architecture, right now it will be mostly done while both the data scientist and data owner are on call, or at least somehow coordinating. Therefore, both parties has to come to an agreement in these terms like what the data look like, dataset size, what transformations are needed to be done etc. Another thing is, the data owner can let the data scientist have a peek into a sample of the data by setting element_quota parameter in the request_handler for say data.get() request. As example, if element_quota = n , DS will get to see n items from the dataset.
   It will allow the DS to get an idea of how the data looks like in general, and design his data transformer and pre-processor accordingly.
   So there are different things can be done at the moment using duet. But overall it will need some kind of agreement between both parties.