Tenseal encrypted state-dict size huge

[umermjd11]

I have a pytorch model

Sequential(
(0): Linear(in_features=784, out_features=200, bias=True)
(1): ReLU()
(2): Dropout(p=0.2, inplace=False)
(3): Linear(in_features=200, out_features=200, bias=True)
(4): ReLU()
(5): Dropout(p=0.2, inplace=False)
(6): Linear(in_features=200, out_features=10, bias=True)
(7): LogSoftmax(dim=1)
)
I am trying to do federated learning
for this I am homomorphically encrypting the state_dict using tenseal.
The problem is that a encrypted tensor of 784 floats takes 201.76 MB storage

if we consider like this the total size of encrypted state-dict will exceed 120 GB.

How can we do federated learning using tenseal considering the storage issue.

[CAPTAIN320]

if we consider like this the total size of encrypted state-dict will exceed 120 GB.

Hello!
How did you figure out the total size of encrypted state-dict?

[umermjd11]

Well, I was not using the proper ckks code. After using a proper method, it was 200 MB for the whole model.

[umermjd11]

just multiply 200 MB by number of vectors in my model.

[CAPTAIN320]

just multiply 200 MB by number of vectors in my model.

understood, thank you!

[CAPTAIN320]

Well, I was not using the proper ckks code. After using a proper method, it was 200 MB for the whole model.

I am using the latest tenseal version, but I am still running into memory issues.
A whole (4D) layer of state_dict cannot be encrypted because it is too large, and even the highest possible polynomial_modulus_degree is not sufficient.
I tried breaking down the layer into smallert chunks and then encrypt and save in an array.
This results in the memory full issue!

Any suggestions please?

[macknight]

hi,
can you share a simple example of using tenseal to encrypt the model's state_dict()?

BR