Skip to content

When is the serializer used? Why for training only but not eval #790

@silentstar425

Description

@silentstar425

I noticed the data processing stage is extremely slow in eval stage. It seems that its caused by not serializing the params.

In training stage msg_type is 'model_para', while in eval stage it's 'evaluate'. But model params are transmitted in both stages.

federatedscope/core/message.py line 106 - 120:

def transform_to_list(self, x):

      if isinstance(x, list) or isinstance(x, tuple):
  
          return [self.transform_to_list(each_x) for each_x in x]
  
      elif isinstance(x, dict):
  
          for key in x.keys():
  
              x[key] = self.transform_to_list(x[key])
  
          return x
  
      else:
  
          if hasattr(x, 'tolist'):
  
              if self.msg_type == 'model_para':
  
                  return self.param_serializer(x)
  
              else:
  
                  return x.tolist()
  
          else:
  
              return x

We can see params are serialized in training stage, but not in other messages. I'm curious why it's designed like that, I’d really appreciate it if you could help me with this.

Metadata

Metadata

Labels

No labels
No labels

Type

No type

Projects

No projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions