How to enable sequence-parallel?

[kback1]

I tried modifying source to pass argument sequence-parallel=True in Transformer Engine here but that resulted in some issues (see thread). How can I set the sequence-parallel argument through NeMo? I am running with:

    model.tensor_model_parallel_size=4 \
    model.pipeline_model_parallel_size=2 \

[ksivaman]

Adding model.sequence_parallel=True to the arg list should do the trick.