Spark Training with Single ID Dataset

[samuel-gerstein]

I wanted to ask for clarification on the utility of Spark training with NeuralForecast. Does it only work when you have multiple ids? I don't have an issue running the reproducible example, but I run into a problem when I use my own dataset which has one ID. Has that functionality not been extended to this use case? If so, I think it deserves some clarification somewhere on the example.

[jmoralez]

Do you have a single serie with enough history to require distributed training?

[samuel-gerstein]

For my purposes it's more so for the performance boost with hyperparameter tuning. Though I guess that would require Nixtla to support Hyperopt with Spark first anyhow.

[jmoralez]

You should be able to use the auto models with ray on databricks

[Rajendra-mckesson]

Hi @samuel-gerstein ;

Do you mind sharing the working sample example for spark training with Neural Forecast. Can not find a example that work. Would appreciate it.

Thanks