Adam is slower than sgd+momentum

[lucasliunju]

Hi, I try to use sgd+momentum and adam. Finally, I find Adam is slower than Momentum, which increases 20% training time. I think Adam just increase the memory and I'm not sure whether that is normal for this time cost.

Thank you!