Walk forward testing & parameter optimisation?

[gcaplan]

Just spotted this exciting and much needed project!

BUT - on a quick scan through the docs and code I don't see any facility for walk forward testing and parameter optimisation.

Am I missing something?

Or if it's not implemented yet, is this something that's on the roadmap for the next 12 months or so?

Despite the many attractions of the platform I can't see how we could commit where these features aren't on offer - and I suspect that we're not alone...

[limx0]

Hi @gcaplan

Could you explain what exactly you mean by / your expectation is for walk forward testing ?

Regarding parameter optimisation, this is something explicitly left out of nautilus trader due to the wide range and high quality of other projects (that can be quite trivially integrated).

[gcaplan]

Hi @limx0 - thanks for the quick reply!

Walk forward testing and optimisation is pretty much an industry standard in enterprise class backtesting, and some of the retail platforms offer it too.

There's a very brief outline here:

https://zorro-project.com/manual/en/numwfocycles.htm

As for parameter optimisation, could you kindly share a quick outline of how that would be integrated? I'm not very familiar with the Python trading ecosystem...

[limx0]

Hi @gcaplan,

So this isn't something that currently exists in NautilusTrader, and probably won't feature in the core codebase, but may make it into a higher level package that @cjdsellers is planning on working on in the near ish future.

At the moment though, we're pretty focused on core backtesting/live features, and a walk forward analysis is also something quite trivially implemented using the existing code, so not likely to be a huge value-add or high priority for NautilusTrader (over many other features we would like to implement).

As an example, Nautilus allows creating a BacktestEngine that you can fill with data and run many times over (with optional start/stop times) that would be suitable for an optimisation or walk-forward analysis. There are many examples in the backtest folder that show setting up of these backtest engines.

Python has a very rich ecosystem of ML/optimisation libraries that could be easily integrated with NautilusTrader, a couple off the top of my head are:

• https://scikit-learn.org/stable/
• http://hyperopt.github.io/hyperopt/
• https://optuna.org/

For integrating, these libraries typically take a single "objective" function with arguments. You could create a function with parameters for the engine and your strategy that creates a BacktestEngine, runs it and and returns the value you are interested in (likely net profit).

As these functions are typically tied to the users strategy and chosen market and easy to write (a single function), its unlikely we would add too much more to NautilusTrader.

I hope this (mostly) answers your question and provides some colour on why these don't exist currently. If anything is unclear or if you have any other questions, please don't hesitate to ask again.

[gcaplan]

Thanks - very helpful!