SearchingTA

Abstract

---

This project aims to search for profitable signals in the derivative market using a combination of trading rules. By selecting a combination of trading rules to generate signals, testing those signals, and then forming a complete trading strategy, including risk management and position sizing, we can optimize trading performance.

Introduction

---

The process of finding profitable trading strategies involves three steps:

1. Searching for a combination of trading rules and validating their signals.
2. Fine-tuning and forming a complete trading strategy.
3. Testing and validating on both in-sample and out-of-sample data.

Feature

• Generate Mock Data for unit testing
• Validate Test Case: Financial, Technical Signal and Backtesting
• Optimize hyperparameters
• Evaluate backtesting and optimization
• Paper trade

Installation

• Requirement: pip
• Create and source new virtual environment in the current working directory with command

python3 -m venv venv
source venv/bin/activate

• Install the dependencies by:

pip install -r requirements.txt

Process of this project:

1. Searching for signal

In this step the main goal is to validate all signals that generated by the model, which take account only Win Rate and mean profit and loss for best performance.

In this step the config for backtest is:

• Balance: unlimited
• Maximum position: Unlimited
• Position sizing: 1 contract per position
• Limit Order for all signals
• Order Price: Close Price of previous candle
• Cost:
  • Transaction fee apply for 1 side: 0.25 points
  • Slippage apply for 1 side: 0.47 (only apply for TP and SL)

2. Optimizing

In this step the main goal is to finetune the parameters found in previous step, and find and optimal parameters to form a complete strategy.

In this step the config for backtest is:

• Balance: (1200 x 25%) * (1 / posisiton size) * 1.5
• Maximum position: Parameter to find
• Position sizing: Parameter to find
• Limit Order for all signals
• Order Price: Close Price of previous candle
• Cost:
  • Transaction fee apply for 1 side: 0.25 points
  • Slippage apply for 1 side: 0.47 (only apply for TP and SL)

Related Work (Background)

---

Traders in the derivatives market often rely on trading signals derived from technical analysis. These signals are generated based on indicator behavior (e.g., selling when RSI > 70 and then falls below 70). This project aims to develop a trading strategy by selecting and combining multiple Trading rules based on Techincal Indicators.

Data

---

The data using in this project is tick data of VN30F1M from 2021-06-01 to 2024-11-01, and bid/ask data to check matching of orders.

To using the technical analysis, data is resample to interval range of 1 min to 60 min.

Data Collection

From class Downloader, the data will be automatically download the data from algotrade database, by specifying start-date, end-date and the ticker, the data will include datetime, tick price, bid price, ask price.

downloader = Downloader()

data_config = {
	'start_date': '2019-01-01',
	'end_date': '2024-11-01',
	'ticker': 'VN30F1M'
}
data = downloader.get_historical_data(**data_config)

ratio = 0.7
len_data = len(data)

insample = data.iloc[:int(ratio * len_data)]
outsample = data.iloc[int(ratio * len_data):]

search_data = insample.iloc[:int(0.3 * len(insample))]

Data:

>>>                         price  bid_price  ask_price  volume
datetime
2021-06-01 09:00:33.039040  1483.8     1483.8     1484.0  1091.0
2021-06-01 09:02:15.648543  1483.0     1483.1     1483.7  2427.0
2021-06-01 09:02:52.934966  1483.1     1483.0     1483.1  2885.0
2021-06-01 09:03:03.053928  1483.3     1483.3     1483.6  3052.0
2021-06-01 09:03:10.533157  1483.5     1483.5     1483.7  3059.0

Data Processing

The Data is processed resample to OHLC data and adding Technical Indicators through processor function in ./utils folder.

The Data Processing is automatically done by Backtesting class, by passing the tick data DataFrame to backtesting class.

Train Test Split

For 3 process to find an optimal strategies, the data is splitted to 3 part:

• In-Sample Data: 70% of the data
• Out-Sample Data: 30% of the data
• Search For Signal: 30% of In Sample data, 2021-06-01 to 2021-11-15

Data for Searching Signal

In Sample Data

Out Sample Data

Implementation

---

To run and get the result:

• Long model:

All the Result of Searching and Optimizing with Out Sample Testing will stored in ./result folder.

python3 -m long

• Short model

All the result stored in ./result_short****

python3 -m long

To get the metrics, run all in visualize.ipynb

To get different result for long and short change name variable to‘_short’ instead of ‘’

Searching for signal

The whole process is optimized by Optuna.

The Objective function here is:

$$ \begin{align*} \text{break even prob} &= \frac{\text{SL} + 2 \cdot \text{cost}}{\text{SL} + \text{TP}} \\ \text{expected pnl} &= \text{TP} \cdot \text{break even prob} \\ \text{mean pnl} &\text{: the mean profit and loss of the strategy} \\ \text{winrate} &\text{: the percentage of winning trades} \end{align*} $$

$$ \text{objective} = \left( \frac{\text{winrate}}{\text{break even prob}} - 1 \right) + \left( \frac{\text{mean pnl}}{\text{expected pnl}} - 1 \right) $$

If less than 50 trades return $-\inf$

The subtraction of 1 is used to adjust the values relative to a baseline (e.g., a win rate at the break-even probability and the mean PnL against the expected PnL).

• We define a trading rule within $./strategy/strategy.py$, with -1 for sell signal and 1 for long signal, 0 for do nothing.

def RSI(df: pd.DataFrame):
	RSI_curr, RSI_prev = df['rsi'].iloc[-1], df['rsi'].iloc[-2]

	if RSI_prev > 70 and RSI_curr <= 70:
		return -1
	elif RSI_prev < 30 and RSI_curr >= 30:
		return 1

	return 0

• Implement Searching rule for trading signal:

config = {
    'number_of_trials': num_trials, # number of trials
    'side': 'long', # side of the models
    'n_jobs': 2,
    'cost': 0.25, # transaction fee for 1 side
    'slippage': 0.47 # slippage for TP and SL
    'TP': (1, 10), # range for Take Profit, 1 to 10 points
    'SL': (1, 10) # range for Stop Loss, 1 to 10 points
}

search_dir = './result/searching'

search = Searching(data=search_data, dir=search_dir, **config)
search.run()

After run the data of the study will be save in search.db, and history trade equity and balance will be save in ./search_dir/trial folder.

Long Model

Best trial is 259:

Trials	Mean Returns (points)	Max_PnL (points)	Min_PnL (points)	Winrate (%)	Break Even Probaility (%)
259	1.466508	5.79	-4.37	34.9	29.4

• Optuna Trials:

• params:

  {
    "TP": 7.0,
    "SL": 1.5,
    "strategies": [
      "RSI",
      "PPO",
      "ADX",
      "CCI",
      "Momentum",
      "Keltner",
      "W_R",
      "Donchian",
      "FI",
      "Vortex"
    ],
    "interval": 45
  }

• PnL distribution:

Short Model

Best trial is 451:

Trials	Mean Returns (points)	Max_PnL (points)	Min_PnL (points)	Winrate (%)	Break Even Probaility (%)
451	2.594286	15.23	-4.37	46.428571	21.7

• Optuna Trials

• params:

  {
    "TP": 10.0,
    "SL": 1.5,
    "strategies": [
      "RSI",
      "PPO",
      "ADX",
      "CCI",
      "Momentum",
      "W_R",
      "Donchian",
      "FI",
      "Vortex"
    ],
    "interval": 47
  }

• PnL distribution:

Optimizing

This part is to find a complete strategy based on best set of signal rules.

The Objective here is maximize:

$$ \begin{align*} S &= \sqrt{252} \cdot \frac{R_p}{\sigma_p} \\ loss &= (\frac{S}{2} - 1) + (\frac{\bar{R_p}}{0.15} - 1) \end{align*} $$

Where:

• $S$ = Sharpe Ratio
• $R_p$ = Expected return of the portfolio
• $\sigma_p$ = Standard deviation of the portfolio returns

# Load best trial params from db
study_name=f"optimizing",
storage="sqlite:///searching.db",
search_study = optuna.load_study(study_name=study_name, storage=storage)
best_search_trial = search_study.best_trial.number

config = {
    'number_of_trials': num_trials, # number of trials
    'side': 'long', # side of the models
    'n_jobs': 2,
    'cost': 0.25, # transaction fee for 1 side
    'slippage': 0.47 # slippage for SL and TP
}

optimize_dir = './result/optimizing'
optimizer = Optimizer(trial=best_search_trial, path=search_dir,
											data=insample, dir=optimize_dir, **config)
optimize_study = optimizer.run(name=str(best_search_trial))

Long Model

Best Trial: 318

trial	sharpe	max_dd (%)	sortino	winrate	monthly_returns (%)	yearly_returns (%)
318	0.96	-43,2	1.7	15,7	4,6	46,9

• Params:

{
	'TP': 12.8,
	'SL': 3.5,
	'position_size': 0.5,
	'max_pos': 9,
	'min_signals': 2,
	'interval': 24,
	'side': 'long',
	'mode': 'one_way',
	'strategies': ['RSI', 'PPO', 'ROC', 'ADX', 'OBV', 'Donchian']
}

• Optuna Trials:

• Equity:

• Returns Distibutions:

• MDD

• Model vs Buy And Hold VN30F1M

Short Model

Best Trial: 337

trial	sharpe	max_dd (%)	sortino	winrate	monthly_returns (%)	yearly_returns (%)
337	1.06	-47,8	1.2	15,7	3.9	40.15

• Params:

{
	'TP': 11.4,
	'SL': 4.5,
	'position_size': 0.5,
	'max_pos': 9,
	'min_signals': 2,
	'interval': 47,
	'side': 'short',
	'mode': 'one_way',
	'strategies': ['RSI', 'ADX', 'CCI', 'Momentum', 'Donchian', 'Vortex']
}

• Optuna Trials:

• Equity

• Returns Distibutions

• MDD

• Model vs Buy And Hold VN30F1M

Out Sample Backtesting

To Validate the optimized strategy we run the Testing Class to run the strategy

# Load Study
study_name=f"optimizing_{best_search_trial}",
storage="sqlite:///searching.db",
optimize_study = optuna.load_study(study_name=study_name, storage=storage)
best_optimize_trial = optimize_study.best_trial.number

config = {
    'number_of_trials': num_trials, # number of trials
    'side': 'long', # side of the models
    'n_jobs': 2,
    'cost': 0.25, # transaction fee for 1 side
}

# Testing
test_dir = './result/testing'
tester = Tester(trial_num=best_optimize_trial, path=optimize_dir, data=outsample, dir=test_dir)
tester.run()

Long Model

trial	sharpe	max_dd (%)	sortino	winrate	monthly_returns (%)	yearly_returns (%)
318	0.94	-15.77	2.17	17	2.4	25.18

• Equity:

• Returns Distibutions:

• MDD

Short Model

trial	sharpe	max_dd (%)	sortino	winrate	monthly_returns (%)	yearly_returns (%)
318	0.25	-38.8	0.36	10	0.9	9.9

• Equity:

• Returns Distibutions

• MDD

• Model vs VN30F1M

Paper Trading

(To be updated)

With seed 42, the result for

• Long model:

	sharpe	max_dd (%)	sortino	winrate	monthly_returns (%)	yearly_returns (%)
In-Sample	0.96	-43,2	1.7	15,7	4,6	46,9
Out-Sample	0.94	-15.77	2.17	17	2.4	25.18

• Short model:

	sharpe	max_dd (%)	sortino	winrate	monthly_returns (%)	yearly_returns (%)
In-Sample	1.069	-47.7	1.23	13	3.98	40.15
Out-Sample	0.25	-38.8	0.36	10	0.98	9.9

Conclusion

The strategy found has a high win rate and generates a high return over time. However, the main issue is that the maximum drawdown is extremely high in in-sample backtesting. Despite the high drawdown, the process of searching and optimizing has identified a profitable strategy with strong consistency between in-sample and out-of-sample testing.

Reference