any plan to add SMA, RSI, ATR indices?

[yunhaizhu]

Do you have any plan to add SMI, RSI, ATR indices to the input state?

[samre12]

All these additional indicators are essentially functions of timeseries of Bitcoin prices and that too specifically closing prices. A convolutional network is able to compute these functions and thus providing them explicitly is redundant.
Rather, introducing highest and lowest prices in an interval and the volume of currency traded gives a better view of the changing trends.

[samre12]

However, I have an earlier commit in the master branch which contains calculation of these indicators and passes them as state representation to the Q network.

[johndpope]

I disagree. There is a contextual LSTM model which I think would fair better results if the context of price were included in training. https://arxiv.org/pdf/1602.06291.pdf

a successful agent needs to decouple from training data and level up in risk / reward plays. the agent should generalize on other data. If you take an Elliot wave trader on any stock - they can make/ predict the counts / predict a couple of scenarios and place the bet accordingly. these bot agents need to grasp these contexts. If RSI is bearish - then in that context - we're anticipating price swings downwards. If the training is over fit here - sure - then the bot - if given the same prices - it will get the bet right. but it would be better to anticipate if there's a high risk reward scenario to put the farm on bet (like George Soros did by borrowing billions of dollars to bet against the pound) becoming one of the richest men in the world. how did he do this? he took a bet that had very little risk / he only had to pay interest on loan.

[johndpope]

related https://github.com/llens/CryptoCurrencyTrader/issues/10

[johndpope]

88ef1b9

[johndpope]

we could branch off into competing forks? the idea being the best code wins to get merged into master.

this preprocessing code expands out features to a very wide selection of indicators.
This could be a curse on dimensionality - but we could iron it down

eg. cci is useful indicator for bearish / bullish channels.
cci = CCI(inputs)

if you fast forward to a finished training sample / given any time + price - you going to get a hodgepodge of signals on different time frames. 5 min / 1 hr / 2 hr etc.
we need to incorporate more granularity on risk / reward into strategy.
https://github.com/maxpaulus/alexis_maxime/blob/bc40344627011f191a5f137fb7988c59355b179c/DataProcessing/strategy_with_prediction.py

https://github.com/bfmill253/ForexDL/blob/3fcbd8cc92ff98363c478b5cdae5c0a2959ef3fc/ForexDL/Data.py

def expandData(startTime="0", instrument="EUR_USD", endTime="0"):
    close, high, low, volume, openMid, time = getData(startTime=startTime, instrument=instrument, endTime=endTime)
    inputs = {
    'open': openMid,
    'high': high,
    'low': low,
    'close': close,
    'volume': volume
    }
    #print (type(inputs.get('open')))
    #Overlap Studies
    sma = SMA(inputs)
    bbuper, bbmiddle, bblower = BBANDS(inputs)
    dema = DEMA(inputs)
    ema = EMA(inputs)
    htTrend = HT_TRENDLINE(inputs)
    kama = KAMA(inputs)
    ma = MA(inputs)
    mama, fama = MAMA(inputs)
    midpoint = MIDPOINT(inputs)
    midprice = MIDPRICE(inputs)
    sar = SAR(inputs)
    sarext = SAREXT(inputs)
    t3 = T3(inputs)
    trima = TRIMA(inputs)
    wma = WMA(inputs)
    
    #Momentum Indicators
    adx = ADX(inputs)
    adxr = ADXR(inputs)
    apo =APO(inputs)
    aroonDown, aroonUp = AROON(inputs)
    aroon = AROONOSC(inputs)
    bop = BOP(inputs)
    cci = CCI(inputs)
    cmo = CMO(inputs)
    dx = DX(inputs)
    macd, macds, macdh = MACD(inputs)
    macdext, macdexts, macdexth = MACDEXT(inputs)
    macdfix, macdfixs, macdfixh = MACDFIX(inputs)
    mfi = MFI(inputs)
    minusdi = MINUS_DI(inputs)
    minusdm = MINUS_DM(inputs)
    mom = MOM(inputs)
    plusdi = PLUS_DI(inputs)
    plusdm = PLUS_DM(inputs)
    ppo = PPO(inputs)
    roc = ROC(inputs)
    rocp = ROCP(inputs)
    rocr = ROCR(inputs)
    rocr100 = ROCR100(inputs)
    rsi = RSI(inputs)
    slowk, slowd = STOCH(inputs)
    fastk, fastd = STOCHF(inputs)
    rsifastk, rsifastd = STOCHRSI(inputs)
    trix = TRIX(inputs)
    ultosc = ULTOSC(inputs)
    willr = WILLR(inputs)
    
    
    #volume indicators
    ad = AD(inputs)
    adosc = ADOSC(inputs)
    obv = OBV(inputs)
    
    #cycle indicators
    ht_dcperiod = HT_DCPERIOD(inputs)
    ht_dcphase = HT_DCPHASE(inputs)
    ht_phasor_inphase, ht_phasor_quadrature = HT_PHASOR(inputs)
    ht_sine, ht_leadsine = HT_SINE(inputs)
    ht_trendmode = HT_TRENDMODE(inputs)
    
    #price transform
    avgprice = AVGPRICE(inputs)
    medprice = MEDPRICE(inputs)
    typrice = TYPPRICE(inputs)
    wclprice = WCLPRICE(inputs)
    
    #volatility ind
    atr = ATR(inputs)
    natr = NATR(inputs)
    trange = TRANGE(inputs)
    
    
    #pattern rec
    twocrows = CDL2CROWS(inputs)
    threeblackcrows = CDL3BLACKCROWS(inputs)
    threeinside = CDL3INSIDE(inputs)
    threelinestrike = CDL3LINESTRIKE(inputs)
    threeoutside = CDL3OUTSIDE(inputs)
    threestarsinsouth = CDL3STARSINSOUTH(inputs)
    threeWhiteSoldiers = CDL3WHITESOLDIERS(inputs)
    abandonBaby = CDLABANDONEDBABY(inputs)
    advanceBlock = CDLADVANCEBLOCK(inputs)
    beltHold = CDLBELTHOLD(inputs)
    breakAway = CDLBREAKAWAY(inputs)
    closingM = CDLCLOSINGMARUBOZU(inputs)
    consealBabySwall = CDLCONCEALBABYSWALL(inputs)
    counterAttack = CDLCOUNTERATTACK(inputs)
    darkCloudCover = CDLDARKCLOUDCOVER(inputs)
    doji = CDLDOJI(inputs)
    dojiStar = CDLDOJISTAR(inputs)
    dragonFlyDoji = CDLDRAGONFLYDOJI(inputs)
    engulfing = CDLENGULFING(inputs)
    eveningDojiStar = CDLEVENINGDOJISTAR(inputs)
    eveningStar = CDLEVENINGSTAR(inputs)
    gapSideSideWhite = CDLGAPSIDESIDEWHITE(inputs)
    graveStoneDoji = CDLGRAVESTONEDOJI(inputs)
    hammer = CDLHAMMER(inputs)
    hangingMan = CDLHANGINGMAN(inputs)
    harami = CDLHARAMI(inputs)
    haramiCross = CDLHARAMICROSS(inputs)
    highWave = CDLHIGHWAVE(inputs)
    hikkake = CDLHIKKAKE(inputs)
    hikkakeMod = CDLHIKKAKEMOD(inputs)
    homingPigeon = CDLHOMINGPIGEON(inputs)
    identical3Crows = CDLIDENTICAL3CROWS(inputs)
    inNeck = CDLINNECK(inputs)
    invertedHammer = CDLINVERTEDHAMMER(inputs)
    kicking = CDLKICKING(inputs)
    kickingByLength = CDLKICKINGBYLENGTH(inputs)
    ladderBottom = CDLLADDERBOTTOM(inputs)
    longLeggedDoji = CDLLONGLEGGEDDOJI(inputs)
    longLine = CDLLONGLINE(inputs)
    maruBozu = CDLMARUBOZU(inputs)
    matchingLow = CDLMATCHINGLOW(inputs)
    matHold = CDLMATHOLD(inputs)
    morningDojiStar = CDLMORNINGDOJISTAR(inputs)
    morningStar = CDLMORNINGSTAR(inputs)
    onNeck = CDLONNECK(inputs)
    piercing = CDLPIERCING(inputs)
    rickShawman = CDLRICKSHAWMAN(inputs)
    riseFall3Methods = CDLRISEFALL3METHODS(inputs)
    seperatingLines = CDLSEPARATINGLINES(inputs)
    shootingStar = CDLSHOOTINGSTAR(inputs)
    shortline = CDLSHORTLINE(inputs)
    spinningTop = CDLSPINNINGTOP(inputs)
    stalledPattern = CDLSTALLEDPATTERN(inputs)
    stickSandwhich = CDLSTICKSANDWICH(inputs)
    takuri = CDLTAKURI(inputs)
    tasukiGap = CDLTASUKIGAP(inputs)
    thrusting = CDLTHRUSTING(inputs)
    triStar = CDLTRISTAR(inputs)
    unique3River = CDLUNIQUE3RIVER(inputs)
    upsideGap2Crows = CDLUPSIDEGAP2CROWS(inputs)
    xSdieGap3Methods = CDLXSIDEGAP3METHODS(inputs)
    
    data = np.array([close, high, low, volume, openMid, time, sma, bbuper, bbmiddle, bblower, dema, ema, htTrend, kama, ma, mama, fama, midpoint, midprice, sar, sarext, t3, trima, wma, adx, adxr, ppo, apo, aroonDown, aroonUp, aroon, bop, cci, cmo, dx, macd, macds, macdh, macdext, macdexts, macdexth, macdfix, macdfixs, macdfixh, mfi, minusdi, minusdm, mom, plusdi, plusdm, roc, rocr, rocp, rocr100, rsi, slowk, slowd, fastk, fastd, rsifastk, rsifastd, trix, ultosc, willr, ad, adosc, obv, ht_dcperiod, ht_dcphase, ht_phasor_inphase, ht_phasor_quadrature, ht_sine, ht_leadsine, ht_trendmode, avgprice, medprice, typrice, wclprice, atr, natr, trange, twocrows, threeblackcrows, threeinside, threelinestrike, threeoutside, threestarsinsouth, threeWhiteSoldiers, abandonBaby, advanceBlock, beltHold, breakAway, closingM, consealBabySwall, counterAttack, darkCloudCover, doji, dojiStar, dragonFlyDoji, engulfing, eveningDojiStar, eveningStar, gapSideSideWhite, graveStoneDoji, hammer, hangingMan, harami, haramiCross, highWave, hikkake,  hikkakeMod, homingPigeon, identical3Crows, inNeck, invertedHammer, kicking, kickingByLength, ladderBottom, longLeggedDoji, longLine, maruBozu, matchingLow, matHold, morningDojiStar, morningStar, onNeck, piercing, rickShawman, riseFall3Methods, seperatingLines, shootingStar, shortline, spinningTop, stalledPattern, stickSandwhich, takuri, tasukiGap, thrusting, triStar, unique3River, upsideGap2Crows, xSdieGap3Methods])
    
    return data

N.B. all these functions are out of the box with TALib.

[samre12]

Competing forks seems to nice idea but currently I don't have much ideas remaining to improve the performance of the agent. In that scenario may be collaboration and sharing ideas seems as a better alternative.
Also, these are a hell lot of indicators in the preprocessing code, currently the input is closing, high, low and volume normalised by their values at previous time steps. Maybe 5-6 more features may steal the deal. What say, which of these should be selected?

[johndpope]

stepping back - rather than adding these as features into data set
why not have q learning action decide which action (indicator) to take/use?
eg.
BUY

• BUYONMOMENTUM
• BUYONVOLATILITY
  -BUYONVOLUME
  -BUYON...
  SELL
• SELLONMOMENTUM
• SELLONVOLATILITY
  -SELLONVOLUME
  -SELLON...
  HOLD
• HOLDONMOMENTUM
• HOLDONVOLATILITY
  -HOLDONVOLUME
  -HOLDON...

the action would need to observe the corresponding value of result & feedback into system.
it reminds me of reinforcement learning for helicopters.

I skimmed through this code by @dandxy89 - but not sure how one could cherry pick this code.
You need neural net to have a notion of a spread of observed values.
RSI is high - (ai should learn - don't enter trade here).
MACD is showing bearish divergence - (be careful - don't buy).
MACD is showing bullish divergence - go all in.

https://github.com/dandxy89/rf_helicopter/blob/master/Model/Q_Learning_Agent.py

alternative could be
create an array of indicators that ai will use and let them learn to trade based off indicator.
we could start with one - the SMA / simple moving average crossover strategy -- the ai should be able to leverage this for better trades.
https://www.youtube.com/watch?v=3N8Pl26N2x4

then step 2- somehow cross bread / concatenate other signals into a super strategy.
I want RSI ai + CCI ai + SMA ai.
the training data - doesn't necessarily need to be authentic - just the reward function needs to be optimized for most profits. If you go to month 1 of btc trading and applied SMA without ai - you will get a base line of how much should be won (assuming you successfully entered / exited every trade).

[samre12]

Seems like a nice idea, need to do a bit more research on this topic. I was also considering using Google Trends per minute statistics as an input to the model as they serve as a proxy to the market sentiments. But currently I am having issue acquiring per minute trends due to rate limits of the API.

[johndpope]

from @antkve,
neuralnetcrypto.py is a trading bot for crypto markets (bitcoin, ethereum, etc). It trains on historical price data, learning to notice patterns in the Relative Strength Index and Moving Average indicators. It does pretty well atm but i'll be extending it to analyze more indicators soon:

https://github.com/antkve/neuralnet/blob/master/neuralnetcrypto.py

[johndpope]

related - https://github.com/songyunfan/DeepRLTrading/blob/master/prediction-using-RL.pdf

[johndpope]

I think if we sunset the buy / sell / hold actions - and simply make the code predict prices - then the scope of the problem will be more fitting. Then we use the trained ai indicator to trade.