Streaming Historical Data... "End time no longer supported"

[jackroc97]

I am trying to use DXLinkStreamer to download historical data. My code looks something like this:

session = Session(...)
# Download last week's 5m candles on current /ES future
start_time = datetime(2025, 1, 6) 
end_time = datetime(2025, 1, 10)
future = Future.get_future(session,  "/ESH5")
async with DXLinkStreamer(session) as streamer:
    await streamer.subscribe_candle([future.streamer_symbol], '5m', start_time, end_time=end_time, extended_trading_hours=True)
    async for candle in streamer.listen(Candle):
        print(candle.model_dump())

When I run this, I am met with tastytrade.utils.TastytradeError: End time no longer supported. The tastytrade api supports historical candles as documented here. Is historical data not yet implemented in this wrapper?

[Graeme22]

It used to work, but I believe it doesn't anymore. You can try manually changing the SDK to allow the end_time parameter but in my testing it wasn't working as of a few months ago.

[militantwalrus]

From my own interaction with TT api support - "known issue" with dxfeed.

I "handle" it by inspecting the candle .time field and comparing it to sentinel values at both "ends".

[Graeme22]

Please note you can get historical data, however! You just can't set an ending date for the range.

[jackroc97]

Thank you both for your replies. I think part of my confusion was also that data comes in "backwards" (most recent to least recent). For posterity, I have posted the solution I implemented below. @militantwalrus I assume your solution looks similar?

async def download_hisorical_data(session: Session, 
                                  symbol: str, 
                                  internval: str,
                                  start: datetime, 
                                  end: datetime = datetime.now()) -> list[dict]:
    data = []
    async with DXLinkStreamer(session) as streamer:
        await streamer.subscribe_candle([symbol], 
                                        internval, 
                                        start_time=start,
                                        extended_trading_hours=True)
        async for candle in streamer.listen(Candle):
            
            # Candles will com in *backwards* (most recent to least recent)
            # Ignore data until it is past the specified end_time
            if candle.time >= end.timestamp() * 1000:
                continue
            elif candle.time == start.timestamp() * 1000:
                break
            else:
                data.append(candle.model_dump())
                
    return data  

# example usage
data = asyncio.run(download_hisorical_data(session, future.streamer_symbol, '5m', start, end))

[RoundToo]

Now that 'end time' is no longer supported, it seems like we can only receive the last x count of candles (usually 8000), regardless of how many additional candles DXFeed stores

[Graeme22]

Now that 'end time' is no longer supported, it seems like we can only receive the last x count of candles (usually 8000), regardless of how many additional candles DXFeed stores

Probably the only way to get this resolved is to reach out to Tasty and get them to fix it.

[militantwalrus]

Thank you both for your replies. I think part of my confusion was also that data comes in "backwards" (most recent to least recent). For posterity, I have posted the solution I implemented below. @militantwalrus I assume your solution looks similar?

            # Candles will com in *backwards* (most recent to least recent)
            # Ignore data until it is past the specified end_time
            if candle.time >= end.timestamp() * 1000:
                continue
            elif candle.time == start.timestamp() * 1000:
                break
            else:
                data.append(candle.model_dump())
                
    return data  

Similar, except that I do if candle.time <= start.timestamp() * 1000 (actually, I call .date() on both, because I usually care about daily candles but....yeah).

Also beware candles with NaN for a lot of the e.g. open, close, vwap, etc.. those seem to crop up for some reason...

[warrenjch]

This is not an issue on tastytrade's side, you can check out https://kb.dxfeed.com/en/data-services/aggregated-services/candle-types.html for a list of their offered historical data types. Dxfeed seems to have split the candle data into two types on their end, and the type that supports start and stop parameters is now called candlewebservice and tastytrade does not have that as part of their data package.