.csv error messsage - it is everywhere!

[rorymac]

Hello All
I am trying to upgrade to the latest pysyst.

When running reports, they crash and I get this as part of the error message :

2023-03-16 17:48:15 {'type': 'slippage_report'} [Warning] No datapaths provided for .csv, will use defaults  (may break in production, should be fine in sim)
Traceback (most recent call last):
  File "/home/rorym/pysystemtrade/syscore/cache.py", line 30, in get
    value_from_store = self._get_from_store(key)
  File "/home/rorym/pysystemtrade/syscore/cache.py", line 50, in _get_from_store
    raise missingData("Missing cache element %s" % key)
syscore.exceptions.missingData: Missing cache element _combined_df_costs/()/{}

[Warning] No datapaths provided for .csv, will use defaults (may break in production, should be fine in sim)
When running Interactive order stack - and viewing at positions I get :
2023-03-16 18:05:27 {'type': ''} [Warning] No datapaths provided for .csv, will use defaults (may break in production, should be fine in sim)

Also running interactive_update_roll_status

Your choice? <RETURN for Manually input instrument codes and manually decide when to roll> 
2023-03-16 19:05:21 {'type': 'Interactive_Update-Roll-Status'} [Warning] No datapaths provided for .csv, will use defaults  (may break in production, should be fine in sim)
Traceback (most recent call last):
  File "/home/rorym/pysystemtrade/sysproduction/linux/scripts/run.py", line 67, in <module>
    func(*args, **kwargs)
  File "/home/rorym/pysystemtrade/sysproduction/interactive_update_roll_status.py", line 56, in interactive_update_roll_status
    function_to_call(data)
  File "/home/rorym/pysystemtrade/sysproduction/interactive_update_roll_status.py", line 121, in update_roll_status_manual_cycle
    instrument_code = get_valid_instrument_code_from_user(
  File "/home/rorym/pysystemtrade/sysproduction/data/prices.py", line 394, in get_valid_instrument_code_from_user
    instrument_code_list = get_list_of_instruments(data, source=source)
  File "/home/rorym/pysystemtrade/sysproduction/data/prices.py", line 427, in get_list_of_instruments
    instrument_list = price_data.get_list_of_instruments_in_multiple_prices()
  File "/home/rorym/pysystemtrade/sysproduction/data/prices.py", line 100, in get_list_of_instruments_in_multiple_prices
    list_of_instruments = self.remove_stale_instruments(list_of_instruments)
  File "/home/rorym/pysystemtrade/sysproduction/data/prices.py", line 224, in remove_stale_instruments
    list_of_instruments.remove(instrument_code)
ValueError: list.remove(x): x not in list

So I have followed the discussion in :
Instrument and roll config now (mostly) stored in .csv file, not database #1054
and followed the advice therein , in particular running sysinit/futures/repocsv_spread_costs.py

So the question is where do I provide the datapaths for .csv ?
And what data paths?

[meldinman]

Had the same error message, turns out it was little bug with the default.yaml file, Rob added a stale instrument row that was returning a string when getting stale instrument

stale_instruments: ['an_instrument_i_no_longer_trade_or_sample']

commenting that out or adding actual stale instruments should fix that..

[robcarver17]

Sorry my bad, fixed the code so it should just work even with the original config .yaml. I didn't see the error because I was testing it with an actual instrument.

[rorymac]

Thank you !

[73ze]

With 5544b5b, which include the fix for #1062, is anyone else getting lots of these lines in echos/run_stack_handler.txt today?
2023-03-21 14:40:16 {'type': ''} [Warning] No datapaths provided for .csv, will use defaults (may break in production, should be fine in sim)

[rorymac]

I have not run stack handler for a few days, will run it tomorrow and report back.

[meldinman]

yes I get the same log error. It's mostly coming from the default error of data_blob class returning the message when csv_data_path is arg_not_supplied...

[robcarver17]

It's because we are now using instrument config.csv and it's a warning rather than an error, so nothing to panic about, but I will remove it. Currently in the weeds of modifying the data storage code to remove dependency on timedStorage

[meldinman]

yup, there will probably be a bunch of redundant logs to clean now. I am going through them over the last few days...

[rorymac]

The "error" is still there, also I do not know if it is related but the Force roll is spawning a load of children.

2023-03-22 08:14:07 {'type': 'stack_handler', 'strategy_name': '_ROLL_PSEUDO_STRATEGY', 'instrument_code': 'GAS_US_mini', 'contract_order_id': 2634, 'instrument_order_id': 2910}  Sending order (Order ID:2634) Type best for _ROLL_PSEUDO_STRATEGY/GAS_US_mini/20230500_20230600, qty [2, -2], fill [0, 0]@ price, None Parent:2910 Children:[3133, 3134, 3135, 3136, 3137, 3138, 3139, 3140, 3141, 3142, 3143, 3144, 3145, 3146, 3147, 3148, 3149, 3150, 3151, 3152, 3153, 3154, 3155, 3156, 3157, 3158, 3159, 3160, 3161, 3162, 3163, 3164, 3165, 3166, 3167, 3168, 3169, 3170, 3171, 3172, 3173, 3174, 3175, 3176, 3177, 3178, 3179, 3180, 3181, 3182, 3183, 3184, 3185, 3186, 3187, 3188, 3189, 3190, 3191, 3192, 3193, 3194, 3195, 3196, 3197, 3198, 3199, 3200, 3201, 3202, 3203, 3204, 3205, 3206, 3207, 3208, 3209, 3210, 3211, 3212, 3213, 3214, 3215, 3216, 3217, 3218, 3219, 3220, 3221, 3222, 3223, 3224, 3225, 3226, 3227, 3228, 3229, 3230, 3231, 3232, 3233] to algo sysexecution.algos.algo_original_best.algoOriginalBest
2023-03-22 08:14:07 {'type': 'stack_handler', 'strategy_name': '_ROLL_PSEUDO_STRATEGY', 'instrument_code': 'GAS_US_mini', 'contract_order_id': 2634, 'instrument_order_id': 2910}  Cut down order to size [2, -2] from [1, -1] because of algo size limit
2023-03-22 08:14:07 {'type': 'stack_handler', 'strategy_name': '_ROLL_PSEUDO_STRATEGY', 'instrument_code': 'GAS_US_mini', 'contract_order_id': 2634, 'instrument_order_id': 2910}  Created a broker order (Order ID:no order ID) Type limit for _ROLL_PSEUDO_STRATEGY/GAS_US_mini/20230500_20230600, qty [1, -1], fill [0, 0]@ price, None Parent:2634 Children:no_children (not yet submitted or written to local DB)
2023-03-22 08:14:07 {'type': 'stack_handler', 'component': 'ibExecutionStackData', 'strategy_name': '_ROLL_PSEUDO_STRATEGY', 'instrument_code': 'GAS_US_mini', 'contract_order_id': 2634, 'broker_order_id': ''}  Going to submit order (Order ID:no order ID) Type limit for _ROLL_PSEUDO_STRATEGY/GAS_US_mini/20230500_20230600, qty [1, -1], fill [0, 0]@ price, None Parent:2634 Children:no_children to IB
2023-03-22 08:14:07 {'type': 'stack_handler', 'component': 'ibExecutionStackData', 'strategy_name': '_ROLL_PSEUDO_STRATEGY', 'instrument_code': 'GAS_US_mini', 'contract_order_id': 2634, 'broker_order_id': ''}  Order submitted to IB
2023-03-22 08:14:07 {'type': 'stack_handler', 'component': 'ibContractPositionData'}  Reqid 762: 10268 The 'EtradeOnly' order attribute is not supported.
2023-03-22 08:14:07 {'type': 'stack_handler', 'component': 'ibFuturesInstrumentData'}  Reqid 762: 10268 The 'EtradeOnly' order attribute is not supported.
2023-03-22 08:14:07 {'type': 'stack_handler', 'component': 'ibFuturesContractData'}  Reqid 762: 10268 The 'EtradeOnly' order attribute is not supported.
2023-03-22 08:14:07 {'type': 'stack_handler', 'component': 'ibFuturesContractPriceData'}  Reqid 762: 10268 The 'EtradeOnly' order attribute is not supported.
2023-03-22 08:14:07 {'type': 'stack_handler', 'component': 'ibStaticData'}  Reqid 762: 10268 The 'EtradeOnly' order attribute is not supported.
2023-03-22 08:14:07 {'type': 'stack_handler', 'component': 'ibExecutionStackData'}  Reqid 762: 10268 The 'EtradeOnly' order attribute is not supported.
2023-03-22 08:14:07 {'type': 'stack_handler', 'strategy_name': '_ROLL_PSEUDO_STRATEGY', 'instrument_code': 'GAS_US_mini', 'contract_order_id': 2634, 'instrument_order_id': 2910, 'broker_order_id': ''}  Submitted order to IB (Order ID:no order ID) Type limit for _ROLL_PSEUDO_STRATEGY/GAS_US_mini/20230500_20230600, qty [1, -1], fill [0, 0]@ price, None Parent:2634 Children:no_children
ERROR:ib_insync.wrapper:Error 10268, reqId 767: The 'EtradeOnly' order attribute is not supported.
WARNING:ib_insync.wrapper:Canceled order: Trade(contract=Future(symbol='QG', multiplier='2500', exchange='NYMEX', currency='USD', comboLegs=[ComboLeg(conId=297498641, ratio=1, action='BUY', exchange='NYMEX', openClose=0, shortSaleSlot=0, designatedLocation='', exemptCode=-1), ComboLeg(conId=297498644, ratio=1, action='SELL', exchange='NYMEX', openClose=0, shortSaleSlot=0, designatedLocation='', exemptCode=-1)]), order=LimitOrder(orderId=767, clientId=21, action='BUY', totalQuantity=1, lmtPrice=-0.245, account='U3661595'), orderStatus=OrderStatus(orderId=767, status='Cancelled', filled=0, remaining=0, avgFillPrice=0.0, permId=0, parentId=0, lastFillPrice=0.0, clientId=0, whyHeld='', mktCapPrice=0.0), fills=[], log=[TradeLogEntry(time=datetime.datetime(2023, 3, 22, 13, 14, 28, 188725, tzinfo=datetime.timezone.utc), status='PendingSubmit', message=''), TradeLogEntry(time=datetime.datetime(2023, 3, 22, 13, 14, 28, 190671, tzinfo=datetime.timezone.utc), status='Cancelled', message="Error 10268, reqId 767: The 'EtradeOnly' order attribute is not supported.")])

[73ze]

Thanks, Everyone. In my case it does not seem to have impacted order execution and does not seem extremely urgent to fix but since that seemed unintended , wanted to report.