OpenStack failure: Invalid client version '' #45
Description
Getting "Invalid client version ''" when trying to add an Icehouse env (2014.1.1) on the latest stable ZenPack (1.2.2) on Zenoss 4.2.4
[zenoss@zenoss bin]$ zencommand run -v10 --showfullcommand --showrawresults --device=23.108.36.162
2014-09-04 14:50:39,345 DEBUG zen.zencommand: Starting PBDaemon initialization
2014-09-04 14:50:39,345 INFO zen.zencommand: Connecting to localhost:8789
2014-09-04 14:50:39,347 DEBUG zen.pbclientfactory: Starting connection...
2014-09-04 14:50:39,347 DEBUG zen.zencommand: Logging in as admin
2014-09-04 14:50:39,348 DEBUG zen.pbclientfactory: Connected
2014-09-04 14:50:39,348 DEBUG zen.pbclientfactory: Cancelling connect timeout
2014-09-04 14:50:39,348 DEBUG zen.pbclientfactory: Sending credentials
2014-09-04 14:50:39,351 DEBUG zen.pbclientfactory: Cancelling connect timeout
2014-09-04 14:50:39,351 INFO zen.zencommand: Connected to ZenHub
2014-09-04 14:50:39,352 DEBUG zen.zencommand: Setting up initial services: EventService, Products.ZenHub.services.CommandPerformanceConfig
2014-09-04 14:50:39,352 DEBUG zen.zencommand: Chaining getInitialServices with d2
2014-09-04 14:50:39,352 DEBUG zen.pbclientfactory: pinging perspective
2014-09-04 14:50:39,355 DEBUG zen.zencommand: Loaded service EventService from zenhub
2014-09-04 14:50:39,355 DEBUG zen.zencommand: Loaded service Products.ZenHub.services.CommandPerformanceConfig from zenhub
2014-09-04 14:50:39,356 DEBUG zen.zencommand: Queued event (total of 1) {'rcvtime': 1409867439.355932, 'severity': 0, 'component': 'zencommand', 'agent': 'zencommand', 'summary': 'started', 'manager': 'zenoss.ubiquityservers.com', 'device': 'localhost', 'eventClass': '/App/Start', 'monitor': 'localhost'}
2014-09-04 14:50:39,356 DEBUG zen.zencommand: Sending 1 events, 0 perf events, 0 heartbeats
2014-09-04 14:50:39,357 DEBUG zen.zencommand: Calling connected.
2014-09-04 14:50:39,357 DEBUG zen.collector.config: Heartbeat timeout set to 900s
2014-09-04 14:50:39,357 DEBUG zen.collector.scheduler: add task configLoader, <Products.ZenCollector.config.ConfigurationLoaderTask object at 0x5a982d0> using 1200 second interval
2014-09-04 14:50:39,358 DEBUG zen.zencommand: Performing periodic maintenance
2014-09-04 14:50:39,358 DEBUG zen.pbclientfactory: perspective ponged
2014-09-04 14:50:39,358 DEBUG zen.pbclientfactory: Cancelling ping timeout
2014-09-04 14:50:39,359 DEBUG zen.collector.scheduler: Task configLoader starting on 1200 second intervals
2014-09-04 14:50:39,359 DEBUG zen.collector.scheduler: Task configLoader changing state from IDLE to QUEUED
2014-09-04 14:50:39,361 DEBUG zen.collector.scheduler: Task configLoader changing state from QUEUED to RUNNING
2014-09-04 14:50:39,361 DEBUG zen.collector.config: configLoader gathering configuration
2014-09-04 14:50:39,361 DEBUG zen.collector.config: Fetching daemon configuration properties
2014-09-04 14:50:39,382 DEBUG zen.collector.scheduler: Task configLoader changing state from RUNNING to FETCHING_MISC_CONFIG
2014-09-04 14:50:39,382 DEBUG zen.zencommand: Updated configCycleInterval preference to 360
2014-09-04 14:50:39,382 DEBUG zen.zencommand: Changing config task interval from 20 to 360 minutes
2014-09-04 14:50:39,382 DEBUG zen.collector.scheduler: Stopping task configLoader, <Products.ZenCollector.config.ConfigurationLoaderTask object at 0x5a982d0>
2014-09-04 14:50:39,382 DEBUG zen.collector.scheduler: call finished LoopingCall<1200>(CallableTask: configLoader, (), *{}) : LoopingCall<1200>(CallableTask: configLoader, (), *{})
2014-09-04 14:50:39,383 INFO zen.collector.scheduler: Detailed Task Statistics:
configLoader Current State: FETCHING_MISC_CONFIG Successful_Runs: 1 Failed_Runs: 0 Missed_Runs: 0
Detailed Task States:
configLoader State: RUNNING Total: 1 Total Elapsed: 0.0206 Min: 0.0206 Max: 0.0206 Mean: 0.0206 StdDev: 0.0000
configLoader State: QUEUED Total: 1 Total Elapsed: 0.0024 Min: 0.0024 Max: 0.0024 Mean: 0.0024 StdDev: 0.0000
2014-09-04 14:50:39,383 DEBUG zen.collector.config: Heartbeat timeout set to 900s
2014-09-04 14:50:39,383 DEBUG zen.collector.scheduler: add task configLoader, <Products.ZenCollector.config.ConfigurationLoaderTask object at 0x5a98050> using 21600 second interval
2014-09-04 14:50:39,384 DEBUG zen.zencommand: Updated defaultRRDCreateCommand preference to ('RRA:AVERAGE:0.5:1:600', 'RRA:AVERAGE:0.5:6:600', 'RRA:AVERAGE:0.5:24:600', 'RRA:AVERAGE:0.5:288:600', 'RRA:MAX:0.5:6:600', 'RRA:MAX:0.5:24:600', 'RRA:MAX:0.5:288:600')
2014-09-04 14:50:39,384 DEBUG zen.collector.config: Fetching threshold classes
2014-09-04 14:50:39,390 DEBUG zen.zencommand: Loading classes ['Products.ZenModel.MinMaxThreshold', 'Products.ZenModel.ValueChangeThreshold']
2014-09-04 14:50:39,391 DEBUG zen.collector.config: Fetching collector thresholds
2014-09-04 14:50:39,629 DEBUG zen.thresholds: Updating threshold ('high event queue', ('localhost collector', ''))
2014-09-04 14:50:39,629 DEBUG zen.thresholds: Updating threshold ('zenmodeler cycle time', ('localhost collector', ''))
2014-09-04 14:50:39,629 DEBUG zen.collector.config: Fetching configurations
2014-09-04 14:50:39,692 DEBUG zen.zencommand: updateDeviceConfigs: updatedConfigs=['23.108.36.162']
2014-09-04 14:50:39,693 DEBUG zen.zencommand: Processing configuration for 23.108.36.162
2014-09-04 14:50:39,693 DEBUG zen.daemon: DummyListener: configuration 23.108.36.162 added
2014-09-04 14:50:39,693 DEBUG zen.collector.tasks: Splitting config 23.108.36.162
2014-09-04 14:50:39,694 DEBUG zen.zencommand: Tasks for config 23.108.36.162: {'23.108.36.162 300 Local': <main.SshPerformanceCollectionTask object at 0x60236d0>}
2014-09-04 14:50:39,694 DEBUG zen.collector.scheduler: add task 23.108.36.162 300 Local, COMMAND schedule Name: 23.108.36.162 300 Local configId: 23.108.36.162 Datasources: 1 using 300 second interval
2014-09-04 14:50:39,694 DEBUG zen.collector.scheduler: Task 23.108.36.162 300 Local starting on 300 second intervals
2014-09-04 14:50:39,695 DEBUG zen.collector.scheduler: Task 23.108.36.162 300 Local changing state from IDLE to QUEUED
2014-09-04 14:50:39,695 DEBUG zen.zencommand: purgeOmittedDevices: deletedConfigs=
2014-09-04 14:50:39,695 DEBUG zen.collector.scheduler: Task configLoader finished, result: 'Configuration loaded'
2014-09-04 14:50:39,696 DEBUG zen.collector.scheduler: Task 23.108.36.162 300 Local changing state from QUEUED to RUNNING
2014-09-04 14:50:39,696 DEBUG zen.zencommand: Running command(s) locally
2014-09-04 14:50:39,696 DEBUG zen.collector.scheduler: Task 23.108.36.162 300 Local changing state from RUNNING to FETCH_DATA
2014-09-04 14:50:39,696 INFO zen.zencommand: Datasource OpenStackEndpoint/openstack command: $ZENHOME/libexec/poll_openstack.py 'admin' '$KEY' 'd889d9097f38460da6340cc4e69c5110' 'http://23.108.36.162:5000/v2.0' '' 'RegionOne'
2014-09-04 14:50:39,696 DEBUG zen.zencommand: Running $ZENHOME/libexec/poll_openstack.py
2014-09-04 14:50:40,235 DEBUG zen.zencommand: Datasource: openstack Received exit code: 0 Output:
'{"events": [{"eventClassKey": "openStackFailure", "summary": "OpenStack failure: Invalid client version ''. must be one of: 3, 2, 1.1", "severity": 5, "eventKey": "openStackFailure"}]}\n'
2014-09-04 14:50:40,235 DEBUG zen.zencommand: Process OpenStackEndpoint/openstack stopped (0), 0.53 seconds elapsed
2014-09-04 14:50:40,235 DEBUG zen.collector.scheduler: Task 23.108.36.162 300 Local changing state from FETCH_DATA to PARSING_DATA
2014-09-04 14:50:40,236 DEBUG zen.collector.scheduler: Task 23.108.36.162 300 Local changing state from PARSING_DATA to STORE_PERF_DATA
2014-09-04 14:50:40,237 DEBUG zen.zencommand: Queued event (total of 1) {'rcvtime': 1409867440.237171, 'severity': 5, 'eventClassKey': u'openStackFailure', 'agent': 'zencommand', 'summary': u"OpenStack failure: Invalid client version ''. must be one of: 3, 2, 1.1", 'manager': 'zenoss.ubiquityservers.com', 'eventKey': u'openStackFailure', 'device': '23.108.36.162', 'device_guid': '0bc6b751-1e14-4ffd-9bd0-ecfa71905727', 'monitor': 'localhost'}
2014-09-04 14:50:40,237 DEBUG zen.zencommand: Queued event (total of 2) {'rcvtime': 1409867440.23758, 'device_guid': '0bc6b751-1e14-4ffd-9bd0-ecfa71905727', 'component': '', 'agent': 'zencommand', 'manager': 'zenoss.ubiquityservers.com', 'device': '23.108.36.162', 'eventClass': '/Cmd/Fail', 'severity': 0, 'monitor': 'localhost', 'summary': 'Datasource OpenStackEndpoint/openstack command completed successfully', 'eventKey': 'openstack'}
2014-09-04 14:50:40,237 DEBUG zen.collector.scheduler: Task 23.108.36.162 300 Local finished, result: [(<Products.ZenRRD.zencommand.Cmd instance at 0x5a99320>, ParsedResults
events: [{'eventClassKey': u'openStackFailure',
'eventKey': u'openStackFailure',
'severity': 5,
'summary': u"OpenStack failure: Invalid client version ''. must be one of: 3, 2, 1.1"},
{'component': '',
'device': '23.108.36.162',
'eventClass': '/Cmd/Fail',
'eventKey': 'openstack',
'severity': 0,
'summary': 'Datasource OpenStackEndpoint/openstack command completed successfully'}]
values: []})]
2014-09-04 14:50:40,239 DEBUG zen.collector.scheduler: Task 23.108.36.162 300 Local changing state from STORE_PERF_DATA to IDLE
2014-09-04 14:50:40,239 INFO zen.zencommand: 1 devices processed (0 datapoints)
2014-09-04 14:50:40,239 INFO zen.collector.scheduler: Tasks: 2 Successful_Runs: 1 Failed_Runs: 0 Missed_Runs: 0 Queued_Tasks: 0 Running_Tasks: 1
2014-09-04 14:50:40,239 DEBUG zen.collector.scheduler: In shutdown stage before
2014-09-04 14:50:40,239 DEBUG zen.collector.scheduler: Stopping running task 23.108.36.162 300 Local
2014-09-04 14:50:40,240 DEBUG zen.collector.scheduler: call finished LoopingCall<300>(CallableTask: 23.108.36.162 300 Local, (), *{}) : LoopingCall<300>(CallableTask: 23.108.36.162 300 Local, (), *{})
2014-09-04 14:50:40,240 DEBUG zen.collector.scheduler: Removing task 23.108.36.162 300 Local
2014-09-04 14:50:40,240 DEBUG zen.collector.scheduler: Removing task configLoader
2014-09-04 14:50:40,240 DEBUG zen.collector.scheduler: tasks to clean set([<Products.ZenCollector.config.ConfigurationLoaderTask object at 0x5a98050>, <Products.ZenCollector.config.ConfigurationLoaderTask object at 0x5a982d0>, <main.SshPerformanceCollectionTask object at 0x60236d0>])
2014-09-04 14:50:40,240 DEBUG zen.collector.scheduler: Cleanup on task configLoader <Products.ZenCollector.config.ConfigurationLoaderTask object at 0x5a98050>
2014-09-04 14:50:40,240 DEBUG zen.collector.scheduler: Scheduler._cleanupTaskComplete: result=None task.name=configLoader
2014-09-04 14:50:40,240 DEBUG zen.collector.scheduler: Cleanup on task configLoader <Products.ZenCollector.config.ConfigurationLoaderTask object at 0x5a982d0>
2014-09-04 14:50:40,241 DEBUG zen.collector.scheduler: Scheduler._cleanupTaskComplete: result=None task.name=configLoader
2014-09-04 14:50:40,241 DEBUG zen.collector.scheduler: Cleanup on task 23.108.36.162 300 Local COMMAND schedule Name: 23.108.36.162 300 Local configId: 23.108.36.162 Datasources: 1
2014-09-04 14:50:40,241 DEBUG zen.collector.scheduler: Scheduler._cleanupTaskComplete: result=None task.name=23.108.36.162 300 Local
2014-09-04 14:50:40,241 DEBUG zen.zencommand: Tried to stop reactor that was stopped
2014-09-04 14:50:40,241 INFO zen.zencommand: Daemon CollectorDaemon shutting down
2014-09-04 14:50:40,242 DEBUG zen.zencommand: Sending 2 events, 0 perf events, 0 heartbeats
2014-09-04 14:50:40,263 DEBUG zen.zencommand: Removing service EventService
2014-09-04 14:50:40,263 DEBUG zen.zencommand: Removing service Products.ZenHub.services.CommandPerformanceConfig
2014-09-04 14:50:40,264 DEBUG zen.pbclientfactory: Lost connection to 127.0.0.1:8789 - [Failure instance: Traceback (failure with no frames): <class 'twisted.internet.error.ConnectionLost'>: Connection to the other side was lost in a non-clean fashion: Connection lost.
]
2014-09-04 14:50:40,264 DEBUG zen.collector.scheduler: In shutdown stage during
2014-09-04 14:50:40,265 DEBUG zen.collector.scheduler: In shutdown stage after