Analysis of Raw Data

[lithium-ions]

This is a more general question, but how is cellpy able to analyze the .res and other "raw" files from battery cyclers? Unfortunately, it is not totally working for my .res files,.

My code

import cellpy
filepath=(r"c:\Users\lujohnso\Documents\snek\virtual_env\pyBamm\TR23111_Test_3E.res")
print(filepath)
c= cellpy.get(filepath) #create new cellpy object
print(c.data.summary.head())

Returns:

> PS C:\Users\lujohnso\Documents\snek\virtual_env\pyBamm> & c:/Users/lujohnso/Documents/snek/virtual_env/pyBamm/Scripts/python.exe c:/Users/lujohnso/Documents/snek/virtual_env/pyBamm/thisIsActuallyCellTest.py
> c:\Users\lujohnso\Documents\snek\virtual_env\pyBamm\TR23111_Test_3E.res
> Traceback (most recent call last):
>   File "c:\Users\lujohnso\Documents\snek\virtual_env\pyBamm\thisIsActuallyCellTest.py", line 4, in <module>
>     c= cellpy.get(filepath) #create new cellpy object
>   File "C:\Users\lujohnso\Documents\snek\virtual_env\pyBamm\lib\site-packages\cellpy\readers\cellreader.py", line 6118, in get
>     cellpy_instance.from_raw(
>   File "C:\Users\lujohnso\Documents\snek\virtual_env\pyBamm\lib\site-packages\cellpy\readers\cellreader.py", line 1200, in from_raw
>     new_data = raw_file_loader(
>   File "C:\Users\lujohnso\Documents\snek\virtual_env\pyBamm\lib\site-packages\cellpy\readers\instruments\base.py", line 271, in loader_executor
>     cellpy_data = self.loader(*args, **kwargs)
>   File "C:\Users\lujohnso\Documents\snek\virtual_env\pyBamm\lib\site-packages\cellpy\readers\instruments\arbin_res.py", line 841, in loader
>     new_data = self._loader_win(
>   File "C:\Users\lujohnso\Documents\snek\virtual_env\pyBamm\lib\site-packages\cellpy\readers\instruments\arbin_res.py", line 557, in _loader_win
>     normal_df = self._load_win_res_auxiliary_table(
>   File "C:\Users\lujohnso\Documents\snek\virtual_env\pyBamm\lib\site-packages\cellpy\readers\instruments\arbin_res.py", line 594, in _load_win_res_auxiliary_table
>     aux_df, aux_global_data_df = self._aux_to_wide(aux_df, aux_global_data_df)
>   File "C:\Users\lujohnso\Documents\snek\virtual_env\pyBamm\lib\site-packages\cellpy\readers\instruments\arbin_res.py", line 647, in _aux_to_wide
>     aux_df = aux_df.unstack(2).unstack(1).dropna(axis=1)
>   File "C:\Users\lujohnso\Documents\snek\virtual_env\pyBamm\lib\site-packages\pandas\core\frame.py", line 8413, in unstack
>     result = unstack(self, level, fill_value)
>   File "C:\Users\lujohnso\Documents\snek\virtual_env\pyBamm\lib\site-packages\pandas\core\reshape\reshape.py", line 478, in unstack
>     return _unstack_frame(obj, level, fill_value=fill_value)
>   File "C:\Users\lujohnso\Documents\snek\virtual_env\pyBamm\lib\site-packages\pandas\core\reshape\reshape.py", line 505, in _unstack_frame
>     unstacker = _Unstacker(obj.index, level=level, constructor=obj._constructor)
>   File "C:\Users\lujohnso\Documents\snek\virtual_env\pyBamm\lib\site-packages\pandas\core\reshape\reshape.py", line 140, in __init__
>     self._make_selectors()
>   File "C:\Users\lujohnso\Documents\snek\virtual_env\pyBamm\lib\site-packages\pandas\core\reshape\reshape.py", line 192, in _make_selectors
>     raise ValueError("Index contains duplicate entries, cannot reshape")
> ValueError: Index contains duplicate entries, cannot reshape

I would greatly appreciate help in understanding why this error may be occuring on basic setup. Or, better, insight into how cellpy is able the raw files from arbin and maccor cyclers would be great as I try to see what may be happening. My data does contain external temperature data.

Thanks!!

[jepegit]

Hi @lithium-ions,

When using the get method, cellpy initializes an instrument-reader (corresponding to the type of file you proved / file-extension, or it can be set as a parameter). It then runs the loader method on the instrument-reader class.

In your case, cellpy crashes when it tries to do a pandas operation of the data it read from the auxiliary table of arbin: aux_df = aux_df.unstack(2).unstack(1).dropna(axis=1) (this is ran inside the ._load_win_res_auxiliary_table method of the reader). So, there is a buggy thing happening during loading the auxiliary table somewhere in cellpy. Sorry for that.

I must admit (a bit shamefully) that we do not record the temperature within the arbin files in our lab that uses arbin testers (we use the arbin testers primarily for material development / coin-cells). We do have all the cells inside temperature chambers, and we write in our log what temperature the chambers are set on. Not perfect, I know, but hope to improve the procedure very soon. As a result, we don't have any own data we can test the code on with auxiliary data. We got some data some years ago (added in an issue on github), and we use that in the test suite. Not enough to catch this bug, unfortunately.

If you are willing to help, either in coding or providing some data, that would be great. If you need some help getting started on coding, let me know.

All the best,
Jan Petter

[lithium-ions]

Hi Jan,

Thanks for getting back to me! With some more work, and actually learning about SQL queries (which when I asked my question before I knew nothing about) I was able to get a system that could handle the auxiliary table. I believe the error occurred because the data had 2 separate temperature probes. This leads to the "Auxiliary_Index" column of "Auxiliary_Table" to be either 0 or 1. In my case 0 corresponds to cell temperature, and 1 corresponds to ambient chamber temperature. I unfortunately cannot provide my .res file to you (I really would if I could).

However, for your reference, this code worked to read the data, and then save a CSV for each cell in the .res file with all relevant data from "Channel_Normal_Table" and "Auxiliary_Table". Forgive me as I am very new to python (I am somewhat more familiar with MATLAB)

importing pyodbc and pandas
import pyodbc
import pandas as pd
#connection string to Arbin.res file save location
constr=(r'DRIVER={Microsoft Access Driver (*.mdb, *.accdb)};'r'DBQ=C:\Users\lujohnso\Documents\snek\virtual_env\pyBamm\TR23049_Test2A_cg_4-19-23.res;')

#query to get global table data from .res
query0='SELECT * FROM Global_Table'
conx=pyodbc.connect(constr)
cursor=conx.cursor();
cursor.execute(query0);
globalData=cursor.fetchall()
header_row0=[]

#getting header names and inserting them
for r in cursor.columns(table='Global_Table'):
#    # print(r.column_name)
    header_row0.append(r.column_name)
globalData.insert(0,header_row0)

#putting global data into pandas dataframe
globalDF=pd.DataFrame(globalData)
globalDF = globalDF.rename(columns=globalDF.iloc[0]).drop(globalDF.index[0]).reset_index(drop=True)
print(globalDF)
cellIndices=globalDF['Test_ID'].values
numCells=max(cellIndices)
testNames=globalDF['Test_Name'].values
print(cellIndices)

#For loop that Makes three SQL queries for data, and creates a pandas dataframe->CSV for each cell of the test contained in the .res file
for r in range(numCells):
    testID=str(cellIndices[r])
    testName=str(testNames[r])
    print(testID)
    print(testName)

    query1="SELECT * FROM Channel_Normal_Table WHERE Test_ID=%s" %(testID) #query for Channel data (dchg energy etc)
    query2="SELECT * FROM Auxiliary_Table WHERE Auxiliary_Index=0 AND Test_ID=%s" %(testID) #query for auxiliary data corresponding to cell temperature data
    query3="SELECT * FROM Auxiliary_Table WHERE Auxiliary_Index=1 AND Test_ID=%s" %(testID) #query for auxiliary data corresponding to ambient temperature data

    conx=pyodbc.connect(constr)
    cursor=conx.cursor();
    cursor.execute(query1);
    #normal channel data
    cyclerData=cursor.fetchall()
    #print(cyclerData[:5])
    cursor.execute(query2)
    cellTemp=cursor.fetchall()
    #print(cellTemp[:5])
    cursor.execute(query3)
    ambientTemp=cursor.fetchall()
    conx.close
    header_row1 = []
    header_row2 = []
    header_row3 = []

    for r in cursor.columns(table='Channel_Normal_Table'):
        # print(r.column_name)
        header_row1.append(r.column_name)
    cyclerData.insert(0,header_row1)    

    for r in cursor.columns(table='Auxiliary_Table'):
        header_row2.append(r.column_name)    
    cellTemp.insert(0,header_row2)

    ambientTemp.insert(0,header_row2)    


    #Creating dataframe 1 (CYCLER DATA)
    df1 = pd.DataFrame(cyclerData)
    df1 = df1.rename(columns=df1.iloc[0]).drop(df1.index[0]).reset_index(drop=True)
    print(df1[:5])
    df1=df1.sort_values(by=['Test_ID','Data_Point'])
    print(df1[:5])

    #Creating dataframe 2 (CELL TEMEPERATURE DATA)
    df2 = pd.DataFrame(cellTemp)
    df2 = df2.rename(columns=df2.iloc[0]).drop(df2.index[0]).reset_index(drop=True)
    df2=df2.sort_values(by=['Test_ID','Data_Point'])
    print(df2[:5])
    df2=df2.rename(columns={'X':'cell_temp'})
    print(df2[:5])

    #Creating dataframe 3 (AMBIENT TEMP DATA)
    df3 = pd.DataFrame(ambientTemp)
    df3 = df3.rename(columns=df3.iloc[0]).drop(df3.index[0]).reset_index(drop=True)
    df3=df3.rename(columns={'X':'ambient_temp'})
    df3=df3.sort_values(by=['Test_ID','Data_Point'])

    df= pd.concat([df1,df2,df3],axis=1)
    print(df[:10])

    #Creating a file name for each cell contained in .res file
    saveString="C:/Users/lujohnso/Documents/snek/virtual_env/pyBamm/%s-cell%s.csv" %(testName,testID)
    #Saving dataframe as csv for analysis in MATLAB/EXCEL etc.
    df.to_csv(saveString)

Let me know if you have questions. I know this code doesn't really use the vast majority of what cell.py can do, but I figured I should share what I got to work. I do imagine the organizational aspects of cell.py would be quite nice to use for storing this data.

Best,
Luke