ActiPass File Exclusion Issue

[loris-lee]

Hi,

I am writing to ask about an issue I encountered while using ActiPass to process data. Even though I select "None" for file exclusion before pressing "Generate Stats," it seems that ActiPass automatically removes files marked as "notok."

I wonder if this is due to using PROPASS mode. Additionally, I would like to know whether the file exclusion options can be changed after generating the statistics.

Thank you for your help!

[pasan-h]

Hi,
Can you follow the criteria for asking help and attach more info? Without those it's difficult to answer. I also wonder why you want stats from such cases? Are you planning on doing a sensitivity analysis for file exclusion?
Also, see highlighted links at the Discussion home for how to request support. Specifically follow. You need to find an example where this happens and attach the quality check visualization for that example.

[loris-lee]

Hi,

Thank you for your response. I appreciate your guidance.

To clarify, I only have access to PROPASS mode, and I noticed that even when selecting "None" for file exclusion before pressing "Generate Stats," 'ProPASS_WideFormat_MasterFile' still automatically removes files marked as "notok."

I am attaching the requested files for further analysis:

Relevant log files from the BatchOut folder.

ActiPASS_BatchLog.txt
ActiPASS_BatchSettings.txt

Regarding your question about why I want statistics from such cases, I value every participant’s data and want to ensure that no usable data is unnecessarily excluded.

Thanks

[pasan-h]

Most probably from ActiPASS perspective, there is No (or not enough) data for it to produce variables. It could also be an error loading files. It could also be due to after auto-trimming data, ActiPASS has no data to generate variables.
Anyway you still didn't attach the most important data, the QC visualization for such case. I think the first step of understanding what is happening after ActiPASS finishes a batch process is to visually check QC figures (see user guide). Variable generation should be the last step (and one shouldn't try to do that if we can afford to do a quality check first).

ActivePAL is restrictive with their file format and open implementation of PAL file parser which is used by ActiPASS can also fail in rare occasions. You can also try to convert problematic files to CSV and try again.

You also didn't explain why you expect ActiPASS to produce data if it finds the file is "NotOK". It literally means from ActiPASS perspective file is not OK. When you set the exclusion to "none" it will try, but there is no guarantee it can. Obviously a software cannot produce variables if something critical has gone wrong in the workflow before variable generation.

[pasan-h]

Hi again,
Perhaps I was not that clear. Anyway in my opinion just trying to include all data in a study just for the sake of not wasting data could lead to even bigger problems than accepting certain amount of exclusion.
ActiPASS facilitates making a good inclusion criteria by having three levels of QC flags (ok, check and notok). It even produces a lot of 24h day specific QC flags.
If it can process data it will also be seen in QC figures. Once you have looked at a lot of (say few hundred) cases of QC figures (mostly cases ActiPASS flag as "Check" or "NotOK" but also "OK") you will gain enough insight into what is happening. Then you will see why a certain case is flag as Check or NotOK (if data can be processed.). But some NotOK cases could be simply due to the fact there is no data (as I wrote above).

Then you can decide which days of which participants you can include. Then do some sensitivity analysis for the people who are excluded and see whether exclusion brings any bias to the study.

Good luck.

[loris-lee]

Thank you, Pasan, for your explanation.

However, I couldn't find the QC visualizations or change the auto-detected QC_Status, even though the user guide states that double-clicking on a row in the QC table should open the corresponding QC visualization.

Attached is a screenshot showing that I clicked 'Show QC Info.'

[pasan-h]

Hi, I am changing the documentation expecting the release of a new version. You will also receive a link to download this new version. Some of the new features shown in above figure are not yet available.
Anyway QC visualizations (if they exist for a cerain case) can be found at: BatchOut folder.

From the two seperate discussion threads you started I can summarise the problems you have faced and why.

1. You have some DATX files coming from actual ActivPAL 4 devices or more recent firmware for ActivPAL 3 devices which cannot be parsed by open-source DATX binary file paser ActiPASS uses. If ActiPASS cannot do that non of the other open source-software will be able to parse them (since all uses the same parser). You need to convert those to correct CSV format (see the other thread you created).

2. ActiPASS cannot process data or generate any results (visualizations, tables) if it cannot load data. So don't expect any output whatsoever if file loading failed. Changing QC_Status, or file-exlcusion settings cannot fix this problem

3. If for some reason ActiPASS non-wear auto-trimmer, trims too much, then QC_Status can be "NotOK". So there can be difference of total duration between ActiPASS and PAL Analysis. In Master QC table ActiPASS shows both original start/stop times as well as start/stop times after auto-trimming data. Those cases will be processed if you change file exclusion criteria in statistics module.

4. Anyway please follow the user guide (you can see the old version here and check QC-visualizations of all processed files. Without doing this you don't "know" how good or bad your results are. I know the tables also provide a lot of QC info. But this qualitative "feel" of your results is very important. Just do this and you will see why this is very important after a while 😀.

5. With upcoming new version of ActiPASS, you can do this easily. But you can just preview files in windows File Explorer just as easily. Just open the the corresponding batch inside "BatchOut" folder in file explorer, turn on file preview and browse the visualizations as seen below.

[loris-lee]

Hi Pasan,

Thank you for your explanation.

I’ve downloaded the latest version of ActiPass and was able to convert the previously unprocessed file to CSV, which worked well.

However, we’ve noticed in some of the data that ActiPass seems to automatically trim the end of certain days and flag those segments as non-wear (NW), even though other software suggests that data should be valid.

For example, in the screenshot below from ActiPass, it shows days trimmed and flagged as NW:

But in another software, the same period is shown as two valid days:

We were wondering if there are any settings that could be adjusted to prevent this.

Thanks again for your continued support.

[loris-lee]

Just following up on this—I'm wondering if having an advanced licence could help address the issue with end-of-day trimming and NW flagging?

[pasan-h]

Hi,
Please start a new thread for topics which are not directly related to the original discussion topic. See guideline for discussions at discussions home.
Also one more thing: You have mentioned "other software" several times in the discussions. In an open community we expect openness. Please just name the exact software and the version. Afterall, we strive for openness so it's kind of insulting when someone try to compare an open tool against "other software" 😊.
Thank you for your understanding.

[loris-lee]

Hi Pasan,

Thank you for the clarification — that makes total sense. I actually avoided naming the software out of politeness, but I understand now that openness is appreciated in this context. I’ll make sure to be more transparent moving forward and will create a new thread for this topic.

Thanks again for your guidance!