cp2kdata/md format conversion fails with PRINT_LEVEL LOW despite external files (pos, frc, ener)

[Yiwei666]

I'm encountering an issue when trying to convert CP2K AIMD simulation data to the dpdata format using dpdata.LabeledSystem with the cp2kdata/md format. My CP2K input (.inp) file has PRINT_LEVEL LOW, but I'm explicitly outputting coordinates, forces, and cell information every 10 steps via the &TRAJECTORY, &FORCES, and &CELL keywords in the &MOTION &PRINT section. I also have the .ener file which contains energy information. The simulation was run in the NVT ensemble, so the cell parameters are constant.

1. cp2k output file

The relevant CP2K input snippet:

@SET SYSNAME molten_oxide
@SET CELL 14.8752

&GLOBAL
    PRINT_LEVEL LOW
    PROJECT_NAME ${SYSNAME}
    RUN_TYPE MD
&END GLOBAL

&FORCE_EVAL
    METHOD  QS
    STRESS_TENSOR  ANALYTICAL
    &DFT
    ......
    &END DFT
    &SUBSYS
    ......
    &END SUBSYS
&END FORCE_EVAL

&MOTION
    &MD
    ......
    &END MD
    &PRINT
        &TRAJECTORY
            FILENAME =${SYSNAME}.pos.xyz
            FORMAT XYZ
            UNIT angstrom
            &EACH
                MD 10
            &END EACH
        &END TRAJECTORY
        &FORCES
            FILENAME =${SYSNAME}.frc.xyz
            FORMAT XYZ
            UNIT amu*angstrom/fs^2
            &EACH
                MD 10
            &END EACH
        &END FORCES
        &CELL
            FILENAME =${SYSNAME}.cell
            &EACH
                MD 10
            &END EACH
        &END CELL
    &END PRINT
&END MOTION

The following output files are generated and are available:

molten_oxide.cell
molten_oxide.frc.xyz
molten_oxide.pos.xyz
molten_oxide-1.ener
tem.out

These files contain all the essential information (atomic coordinates, forces, energies, and cell parameters) needed for training a DeepMD potential.

2. Problem Encountered:

When I attempt to load the data using dpdata.LabeledSystem with cp2k_output_name='tem.out', I get an error stating that PRINT_LEVEL LOW does not provide necessary information.

1. First attempt (failing code):

import dpdata
import numpy as np

data = dpdata.LabeledSystem('./', cp2k_output_name='tem.out', fmt='cp2kdata/md')
total_frames = len(data)
print(f'# frame number {total_frames}')

• Error output for first attempt:

Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "C:\Users\sun78\anaconda3\lib\site-packages\dpdata\system.py", line 197, in __init__
    self.from_fmt(
  File "C:\Users\sun78\anaconda3\lib\site-packages\dpdata\system.py", line 234, in from_fmt
    return self.from_fmt_obj(load_format(fmt), file_name, **kwargs)
  File "C:\Users\sun78\anaconda3\lib\site-packages\dpdata\system.py", line 1226, in from_fmt_obj
    data = fmtobj.from_labeled_system(file_name, **kwargs)
  File "C:\Users\sun78\anaconda3\lib\site-packages\cp2kdata\dpdata_plugin.py", line 92, in from_labeled_system
    cp2kmd = Cp2kOutput(output_file=cp2k_output_name,
  File "C:\Users\sun78\anaconda3\lib\site-packages\cp2kdata\output.py", line 78, in __init__
    raise ValueError(
ValueError: please provide cp2k output file with MEDIUM print level. Print Level Low doesn't provide necessary information for initialize the cp2kdata class.

I noticed a similar discussion #723 where users had frc.xyz, pos.xyz, and .ener files for NVT simulations, but faced issues due to PRINT_LEVEL LOW. Following the suggestion in that discussion to explicitly provide the cell parameters and set cp2k_output_name=None, I tried the following:

2. Second attempt (failing code):

import numpy as np
import dpdata

cells = np.array([[14.8752, 0, 0],
                  [0, 14.8752, 0],
                  [0, 0, 14.8752]])
data = dpdata.LabeledSystem("./",
                            cp2k_output_name=None,
                            cells=cells,
                            fmt="cp2kdata/md")

total_frames = len(data)
print(f'# frame number {total_frames} ')

• Error output for second attempt:

Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "C:\Users\sun78\anaconda3\lib\site-packages\dpdata\system.py", line 197, in __init__
    self.from_fmt(
  File "C:\Users\sun78\anaconda3\lib\site-packages\dpdata\system.py", line 234, in from_fmt
    return self.from_fmt_obj(load_format(fmt), file_name, **kwargs)
  File "C:\Users\sun78\anaconda3\lib\site-packages\dpdata\system.py", line 1224, in from_fmt_obj
    data = fmtobj.from_labeled_system(file_name, **kwargs)
  File "C:\Users\sun78\anaconda3\lib\site-packages\cp2kdata\dpdata_plugin.py", line 92, in from_labeled_system
    cp2kmd = Cp2kOutput(output_file=cp2k_output_name,
  File "C:\Users\sun78\anaconda3\lib\site-packages\cp2kdata\output.py", line 119, in __init__
    parse_run_type()
  File "C:\Users\sun78\anaconda3\lib\site-packages\cp2kdata\output.py", line 373, in parse_md
    self.md_info = parse_md_info(self.filename)
  File "C:\Users\sun78\anaconda3\lib\site-packages\cp2kdata\block_parser\header_info.py", line 165, in parse_md_info
    md_info = regrep(
  File "C:\Users\sun78\anaconda3\lib\site-packages\monty\re.py", line 36, in regrep
    gen = reverse_readfile(filename) if reverse else zopen(filename, "rt")
  File "C:\Users\sun78\anaconda3\lib\site-packages\monty\io.py", line 37, in zopen
    name, ext = os.path.splitext(filename)
  File "C:\Users\sun78\anaconda3\lib\ntpath.py", line 204, in splitext
    p = os.fspath(p)
TypeError: expected str, bytes or os.PathLike object, not NoneType

This second error seems to occur because cp2kdata still tries to process cp2k_output_name internally even when it's set to None, leading to a TypeError.

3. Expected behavior:

Could you please provide guidance for this issue? It would be very helpful for users who have accumulated AIMD data with PRINT_LEVEL LOW but have all the critical DeepMD training data available in auxiliary files.

Versions:

• dpdata version: 0.2.24
• cp2kdata version: 0.7.2
• CP2K version: 8.1

[robinzyb]

This is indeed a bug. I have fixed it. Please updated cp2kdata to v0.7.3
and follow the instruction here
https://github.com/robinzyb/cp2kdata/blob/main/docs/dpdata_plugin.md#md-without-cp2k-outputlog-files

[Yiwei666]

Hi @robinzyb,

Thank you for your response and for clarifying that cp2kdata assumes forces in frc.xyz are in Hartree/Bohr by default and converts them to eV/Angstrom for use with DeePMD-kit. However, I have a follow-up question to ensure I understand correctly.

If a user provides forces in frc.xyz in a non-default unit, such as amu*angstrom/fs^2, is cp2kdata able to automatically detect the user-specified force units and convert them to eV/Angstrom? Or would users need to manually convert the forces to Hartree/Bohr before using cp2kdata, since it might be challenging for cp2kdata to identify the units from frc.xyz files alone?

I appreciate your clarification on this point, as it would help ensure proper usage of cp2kdata in cases where non-default units are used.

Best regards,
yiwei

[robinzyb]

I assume that frc files are in the deafult unit. If not, users should coverted to Hatree/Bohr by themselves before sending to cp2kdata.

[Yiwei666]

Hi all,

For the CP2K AIMD data conversion issue with PRINT_LEVEL LOW, here's a temporary workaround:

1. Rerun a short CP2K AIMD simulation (e.g., 10 steps) for the system using the same input parameters but with PRINT_LEVEL MEDIUM. This generates a verbose log file (e.g., tem.out).

2. Extract the header: From this tem.out, copy the initial section up to and including the line:

 DBCSR| CPU Multiplication driver                                           XSMM
 DBCSR| Multrec recursion limit                                              512
 DBCSR| Multiplication stack size                                           1000
 DBCSR| Maximum elements for images                                    UNLIMITED
 ···
 ···
 ···
 MD_PAR| Ensemble type                                                      NVT

Save this as a new file, for instance, tem_header.out.

3. Organize files: Put tem_header.out, the original frc.xyz , and pos.xyz files in the same directory.

4. Load with dpdata: Use this Python script in that directory:

import dpdata

data = dpdata.LabeledSystem('./', cp2k_output_name='tem_header.out', fmt='cp2kdata/md')
total_frames = len(data)
print(f'# frame number {total_frames}')

Hope this is useful. I'm looking forward to the fundamental fix in the new cp2kdata version.

Best,
yiwei

[robinzyb]

There is no bug in the code

The folder strucutre is as follows:

.
├── B2O3-frc-1.xyz
├── B2O3-pos-1.xyz
├── B2O3.cell
├── OT.inp

1. Only *-pos-* and *-frc-* files can be detected. don't forget the -.
2. move -1.ener file out of folder since it has different frame numbers from -pos-

import os


%set_env CP2KDATA_LOG_LEVEL DEBUG

import dpdata
import numpy as np

cp2kmd_dir = "./01_cp2k/"
cp2kmd_output_name = None

#dp = dpdata.LabeledSystem(cp2kmd_dir, cp2k_output_name=cp2kmd_output_name, fmt="cp2kdata/md")
cells = np.array([[9.402,0,0],
                [0,9.402,0],
                [0,0,9.402]])
dp = dpdata.LabeledSystem(cp2kmd_dir, cp2k_output_name=cp2kmd_output_name, cells=cells, ensemble_type="NVT", fmt="cp2kdata/md")

CP2KDATA| 2025-06-05 21:50:07,132 - DEBUG    cp2kdata.dpdata_plugin                  : --- You are parsing data using package Cp2kData ---
CP2KDATA| 2025-06-05 21:50:07,134 - DEBUG    cp2kdata.utils                          : Parsing Structures from ./01_cp2k/B2O3-pos-1.xyz
CP2KDATA| 2025-06-05 21:50:07,221 - DEBUG    cp2kdata.utils                          : Parsing Forces     from ./01_cp2k/B2O3-frc-1.xyz
CP2KDATA| 2025-06-05 21:50:07,243 - DEBUG    cp2kdata.output                         : No stress tensor information found, omitted.
CP2KDATA| 2025-06-05 21:50:07,244 - DEBUG    cp2kdata.utils                          : Parsing Cells      from ./01_cp2k/B2O3.cell
CP2KDATA| 2025-06-05 21:50:07,245 - DEBUG    cp2kdata.dpdata_plugin                  : Missing the atomic kind informations, atom names are true chemical symbols.
CP2KDATA| 2025-06-05 21:50:07,246 - DEBUG    cp2kdata.dpdata_plugin                  : --- You are parsing data using package Cp2kData ---
env: CP2KDATA_LOG_LEVEL=DEBUG