About Waymo Open Dataset preprocessing

[RenyunLi0116]

Hi, after I installed the package as requirement.txt and tried to install with python3 -m pip install gcsfs waymo-open-dataset-tf-2-12-0==1.6.4, I got error:

_ERROR: Cannot install dask[dataframe]==2023.3.1, waymo-open-dataset-tf-2-12-0 and waymo-open-dataset-tf-2-12-0==1.6.4 because these package versions have conflicting dependencies.

The conflict is caused by:
waymo-open-dataset-tf-2-12-0 1.6.4 depends on numpy==1.23
dask[dataframe] 2023.3.1 depends on numpy>=1.21; extra == "dataframe"
matplotlib 3.6.1 depends on numpy>=1.19
pandas 1.5.3 depends on numpy>=1.21.0; python_version >= "3.10"
pandas 1.5.3 depends on numpy>=1.23.2; python_version >= "3.11"

To fix this you could try to:

1. loosen the range of package versions you've specified
2. remove package versions to allow pip to attempt to solve the dependency conflict

ERROR: ResolutionImpossible: for help visit https://pip.pypa.io/en/latest/topics/dependency-resolution/#dealing-with-dependency-conflicts
(waymo_preprocess) ga043:lh3862 22:48:24 O_O_

[RenyunLi0116]

I created a new python 3.10 env and installed python3 -m pip install gcsfs waymo-open-dataset-tf-2-12-0==1.6.4, it worked.

But when I preprocessed Waymo Open Dataset, I got error:

Traceback (most recent call last):
File "/scratch/lh3862/project/dust3r/datasets_preprocess/preprocess_waymo.py", line 257, in
main(args.waymo_dir, args.precomputed_pairs, args.output_dir, workers=args.workers)
File "/scratch/lh3862/project/dust3r/datasets_preprocess/preprocess_waymo.py", line 48, in main
extract_frames(waymo_root, output_dir, workers=workers)
File "/scratch/lh3862/project/dust3r/datasets_preprocess/preprocess_waymo.py", line 78, in extract_frames
parallel_map(process_one_seq, args, star_args=True, workers=workers)
File "/scratch/lh3862/project/dust3r/dust3r/utils/parallel.py", line 67, in parallel_processes
return parallel_threads(*args, **kwargs)
File "/scratch/lh3862/project/dust3r/dust3r/utils/parallel.py", line 44, in parallel_threads
front.append(function(*a) if star_args else function(**a) if kw_args else function(a))
File "/scratch/lh3862/project/dust3r/datasets_preprocess/preprocess_waymo.py", line 90, in process_one_seq
calib, frames = extract_frames_one_seq(osp.join(db_root, seq))
File "/scratch/lh3862/project/dust3r/datasets_preprocess/preprocess_waymo.py", line 117, in extract_frames_one_seq
frame.ParseFromString(bytearray(data.numpy()))
TypeError: expected bytes, bytearray found

Did anyone meet this problem?