Bug: pileup with -f FRAG yields OverflowError

[jacob-greene]

Thanks for the work on such a useful tool. The pileup feature is failing for me when I use -f FRAG. This works for -f BED. See below. This failed both from the very large bed file, and the bed file with only the first 10k reads (head -n 10000). Thanks

(macs3) (jgreene3) jgreene3@gizmok54:~$ macs3 pileup -i "$macs3_files_dir/K27me3_K_no_chrM_head10000.bed" -o "$macs3_files_dir/K27me3_K_no_chrM_piledup.bedGraph" -f FRAG
INFO @ 25 Mar 2025 11:45:46: [33 MB] # Existing file /fh/fast/henikoff_s/user/jgreene/projects/H3K27me2/data/2024/SH_all_data/merged_macs3/K27me3_K_no_chrM_piledup.bedGraph will be replaced!
INFO @ 25 Mar 2025 11:45:46: [33 MB] # read alignment files...
INFO @ 25 Mar 2025 11:45:46: [33 MB] # read input file in Paired-end mode.
INFO @ 25 Mar 2025 11:45:46: [33 MB] # read fragments...
Traceback (most recent call last):
File "/home/jgreene3/.local/bin/macs3", line 1017, in
main()
File "/home/jgreene3/.local/bin/macs3", line 99, in main
run(args)
File "/home/jgreene3/.local/lib/python3.9/site-packages/MACS3/Commands/pileup_cmd.py", line 52, in run
treat = load_frag_files_options(options) # return PETrackI object
File "/home/jgreene3/.local/lib/python3.9/site-packages/MACS3/Commands/pileup_cmd.py", line 119, in load_frag_files_options
treat = tp.build_petrack()
File "MACS3/IO/Parser.py", line 1531, in MACS3.IO.Parser.FragParser.build_petrack
File "MACS3/IO/Parser.py", line 1556, in MACS3.IO.Parser.FragParser.build_petrack
OverflowError: value too large to convert to unsigned char

[danielee0707]

Same issue

WARNING @ 30 Mar 2025 18:37:35: [41 MB] Since the format is 'FRAG', `--keep-dup` will be set as 'all'.
INFO  @ 30 Mar 2025 18:37:35: [41 MB]
# Command line: callpeak -t atac_fragments_human.tsv -f FRAG --max-count 1 --barcodes filtered_feature_bc_matrix/barcodes.tsv -g hs -n broad_k27ac_human --verbose 3 --SPMR --bdg --min-length 50 -q 0.1 --broad --broad-cutoff 0.1 --cutoff-analysis --fe-cutoff 0.8
# ARGUMENTS LIST:
# name = broad_k27ac_human
# format = FRAG
# ChIP-seq file = ['atac_fragments_human.tsv']
# control file = None
# effective genome size = 2.91e+09
# band width = 300
# model fold = [5, 50]
# Maximum count in fragment file is set as 1
# qvalue cutoff for narrow/strong regions = 1.00e-01
# qvalue cutoff for broad/weak regions = 1.00e-01
# The maximum gap between significant sites is assigned as the read length/tag size.
# The minimum length of peaks = 50
# Larger dataset will be scaled towards smaller dataset.
# Range for calculating regional lambda is: 10000 bps
# Broad region calling is on
# Additional cutoff on fold-enrichment is: 0.80
# Paired-End mode is on
# MACS will save fragment pileup signal per million reads

INFO  @ 30 Mar 2025 18:37:35: [41 MB] #1 read fragment files...
INFO  @ 30 Mar 2025 18:37:35: [41 MB] #1 read treatment fragments...
INFO  @ 30 Mar 2025 18:37:37: [140 MB]  1000000 fragments parsed
INFO  @ 30 Mar 2025 18:37:38: [158 MB]  2000000 fragments parsed
INFO  @ 30 Mar 2025 18:37:40: [178 MB]  3000000 fragments parsed
INFO  @ 30 Mar 2025 18:37:41: [196 MB]  4000000 fragments parsed
INFO  @ 30 Mar 2025 18:37:43: [207 MB]  5000000 fragments parsed
Traceback (most recent call last):
  File "/home/chlseven/chen/tools/mambaforge/bin/macs3", line 1017, in <module>
    main()
  File "/home/chlseven/chen/tools/mambaforge/bin/macs3", line 55, in main
    run(args)
  File "/home/chlseven/chen/tools/mambaforge/lib/python3.12/site-packages/MACS3/Commands/callpeak_cmd.py", line 71, in run
    (treat, control) = load_frag_files_options(options)
                       ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/chlseven/chen/tools/mambaforge/lib/python3.12/site-packages/MACS3/Commands/callpeak_cmd.py", line 349, in load_frag_files_options
    treat = tp.build_petrack(max_count=options.maxcount)
            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "MACS3/IO/Parser.py", line 1531, in MACS3.IO.Parser.FragParser.build_petrack
  File "MACS3/IO/Parser.py", line 1556, in MACS3.IO.Parser.FragParser.build_petrack
OverflowError: value too large to convert to unsigned char

[taoliu]

@danielee0707 @jacob-greene Thank you for reporting this! It may happen when the column 5th contains values larger than 255. I will try to incease the upper limit to 2**16-1=65535, and throw some warning message if the count is found to be too large.