mapfile_parser

Map file parser library focusing decompilation projects.

This library is available for Python3 and Rust

Features

• Fast parsing written in Rust.
• Support map formats:
  • GNU ld
  • clang lld
  • Metrowerks ld
• Built-in cli utilities to process the parsed map file (see Examples).

Installing

Python version

See this package at https://pypi.org/project/mapfile_parser/.

The recommended way to install is using from the PyPi release, via pip:

python3 -m pip install -U mapfile_parser

If you use a requirements.txt file in your repository, then you can add this library with the following line:

mapfile_parser>=2.9.2,<3.0.0

Development version

The unstable development version is located at the develop branch. PRs should be made into that branch instead of the main one.

The recommended way to install a locally cloned repo is by passing the -e (editable) flag to pip.

python3 -m pip install -e .

In case you want to mess with the latest development version without wanting to clone the repository, then you could use the following command:

python3 -m pip uninstall mapfile_parser
python3 -m pip install git+https://github.com/Decompollaborate/mapfile_parser.git@develop

NOTE: Installing the development version is not recommended unless you know what you are doing. Proceed at your own risk.

Rust version

See this crate at https://crates.io/crates/mapfile_parser.

To add this library to your project using Cargo:

cargo add mapfile_parser

Or add the following line manually to your Cargo.toml file:

mapfile_parser = "2.9.2"

Versioning and changelog

This library follows Semantic Versioning. We try to always keep backwards compatibility, so no breaking changes should happen until a major release (i.e. jumping from 2.X.X to 3.0.0).

To see what changed on each release check either the CHANGELOG.md file or check the releases page on Github. You can also use this link to check the latest release.

Examples

Various cli examples are provided in the frontends folder. Most of them are re-implementations of already existing tools using this library to show how to use this library and inspire new ideas.

The list can be checked in runtime with python3 -m mapfile_parser --help.

Each one of them can be executed with python3 -m mapfile_parser utilityname, for example python3 -m mapfile_parser pj64_syms.

• bss_check: Check that globally visible bss has not been reordered.
• first_diff: Find the first difference(s) between the built ROM and the base ROM.
• jsonify: Converts a mapfile into a json format.
• pj64_syms: Produce a PJ64-compatible symbol map.
• objdiff_report: Computes current progress of the matched functions. Expects .NON_MATCHING marker symbols on the mapfile to know which symbols are not matched yet.
• progress: Computes current progress of the matched functions. Relies on a splat folder structure and each matched functions no longer having an .s file (i.e: delete the file after matching it).
• sym_info: Display various information about a symbol or address.
• symbol_sizes_csv: Produces a csv summarizing the files sizes by parsing a map file.
• upload_frogress: Uploads current progress (calculated by the progress utility) of the matched functions to frogress.

All these utilities support automatic scanning for a decomp.yaml file from the decomp_settings project. This is the recommended way to use these utilities, because otherwise they need a large number of long parameters to work. They are not meant to be used directly, instead it is recommended to write a small script around them if a project does not desire to adopt the decomp.yaml format.