README.md: lots of updates

Author: andrewdavidsmith

README.md

@@ -1,12 +1,78 @@
 # Transferase
+
 The transferase system for retrieving methylomes from methbase.
 
 The transferase program is `xfrase` which is quicker to type and will
-be used below. There are several commands within `xfrase` and the best way to
-start is to understand the `dnmtools roi` command, as the information
-functionality provided by `xfrase` is the same. If you need to learn
-about `dnmtools roi` you can find the docs
-[here](https://dnmtools.readthedocs.io/en/latest/roi/)
+be used below. There are several commands within `xfrase` and the best
+way to start is to understand the `dnmtools roi` command, as the
+information functionality provided by `xfrase` is the same. If you
+need to learn about `dnmtools roi` you can find the docs
+[here](https://dnmtools.readthedocs.io/en/latest/roi)
+
+## Installing transferase
+
+# Install the pre-compiled binary
+
+If you are on a reasonably recent Linux (i.e., no older than 10
+yeads), then you can install the binary distribution. First
+download it like this:
+```console
+wget https://github.com/andrewdavidsmith/transferase/releases/download/v0.2.0/transferase-0.2.0-Linux.sh
+```
+
+Then run the downloaded installer (likely you want to first install it
+beneath your home dir):
+```console
+./transferase-0.2.0-Linux.sh --prefix=${PREFIX}
+```
+
+This will prompt you to accept the license, and then it will install
+the `xfrase` binaries in `${PREFIX}/bin`, along with some config files
+in `${PREFIX}/share`. If you want to install it system-wide, and have
+the admin privs, you can do:
+```console
+./transferase-0.2.0-Linux.sh --prefix=/usr/local
+```
+
+If you are on Debian or Ubuntu, and have admin privileges, you can use
+the Debian package and then transferase will be tracked in the package
+management system.  Get the file like this:
+```console
+wget https://github.com/andrewdavidsmith/transferase/releases/download/v0.2.0/transferase-0.2.0-Linux.deb
+```
+
+And then install it like this:
+```console
+sudo dpkg -i ./transferase-0.2.0-Linux.deb
+```
+
+# Building the source
+
+Not recommended unless you know what you are doing. You will need the
+following:
+
+* A compiler that can handle most of C++23, one of
+  - GCC >= [14.2.0](https://gcc.gnu.org/pub/gcc/releases/gcc-14.2.0/gcc-14.2.0.tar.gz)
+  - Clang >= [20.0.0](https://github.com/llvm/llvm-project.git) (no release as of 12/2024)
+* Boost >= [1.85](https://archives.boost.io/release/${BOOST_VERSION}/source/boost_1_85.tar.bz2)
+* CMake >= [3.30](https://github.com/Kitware/CMake/releases/download/v3.30.2/cmake-3.30.2.tar.gz)
+* ZLib, any version, just install it with `apt install zlib1g-dev`,
+  `mamba install -c conda-forge zlib`, etc. From source is fast and
+  easy.
+
+However you install these, remember where you put them and update your
+paths accordingly.
+
+Since transferase uses CMake to generate the build system, there are
+multiple ways to do it, but I like this:
+```shell
+tar -xf transferase-0.2.0-Source.tar.gz
+cd transferase-0.2.0-Source
+cmake -B build \
+  -DCMAKE_BUILD_TYPE=Build    # for a faster xfrase
+cmake --build build -j64      # i.e., if you have 64 cores
+cmake --install build --prefix=${HOME}  # or wherever
+```
 
 ## Make an index file
 
@@ -16,10 +82,12 @@ reference file name is `hg38.fa`, then do this:
 ```console
 xfrase index -v -g hg38.fa -x hg38.cpg_idx
 ```
+
 If `hg38.fa` is roughly 3.0G in size, then you should expect the index
 file `hg38.cpg_idx` to be about 113M in size. This command will also
 create an "index metadata" file `hg38.cpg_idx.json`, which is named by
-just adding the `.json` extension to the provided output file.
+just adding the `.json` extension to the provided output file. You can
+also start with a gzip format file like `hg38.fa.gz`.
 
 ## Make a methylome file
 
@@ -33,52 +101,53 @@ used by `dnmtools`. We again assume that the reference genome is
 `SRX012345.xsym.gz` file. This ensures a correspondence between the
 `SRX012345.xsym.gz` file and the `hg38.cpg_idx` file we will need.
 Here is how to convert such a file into the xfrase format:
-
 ```console
 xfrase compress -v -x hg38.cpg_idx -m SRX012345.xsym.gz -o SRX012345.m16
 ```
 
-If the chromosomes appear out-of-order in `hg38.cpg_idx` and
-`SRX012345.xsym.gz` an error will be reported. As with the `hg38.cpg_idx`
-index file, the methylome file `SRX012345.m16` will be accompanied by
-a metadata file with an additional json extension: `SRX012345.m16.json`.
+As with the `hg38.cpg_idx` index file, the methylome file
+`SRX012345.m16` will be accompanied by a metadata file with an
+additional json extension: `SRX012345.m16.json`.
 
-If you begin with a counts format file, for example `SRX012345.sym`,
-created using the `dnmtools counts` and then `dnmtools sym` commands,
-then you will need to first convert it into `.xsym` format. You can do
-this as follows:
+If you begin with a
+[counts](https://dnmtools.readthedocs.io/en/latest/counts) format
+file, for example `SRX012345.sym`, created using the `dnmtools counts`
+and then `dnmtools sym` commands, then you will need to first convert
+it into `.xsym` format (whether gzipped or not). You can do this as
+follows:
 ```console
 dnmtools xcounts -z -o SRX012345.xsym.gz -c hg38.fa -v SRX012345.sym
 ```
-Once again, be sure to always use the same `hg38.fa` file.  A hash
-function will be used internally to `xfrase` to ensure that the index
-and methylome files correspond to the same reference genome file.
 
-## Run the `lookup` command locally
+Once again, be sure to always use the same `hg38.fa` file.  A hash is
+generated and used internally to `xfrase` to ensure that the index and
+methylome files correspond to the same reference genome file.
 
-This step is to make sure everything is sensible. You will need a set
-of genomic intervals of interest. In this example these will be named
-`intervals.bed`. You also need the index file and the methylome file
-explained in the above steps.
-```console
-xfrase lookup local --log-level debug -x hg38.cpg_idx -m SRX012345.m16 -o intervals_local_output.bed -i intervals.bed
-```
-The index file `hg38.cpg_idx` and the methylome file `SRX012345.m16`
-are the same as explained above. At the time of writing, the
-`intervals.bed` file must be 6-column bed format, so if yours is only
-3 columns, you can use a simple awk command to pad it out as follows:
+## Run the `intervals` command locally
+
+This step is to make sure everything is sensible. Or you might just
+want to keep using this tool for your own analysis (it's fast). You
+will need a set of genomic intervals of interest. In this example
+these will be named `intervals.bed`. You also need the index file and
+the methylome file explained in the above steps.
 ```console
-awk -v OFS="\t" '{print $1,$2,$3,"X",0,"+"}' intervals.bed3 > intervals.bed
+xfrase intervals local -v debug -x hg38.cpg_idx -m SRX012345.m16 -o local_output.bed -i intervals.bed
 ```
 
-The output in the `intervals_local_output.bed` file should be
-consistent with the information in the command:
-
+The index file `hg38.cpg_idx` and the methylome file `SRX012345.m16`
+are the same as explained above. The `intervals.bed` file may contain
+any number of columns, but the first 3 columns must be in 3-column BED
+format: chrom, start, stop for each interval.  The output in the
+`local_output.bed` file should be consistent with the information in
+the command:
 ```console
 dnmtools roi -o intervals.roi intervals.bed SRX012345.xsym.gz
 ```
-The format of the output might be different, but the methylation
+
+The format of these output files are different, but the methylation
 levels on each line (i.e., for each interval) should be identical.
+Note that `dnmtools roi` can fail if intervals are nested, while
+`xfrase intervals` command will still work.
 
 ## Run the `server` command
 
@@ -94,25 +163,25 @@ relevant genome assembly. For now, using the above examples, we
 would have a single index and a single methylome. I will assume these
 are in subdirectories, named `indexes` and `methylomes` respectively,
 of the current directory. Here is a command that will start the server:
-
 ```console
 xfrase server -v debug -s localhost -p 5000 -m methylomes -x indexes
 ```
+
 Not that this will fail with an appropriate error message if port 5000
 is already be in use, and you can just try 5001, etc., until one is
-free. The `-v debug` will ensure you see info beyond just the errors. This
-informtion is logged to the terminal by default.
+free. The `-v debug` will ensure you see info beyond just the
+errors. This informtion is logged to the terminal by default.
 
-## Run the `lookup` command remotely
+## Run the `intervals` command remotely
 
 We will assume for now that "remote" server is running on the local
 machine (localhost) and using port is 5000 (the default). The
-following command should give identical earlier `lookup` command:
-
+following command should give identical earlier `intervals` command:
 ```console
-xfrase lookup remote -v debug  -s localhost -x indexes/hg38.cpg_idx \
-    -o intervals_remote_outout.bed -a SRX012345 -i intervals.bed
+xfrase intervals remote -v debug -s localhost -x indexes/hg38.cpg_idx \
+    -o remote_output.bed -a SRX012345 -i intervals.bed
 ```
+
 Note that now `SRX012345` is not a file this time. Rather, it is a
 methylome name or accession, and should be available on the server. If
 the server can't find the named methylome, it will respond indicating