Skip to content

Commit 56931f2

Browse files
IlyaMuravjovIlyaMuravjov
committed
Reorganize markdown docs
1 parent 7a2d326 commit 56931f2

File tree

8 files changed

+428
-251
lines changed

8 files changed

+428
-251
lines changed

‎README.md

Lines changed: 10 additions & 53 deletions
Original file line numberDiff line numberDiff line change
@@ -1,76 +1,33 @@
11
## CFPQ_PyAlgo
22

33
The CFPQ_PyAlgo is a repository for developing, testing and evaluating solvers for
4-
Formal-Language-Constrained Path Problems, such as Context-Free Path Queries and Regular Path Queries.
4+
Formal-Language-Constrained Path Problems, such as Context-Free Path Queries (CFPQ) and Regular Path Queries (RPQ).
55

6-
All algorithms are based on the [GraphBLAS](http://graphblas.org/index.php?title=Graph_BLAS_Forum) framework that allows to represent graphs as matrices
6+
All algorithms are based on the [GraphBLAS](http://graphblas.org) framework that allows to represent graphs as matrices
77
and work with them in terms of linear algebra.
88

99
## Installation
10-
First of all you need to clone repository with its submodules:
1110

12-
```bash
13-
git clone --recurse-submodules -b murav/optimize-matrix https://github.com/JetBrains-Research/CFPQ_PyAlgo.git
14-
cd CFPQ_PyAlgo/
15-
git submodule init
16-
git submodule update
17-
```
18-
Then the easiest way to get started is to use Docker. An alternative is to install everything directly.
19-
20-
### Using Docker
21-
The first way to start is to use Docker:
22-
23-
```bash
24-
# build docker image
25-
docker build --tag cfpq_py_algo .
26-
27-
# run docker container
28-
docker run --rm -it -v ${PWD}:/CFPQ_PyAlgo cfpq_py_algo bash
29-
```
30-
After it, you can develop everything locally and run tests and benchmarks inside the container.
31-
Also, you can use PyCharm Professional and [configure an interpreter using Docker](https://www.jetbrains.com/help/pycharm/using-docker-as-a-remote-interpreter.html).
32-
33-
### Direct install
34-
The other way is to install everything into your local Python 3.9 interpreter or virtual environment.
35-
36-
First of all you need to install [pygraphblas](https://github.com/michelp/pygraphblas) package.
37-
```bash
38-
pip3 install pygraphblas==5.1.8.0
39-
```
40-
Secondly you need to install `cfpq_data_devtools` package and other requirements:
41-
42-
```bash
43-
cd deps/CFPQ_Data
44-
pip3 install -r requirements.txt
45-
python3 setup.py install --user
46-
47-
cd ../../
48-
pip3 install pygraphblas==5.1.8.0 # optional (needed for legacy algorithms and their tests)
49-
pip3 install -r requirements.txt
50-
```
51-
To check if the installation was successful you can run simple tests
52-
```bash
53-
python3 -m pytest test -v -m "CI"
54-
```
11+
For the installation instructions, refer to [docs/install.md](docs/install.md).
5512

5613
## CLI
57-
CFPQ_Algo provides a command line interface for running
14+
CFPQ_PyAlgo provides a command line interface for running
5815
all-pairs CFPQ solver with relation query semantics.
5916

60-
See [cfpq_cli/README](cfpq_cli/README.md) for more details.
17+
For more details, refer to [docs/cli.md](docs/cli.md).
6118

6219
## Evaluation
6320

64-
CFPQ_PyAlgo provides scripts for performing evaluating performance
65-
of various CFPQ solvers (icluding third-party ones).
21+
CFPQ_PyAlgo provides scripts for evaluating performance
22+
of various CFPQ solvers (including third-party ones).
6623

67-
See [cfpq_eval/README](cfpq_eval/README.md) for more details.
24+
For more details, refer to [docs/eval.md](docs/eval.md).
6825

6926
## Project structure
70-
The global project structure is the following:
27+
The global project structure is the following.
7128

7229
```
73-
├── cfpq_algo - new optimized CFPQ algorithm implementations
30+
├── cfpq_algo - FastMatrixCFPQ and MatrixCFPQ algorithms implementations
7431
├── cfpq_cli - scripts for running CFPQ algorithms
7532
├── cfpq_eval - scripts for evaluating performance of various CFPQ solvers (icluding third-party ones)
7633
├── cfpq_matrix - matrix wrappers that improve performance of operations with matrices

‎cfpq_cli/README.md

Lines changed: 3 additions & 102 deletions
Original file line numberDiff line numberDiff line change
@@ -1,106 +1,7 @@
1-
# CFPQ_CLI
1+
## CFPQ CLI
22

33
The `cfpq_cli` module provides a Command Line Interface (CLI) for solving
4-
Context-Free Language Reachability (CFL-r) problem for all vertex pairs
4+
[the Context-Free Language Reachability (CFL-r) problem](../docs/clfr_problem) for all vertex pairs
55
in a graph with respect to a specified context-free grammar.
66

7-
## Getting Started
8-
9-
Ensure the CFPQ_PyAlgo project is properly set up on your system before using the CLI.
10-
Setup instructions are available in the project's main [README](../README.md).
11-
12-
## Usage
13-
14-
### Running the Script
15-
16-
For detailed information on script options, execute the following command:
17-
18-
```bash
19-
cd .. # Should be run from CFPQ_PyAlgo project root directory
20-
python3 -m cfpq_cli.run_all_pairs_cflr --help
21-
```
22-
23-
The basic command usage is as follows:
24-
25-
```
26-
python3 -m cfpq_cli.run_all_pairs_cflr [OPTIONS] ALGORITHM GRAPH GRAMMAR
27-
```
28-
29-
- `ALGORITHM` selects the algorithm. The available options are `IncrementalAllPairsCFLReachabilityMatrix` and `NonIncrementalAllPairsCFLReachabilityMatrix`.
30-
- `GRAPH` specifies the path to the graph file.
31-
- `GRAMMAR` indicates the path to the grammar file.
32-
33-
#### Optional Arguments
34-
35-
- `--time-limit TIME_LIMIT` sets the maximum execution time in seconds.
36-
- `--out OUT` specifies the output file for saving vertex pairs.
37-
- `--disable-optimize-block-matrix` disables the optimization of block matrices.
38-
- `--disable-optimize-empty` disables the optimization for empty matrices.
39-
- `--disable-lazy-add` disables lazy addition optimization.
40-
- `--disable-optimize-format` disables optimization of matrix formats.
41-
42-
### Example
43-
44-
To solve the CFL-R problem using an incremental algorithm with a 60-second time limit for
45-
[indexed_tree.g](../test/pocr_data/indexed_an_bn/indexed_tree.g) and
46-
[an_bn_indexed.cnf](../test/pocr_data/indexed_an_bn/an_bn_indexed.cnf) and get results in
47-
[results.txt](../results.txt) execute:
48-
49-
```bash
50-
cd .. # Should be run from CFPQ_PyAlgo project root directory
51-
python3 -m cfpq_cli.run_all_pairs_cflr \
52-
IncrementalAllPairsCFLReachabilityMatrix \
53-
test/pocr_data/indexed_an_bn/indexed_tree.g \
54-
test/pocr_data/indexed_an_bn/an_bn_indexed.cnf \
55-
--time-limit 60 \
56-
--out results.txt
57-
```
58-
59-
### Grammar Format
60-
61-
The grammar file should be formatted with each production rule on a separate line, adhering to the following schema:
62-
63-
```
64-
<LEFT_SYMBOL> [RIGHT_SYMBOL_1] [RIGHT_SYMBOL_2]
65-
```
66-
67-
- `<LEFT_SYMBOL>`: the symbol on the left-hand side of a production rule.
68-
- `<RIGHT_SYMBOL_1>` and `<RIGHT_SYMBOL_2>`: the symbols on the right-hand side of the production rule, each of them is optional.
69-
- The symbols must be separated by whitespace.
70-
- The last two line specify the start symbol in the format
71-
```
72-
Count:
73-
<START_SYMBOL>
74-
```
75-
76-
#### Example
77-
```
78-
S AS_i b_i
79-
AS_i a_i S
80-
S c
81-
82-
Count:
83-
S
84-
```
85-
86-
### Graph Format
87-
88-
The graph file should represent edges using the format:
89-
90-
```
91-
<EDGE_SOURCE> <EDGE_DESTINATION> <EDGE_LABEL> [LABEL_INDEX]
92-
```
93-
94-
- `<EDGE_SOURCE>` and `<EDGE_DESTINATION>`: specify the source and destination nodes of an edge.
95-
- `<EDGE_LABEL>`: the label associated with the edge.
96-
- `[LABEL_INDEX]`: an optional index for labels with subscripts, indicating the subscript value.
97-
- The symbols must be separated by whitespace
98-
- Labels with subscripts must end with "\_i". For example, an edge $1 \xrightarrow{x_10} 2$ is denoted by `1 2 x_i 10`.
99-
100-
#### Example
101-
```
102-
1 2 a_i 1
103-
2 3 b_i 1
104-
2 4 b_i 2
105-
1 5 c
106-
```
7+
For more details, refer to [docs/cli.md](../docs/cli.md).

‎cfpq_eval/README.md

Lines changed: 4 additions & 96 deletions
Original file line numberDiff line numberDiff line change
@@ -1,98 +1,6 @@
1-
# CFPQ Evaluation
1+
## CFPQ Evaluator
22

3-
The `cfpq_eval` module evaluates performance of various CFPQ solvers,
4-
integrating with both CFPQ_PyAlgo itself and third-party tools.
3+
The `cfpq_eval` module is responsible for evaluating performance of various Context-Free Path Querying (CFPQ) solvers,
4+
including both CFPQ_PyAlgo itself and third-party tools.
55

6-
## Setting up the environment
7-
8-
Build and run a Docker container for evaluation using [Dockerfile-all-tools](../Dockerfile-all-tools).
9-
10-
Build Docker image:
11-
```bash
12-
cd .. # Should be run from CFPQ_PyAlgo project root directory
13-
14-
# Load base image
15-
wget -O pearl.tar.gz https://figshare.com/ndownloader/files/42214812
16-
docker load --input pearl.tar.gz
17-
rm pearl.tar.gz
18-
19-
# Build eval image
20-
docker build -f Dockerfile-all-tools -t cfpq/py_algo_eval .
21-
```
22-
23-
Run Docker container:
24-
```bash
25-
docker run -it cfpq/py_algo_eval bash
26-
```
27-
28-
## Running the Script
29-
30-
For detailed information on evaluation script options, execute the following command:
31-
32-
```bash
33-
cd .. # Should be run from CFPQ_PyAlgo project root directory
34-
python3 -m cfpq_eval.eval_all_pairs_cflr --help
35-
```
36-
37-
The basic command usage is as follows:
38-
39-
```
40-
# Should be run in cfpq_eval Docker container
41-
python3 -m cfpq_eval.eval_all_pairs_cflr algo_config.csv data_config.csv results_path [--rounds ROUNDS] [--timeout TIMEOUT]
42-
```
43-
44-
- `algo_config.csv` specifies algorithm configurations (e.g. `configs/algo/fast_matrix_cfpq.csv`).
45-
- `data_config.csv` specifies the dataset (e.g. `configs/data/small_examples.csv`).
46-
- `results_path` specifies path for saving raw results.
47-
- `--rounds` sets run times per config (default is 1).
48-
- `--timeout` limits each configuration's execution time in seconds (optional).
49-
50-
## Configuration Files
51-
52-
### Premade Configurations
53-
54-
The CFPQ_eval Docker image includes premade configurations located in the `/py_algo/configs` folder.
55-
### Algorithm Configuration
56-
57-
The `algo_config.csv` outlines algorithms and settings. Supported algorithms:
58-
59-
- `IncrementalAllPairsCFLReachabilityMatrix`
60-
- `NonIncrementalAllPairsCFLReachabilityMatrix`
61-
- `pocr`
62-
- `pearl`
63-
- `graspan`
64-
- `gigascale`
65-
66-
For Matrix-based algorithms options described in [cfpq_cli/README](../cfpq_cli/README.md)
67-
can be used to alter the behaviour.
68-
69-
#### Example
70-
71-
```
72-
algo_name,algo_settings
73-
"Matrix (some optimizations disabled)",IncrementalAllPairsCFLReachabilityMatrix --disable-optimize-empty --disable-lazy-add
74-
"pocr",pocr
75-
```
76-
77-
### Data Configuration
78-
79-
The `data_config.csv` pairs graph and grammar files,
80-
referenced files should be in format described in [cfpq_cli/README](../cfpq_cli/README.md).
81-
82-
#### Example
83-
84-
```
85-
graph_path,grammar_path
86-
data/graphs/aa/leela.g,data/grammars/aa.cnf
87-
data/graphs/java/eclipse.g,data/grammars/java_points_to.cnf
88-
```
89-
90-
## Interpreting Results
91-
92-
Raw data is saved to `results_path`, while quick summary including mean execution time,
93-
memory usage, and output size are rendered in standard output stream.
94-
95-
## Custom Tools Integration
96-
97-
Custom CFPQ solvers can be evaluated by implementing `AllPairsCflrToolRunner` interface
98-
and updating `run_appropriate_all_pairs_cflr_tool()` function.
6+
For more details, refer to [docs/eval.md](../docs/eval.md).

‎docs/README.md

Lines changed: 10 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,10 @@
1+
## CFPQ_PyAlgo Documentation
2+
3+
This folder contains documentation for CFPQ_PyAlgo project.
4+
5+
## Contents
6+
7+
- [Solver installation](install.md)
8+
- [Solver usage](cli.md)
9+
- [Performance evaluator installation](eval_install.md)
10+
- [Performance evaluator usage](eval.md)

0 commit comments

Comments
 (0)