Skip to content
This repository was archived by the owner on Dec 29, 2022. It is now read-only.

Commit 131168b

Browse files
authored
Merge pull request #1766 from JmPotato/update_architecture_md
Update the architecture.md file with the latest repo and code links
2 parents 3df7438 + 122ee0e commit 131168b

File tree

1 file changed

+30
-26
lines changed

1 file changed

+30
-26
lines changed

architecture.md

Lines changed: 30 additions & 26 deletions
Original file line numberDiff line numberDiff line change
@@ -5,7 +5,7 @@ In addition to the document below, an architecture overview can be found at @nrc
55

66
Here we aim to explain in-depth how RLS obtains the underlying data to drive its indexing features as the context for the upcoming IDE planning and discussion at the 2019 Rust All-Hands.
77

8-
Also the [rust-analyzer](https://github.com/rust-analyzer/rust-analyzer/blob/e0d8c86563b72e5414cf10fe16da5e88201447e2/guide.md) guide is a great resource as it covers a lot of common ground.
8+
Also the [rust-analyzer](https://github.com/rust-analyzer/rust-analyzer/blob/1d53f695f0408f47c5cce5cefa471eb0e86b0db7/docs/dev/guide.md) guide is a great resource as it covers a lot of common ground.
99

1010
## High-level overview
1111

@@ -27,16 +27,17 @@ once, where we dump and cache the resulting data into a JSON file, which only ne
2727
## Information flow (in-depth)
2828
The current flow is as follows:
2929
```
30-
rustc -> librustc_save_analysis -> rls_data -> rls_analysis -> rls
30+
rustc -> rustc_save_analysis -> rls_data -> rls_analysis -> rls
3131
```
3232

33-
### [librustc_save_analysis](https://github.com/rust-lang/rust/tree/master/src/librustc_save_analysis)
33+
### [rustc_save_analysis](https://github.com/rust-lang/rust/tree/master/compiler/rustc_save_analysis)
3434

35-
The Rust compiler includes the [`librustc_save_analysis`](https://github.com/rust-lang/rust/tree/master/src/librustc_save_analysis) crate, which allows to dump the knowledge about the currently compiled crate. The main entry point is [`process_crate`](https://github.com/rust-lang/rust/blob/7164a9f151a56316a382d8bc2b15ccf373e129ca/src/librustc_save_analysis/lib.rs#L1119), which walks the post-macro-expansion AST and [saves](https://github.com/rust-lang/rust/blob/7164a9f151a56316a382d8bc2b15ccf373e129ca/src/librustc_save_analysis/lib.rs#L1146) the collected knowledge either by [dumping to a JSON file](https://github.com/rust-lang/rust/blob/7164a9f151a56316a382d8bc2b15ccf373e129ca/src/librustc_save_analysis/lib.rs#L1074-L1090) or by [calling back with resulting data structure](https://github.com/rust-lang/rust/blob/7164a9f151a56316a382d8bc2b15ccf373e129ca/src/librustc_save_analysis/lib.rs#L1092-L1117).
35+
The Rust compiler includes the [`rustc_save_analysis`](https://github.com/rust-lang/rust/tree/master/compiler/rustc_save_analysis) crate, which allows to dump the knowledge about the currently compiled crate. The main entry point is [`process_crate`](https://github.com/rust-lang/rust/blob/e08d5693609a659e45025b8ea4dbd9efa342fa68/compiler/rustc_save_analysis/src/lib.rs#L978), which walks the post-macro-expansion AST and [saves](https://github.com/rust-lang/rust/blob/e08d5693609a659e45025b8ea4dbd9efa342fa68/compiler/rustc_save_analysis/src/lib.rs#L1011) the collected knowledge either by [dumping to a JSON file](https://github.com/rust-lang/rust/blob/e08d5693609a659e45025b8ea4dbd9efa342fa68/compiler/rustc_save_analysis/src/lib.rs#L953-L965) or by [calling back with resulting data structure](https://github.com/rust-lang/rust/blob/e08d5693609a659e45025b8ea4dbd9efa342fa68/compiler/rustc_save_analysis/src/lib.rs#L967-L976).
3636

37-
### [rls_data](https://github.com/rust-dev-tools/rls-data)
37+
### [rls_data](https://github.com/rust-lang/rls/tree/master/rls-data)
38+
39+
As mentioned previously, the returned data structure is [`rls_data::Analysis`](https://github.com/rust-lang/rls/blob/3df74381f37617ec800537c11fb0c3130f5f3616/rls-data/src/lib.rs#L11-L26) inside the [`rls_data`](https://github.com/rust-lang/rls/tree/master/rls-data) crate:
3840

39-
As mentioned previously, the returned data structure is [`rls_data::Analysis`](https://github.com/rust-dev-tools/rls-data/blob/9edbe8b4947c10ef670c4723be375c6944cab640/src/lib.rs#L30-L48) inside the [`rls_data`](https://github.com/rust-dev-tools/rls-data) crate:
4041
```rust
4142
/// Basically a type alias, we refer to nodes with HIR ids.
4243
/// All of the below nodes are either identified by or refer to those IDs.
@@ -70,31 +71,31 @@ pub struct Analysis {
7071
}
7172
```
7273

73-
### [rls_analysis](https://github.com/rust-dev-tools/rls-analysis)
74+
### [rls_analysis](https://github.com/rust-lang/rls/tree/master/rls-analysis)
7475

75-
This [crate](https://github.com/rust-dev-tools/rls-analysis) is responsible for loading and stitching multiple of
76+
This [crate](https://github.com/rust-lang/rls/tree/master/rls-analysis) is responsible for loading and stitching multiple of
7677
the `rls_data::Analysis` data structures into a single, coherent interface.
7778

7879
Whereas `rls_data` format can be considered an implementation detail that might
7980
change, this crate aims to provide a 'stable' API.
8081

8182
Another reason behind that is that each of those structures contains data centric
82-
to the crate that was being compiled - this [lowering](https://github.com/rust-dev-tools/rls-analysis/blob/bd82c9b38b56e53bbfb199569a32b392056964fd/src/lowering.rs#L167)
83+
to the crate that was being compiled - this [lowering](https://github.com/rust-lang/rls/blob/3df74381f37617ec800537c11fb0c3130f5f3616/rls-analysis/src/lowering.rs#L161)
8384
cross-references the data
8485
and indexes it, resulting in a database spanning multiple crates that can be
8586
queried like 'across known crates, find all references to a definition at a
8687
given span' or similarly.
8788

8889
We are capable of updating the index with new crate data. Whenever we encounter
89-
a new crate, we [record and translate](https://github.com/rust-dev-tools/rls-analysis/blob/bd82c9b38b56e53bbfb199569a32b392056964fd/src/lowering.rs#L131-L154)
90+
a new crate, we [record and translate](https://github.com/rust-lang/rls/blob/3df74381f37617ec800537c11fb0c3130f5f3616/rls-analysis/src/lowering.rs#L122-L144)
9091
the crate id into our database-wide crate id mapping.
9192

9293
However, if data for an already lowered crate is loaded again, we simply
9394
replace the definitions for a given crate and re-index.
9495

9596
One interesting edge case is when we lower data for crates having the same name, such
9697
as binary and `#[cfg(test)]`-compiled version of it. We need to ensure we lower a given definition
97-
[only once](https://github.com/rust-dev-tools/rls-analysis/blob/bd82c9b38b56e53bbfb199569a32b392056964fd/src/lowering.rs#L258-L263)
98+
[only once](https://github.com/rust-lang/rls/blob/3df74381f37617ec800537c11fb0c3130f5f3616/rls-analysis/src/lowering.rs#L271-L276)
9899
, even if it technically is repeated across multiple crates.
99100

100101
### rls
@@ -104,6 +105,7 @@ the data - that happens inside the RLS.
104105

105106
In general, apart from being an LSP server, the RLS is also concerned with
106107
build orchestration, and coordination of other components, such as
108+
107109
* Racer for autocompletion
108110
* Cargo for project layout detection and initial build coordination
109111
* internal virtual file system (VFS) for handling in-memory text buffers,
@@ -113,7 +115,7 @@ build orchestration, and coordination of other components, such as
113115

114116
After doing initial compilation with Cargo, we cache a subgraph of the inter-crate
115117
dependency graph along with compilation invocations and input files for the
116-
crates we're interested in (inside the [primary or path-based packages](https://github.com/rust-lang/rls/blob/d7c2eb8b641ae7e6d7145c268249f28efcf5467c/src/build/cargo.rs#L376-L381))
118+
crates we're interested in (inside the [primary or path-based packages](https://github.com/rust-lang/rls/blob/3df74381f37617ec800537c11fb0c3130f5f3616/rls/src/build/cargo.rs#L358-L363))
117119
which we later rerun manually.
118120

119121
In our case Cargo is configured to use a separate target directory
@@ -131,31 +133,33 @@ proc macros initially and when needed (e.g. by modifying a file causing
131133

132134
## Build scheduling
133135

134-
On every relevant file change we [mark files as dirty](https://github.com/rust-lang/rls/blob/d7c2eb8b641ae7e6d7145c268249f28efcf5467c/src/actions/notifications.rs#L131) and schedule a normal build.
136+
On every relevant file change we [mark files as dirty](https://github.com/rust-lang/rls/blob/3df74381f37617ec800537c11fb0c3130f5f3616/rls/src/actions/notifications.rs#L113) and schedule a normal build.
137+
138+
We currently discern two [build priorities](https://github.com/rust-lang/rls/blob/3df74381f37617ec800537c11fb0c3130f5f3616/rls/src/build/mod.rs#L114-L124):
135139

136-
We currently discern two [build priorities](https://github.com/rust-lang/rls/blob/67bce0bdcf2db1d3c05bb1a3d87df9e66eaec7db/src/build/mod.rs#L124-L133):
137140
* Normal
138141
* Cargo
139142

140143
The latter is scheduled whenever a change happened that can impact entire
141144
project. This includes:
142-
* [initial build](https://github.com/rust-lang/rls/blob/67f2a86c13a34dcb231436c2f1db8900fece3c09/src/actions/mod.rs#L331)
143-
* [configuration change](https://github.com/rust-lang/rls/blob/67f2a86c13a34dcb231436c2f1db8900fece3c09/src/actions/notifications.rs#L207)
145+
146+
* [initial build](https://github.com/rust-lang/rls/blob/3df74381f37617ec800537c11fb0c3130f5f3616/rls/src/actions/mod.rs#L328)
147+
* [configuration change](https://github.com/rust-lang/rls/blob/3df74381f37617ec800537c11fb0c3130f5f3616/rls/src/actions/notifications.rs#L116)
144148
(can potentially build different set of packages)
145-
* [Cargo.toml change](https://github.com/rust-lang/rls/blob/67f2a86c13a34dcb231436c2f1db8900fece3c09/src/actions/notifications.rs#L273)
149+
* [Cargo.toml change](https://github.com/rust-lang/rls/blob/3df74381f37617ec800537c11fb0c3130f5f3616/rls/src/actions/notifications.rs#L264)
146150
(ditto)
147-
* [build directory](https://github.com/rust-lang/rls/blob/67bce0bdcf2db1d3c05bb1a3d87df9e66eaec7db/src/build/mod.rs#L479-L486) changed
148-
* [modified file](https://github.com/rust-lang/rls/blob/d6570bc62575e03412340e55620cbf24fe59f772/src/build/cargo_plan.rs#L379-L383) in a package we didn't build
149-
* [build.rs](https://github.com/rust-lang/rls/blob/d6570bc62575e03412340e55620cbf24fe59f772/src/build/cargo_plan.rs#L392-L396) modification
151+
* [build directory](https://github.com/rust-lang/rls/blob/3df74381f37617ec800537c11fb0c3130f5f3616/rls/src/build/mod.rs#L468-L472) changed
152+
* [modified file](https://github.com/rust-lang/rls/blob/3df74381f37617ec800537c11fb0c3130f5f3616/rls/src/build/cargo_plan.rs#L350-L354) in a package we didn't build
153+
* [build.rs](https://github.com/rust-lang/rls/blob/3df74381f37617ec800537c11fb0c3130f5f3616/rls/src/build/cargo_plan.rs#L359-L360) modification
150154

151155
On a normal build, we map from dirty files to dirty crates, sort those
152156
topologically and run rustc in-process for each crate ourselves.
153157
With each compilation in-process we directly
154-
[receive `rls_data::Analysis`](https://github.com/rust-lang/rls/blob/41bc0bf70bbbc8661f0c7f9cef700be5e105a926/src/build/rustc.rs#L334-L347)
158+
[receive `rls_data::Analysis`](https://github.com/rust-lang/rls/blob/3df74381f37617ec800537c11fb0c3130f5f3616/rls/src/build/rustc.rs#L293-L305)
155159
in a callback,
156-
[mark corresponding files as built](https://github.com/rust-lang/rls/blob/67bce0bdcf2db1d3c05bb1a3d87df9e66eaec7db/src/build/mod.rs#L493-L505)
160+
[mark corresponding files as built](https://github.com/rust-lang/rls/blob/3df74381f37617ec800537c11fb0c3130f5f3616/rls/src/build/mod.rs#L478-L490)
157161
and finally
158-
[update our analysis database](https://github.com/rust-lang/rls/blob/d7c2eb8b641ae7e6d7145c268249f28efcf5467c/src/actions/post_build.rs#L208-L223)
162+
[update our analysis database](https://github.com/rust-lang/rls/blob/3df74381f37617ec800537c11fb0c3130f5f3616/rls/src/actions/post_build.rs#L180-L184)
159163
with currently built data for each rebuilt crate.
160164

161165
If there are still files that are modified after we scheduled a build (user kept
@@ -173,10 +177,10 @@ JSON save-analysis files for our non-path dependencies.
173177
### VFS
174178

175179
To allow running analysis on unsaved in-memory text buffers, we use the
176-
[`rls-vfs`](https://github.com/rust-dev-tools/rls-vfs)
180+
[`rls-vfs`](https://github.com/rust-lang/rls/tree/master/rls-vfs)
177181
crate to act as our virtual file system.
178182

179-
The Rust compiler supports using custom file providers via [`FileLoader`](https://github.com/rust-lang/rust/blob/79d8a0fcefa5134db2a94739b1d18daa01fc6e9f/src/libsyntax/source_map.rs#L58-L68) trait, which [we use](https://github.com/rust-lang/rls/blob/67bce0bdcf2db1d3c05bb1a3d87df9e66eaec7db/src/build/rustc.rs#L385-L402).
183+
The Rust compiler supports using custom file providers via [`FileLoader`](https://github.com/rust-lang/rust/blame/f19851069efd6ee1fe899a469f08ad2d66e76050/compiler/rustc_span/src/source_map.rs#L98-L105) trait, which [we use](https://github.com/rust-lang/rls/blob/3df74381f37617ec800537c11fb0c3130f5f3616/rls/src/build/rustc.rs#L355-L367).
180184

181185
It delegates to the real file system
182-
whenever there are no buffered changes to a file but serves the unsaved buffers [from the VFS](https://github.com/rust-lang/rls/blob/67bce0bdcf2db1d3c05bb1a3d87df9e66eaec7db/src/build/rustc.rs#L79) otherwise.
186+
whenever there are no buffered changes to a file but serves the unsaved buffers [from the VFS](https://github.com/rust-lang/rls/blob/3df74381f37617ec800537c11fb0c3130f5f3616/rls/src/build/rustc.rs#L54) otherwise.

0 commit comments

Comments
 (0)