Skip to content

Commit fc1f174

Browse files
vchuravyvchuravy
committed
add a tutorial on preferences
1 parent 008df69 commit fc1f174

File tree

4 files changed

+203
-0
lines changed

4 files changed

+203
-0
lines changed

‎_layout/masthead.html

Lines changed: 2 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -5,6 +5,8 @@
55
<a class="site-title" href="/">JuliaParallel </a>
66
<ul class="visible-links">
77
<li class="masthead__menu-item"><a href="/news/">News</a></li>
8+
<li class="masthead__menu-item"><a href="/teaching/">Teaching</a></li>
9+
<li class="masthead__menu-item"><a href="/tutorials/">Tutorials</a></li>
810
<li class="masthead__menu-item"><a href="https://github.com/JuliaParallel">Github</a></li>
911
</ul>
1012
<button class="greedy-nav__toggle hidden" type="button">

‎teaching/index.md

Lines changed: 10 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,10 @@
1+
@def title = "Teaching"
2+
3+
# Tutorials
4+
5+
Here you can find various tutorials given over the years on the usage of
6+
the JuliaParallel software stack.
7+
8+
| Date | Location | Title | Video | Material | Speaker |
9+
| ---- | -------- | ----- | ----- | -------- | ------- |
10+
| | | | | | |

‎tutorials/index.md

Lines changed: 12 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,12 @@
1+
@def title = "Tutorials"
2+
3+
# Tutorials
4+
5+
## MPI.jl
6+
7+
1. Examples
8+
1. [Hello world](https://juliaparallel.org/MPI.jl/latest/examples/01-hello/)
9+
10+
## HPC Site configuration
11+
12+
1. [Using Preferences](preferences)

‎tutorials/preferences.md

Lines changed: 179 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,179 @@
1+
@def title = "Preferences"
2+
@def hascode = true
3+
4+
# Preferences
5+
6+
Since Julia 1.6 [`Preferences.jl`](https://github.com/JuliaPackaging/Preferences.jl)
7+
allows packages to declare options/preferences that the user can set on a per project
8+
level.
9+
10+
Preferences are read and written from Julia with the Preferences.jl package.
11+
12+
```julia
13+
module MyPackage
14+
15+
using Preferences
16+
17+
const favorite_icecream = @load_preference("icecream", "vanilla")
18+
19+
function change_mind!(flavor)
20+
@set_preferences!("icecream"=>flavor)
21+
end
22+
23+
end
24+
```
25+
When a preference is set it is stored in a file called `LocalPreferences.toml`
26+
next to your current `Project.toml`.
27+
28+
#### LocalPreferences.toml
29+
30+
```toml
31+
[MyPackage]
32+
icecream = "chocolate"
33+
```
34+
35+
Note how the `LocalPreferences.toml` is scoped by package name, and thus each
36+
package has it's own namespace.
37+
38+
## Setting preferences of dependencies
39+
40+
A common use-case of preferences is to configure the library path of libraries
41+
provided through `_jll` packages.
42+
43+
As an example let's add `UCX.jl` as a dependency to a package.
44+
45+
```sh
46+
> cat Project.toml
47+
[deps]
48+
Preferences = "21216c6a-2e73-6563-6e65-726566657250"
49+
UCX = "6b349878-927d-5bd5-ab28-bc3aa4175a33"
50+
```
51+
52+
and we can see that it currently uses the `libucp.so` from `UCX_jll`.
53+
54+
```julia-repl
55+
julia> using UCX
56+
julia> UCX.API.libucp
57+
/home/vchuravy/.julia/artifacts/33301dce3561b1e57216ae5a4fc16d847e066a1d/lib/libucp.so
58+
```
59+
60+
Now if I want to change the library used we have two options. Firstly the manual
61+
one where we directly manipulate the `LocalPreferences.toml`, and secondly using
62+
Prefrences `set_preference(UCX.UCX_jll, "libucp_path"=>"...")`.
63+
64+
### Manual approach
65+
66+
The `LocalPreferences.toml` we create next to our `Project.toml` will look like:
67+
68+
```sh
69+
> cat LocalPreferences.toml
70+
[UCX_jll]
71+
libucp_path="/home/vchuravy/builds/ucx/lib/libucp.so"
72+
```
73+
74+
Running the same commands as before:
75+
76+
```julia-repl
77+
julia> using UCX
78+
julia> UCX.API.libucp
79+
/home/vchuravy/.julia/artifacts/33301dce3561b1e57216ae5a4fc16d847e066a1d/lib/libucp.so
80+
```
81+
82+
We see that nothing has changed. Why? Julia internally referers to packages by UUIDs
83+
so if you take another look at the `Project.toml` you can see that in `[deps]` we
84+
declare a mapping from name to UUID, and the `LocalPreferences.toml` needs exactly
85+
this mapping to know what package owns the namespace. We could add `UCX_jll` to our
86+
`[deps]` section in our package but that would unecessarily polute our dependency set.
87+
Instead the right solution is to add `UCX_jll` to the `[extras]` section.
88+
89+
```sh
90+
> cat Project.toml
91+
[deps]
92+
Preferences = "21216c6a-2e73-6563-6e65-726566657250"
93+
UCX = "6b349878-927d-5bd5-ab28-bc3aa4175a33"
94+
95+
[extras]
96+
UCX_jll = "16e4e860-d6b8-5056-a518-93e88b6392ae"
97+
```
98+
99+
Loading UCX now delightfully fails since of course we haven't done the work of placing
100+
a library in that location.
101+
102+
```julia-repl
103+
julia> using UCX
104+
ERROR: InitError: could not load library "/home/vchuravy/builds/ucx/lib/libucp.so"
105+
/home/vchuravy/builds/ucx/lib/libucp.so: cannot open shared object file: No such file or directory
106+
```
107+
108+
Keep this in mind if you are ever wondering why settings in `LocalPreferences.toml`
109+
don't seem to take effect.
110+
111+
### The "proper" way
112+
113+
Let's reset our environment to the state it was before:
114+
115+
```sh
116+
> cat Project.toml
117+
[deps]
118+
Preferences = "21216c6a-2e73-6563-6e65-726566657250"
119+
UCX = "6b349878-927d-5bd5-ab28-bc3aa4175a33"
120+
```
121+
122+
```julia-repl
123+
julia> using UCX
124+
julia> @eval UCX using UCX_jll
125+
julia> using Preferences
126+
julia> set_preferences!(UCX.UCX_jll, "libucp_path" => "/home/vchuravy/builds/ucx/lib/libucp.so")
127+
julia> UCX.UCX_jll.libucp_path
128+
"/home/vchuravy/.julia/artifacts/33301dce3561b1e57216ae5a4fc16d847e066a1d/lib/libucp.so"
129+
```
130+
131+
As we can see the preference has not yet taken effect, and it will require a
132+
restart of Julia. This is because the preference lookup is cached during package
133+
precompilation.
134+
135+
Inspecting our `Project.toml` and `LocalPreferences.toml` we see that `Preferences.jl`
136+
automatically added `UCX_jll` to the `[extras]` section.
137+
138+
```sh
139+
> cat Project.toml
140+
[deps]
141+
Preferences = "21216c6a-2e73-6563-6e65-726566657250"
142+
UCX = "6b349878-927d-5bd5-ab28-bc3aa4175a33"
143+
144+
[extras]
145+
UCX_jll = "16e4e860-d6b8-5056-a518-93e88b6392ae"
146+
```
147+
148+
```sh
149+
> cat example/LocalPreferences.toml
150+
[UCX_jll]
151+
libucp_path = "/home/vchuravy/builds/ucx/lib/libucp.so"
152+
```
153+
154+
After restarting we see the expected behavior.
155+
156+
```julia-repl
157+
julia> using UCX
158+
ERROR: InitError: could not load library "/home/vchuravy/builds/ucx/lib/libucp.so"
159+
/home/vchuravy/builds/ucx/lib/libucp.so: cannot open shared object file: No such file or directory
160+
```
161+
162+
## Interactions with the `JULIA_LOAD_PATH`
163+
164+
Preferences are merged across the entire `JULIA_LOAD_PATH` to illustrate this
165+
let's create a new directory `preferences` and mv over our current `LocalPreferences.toml`.
166+
167+
```sh
168+
mkdir preferences
169+
mv example/LocalPreferences.toml preferences/
170+
printf "[extras]\nUCX_jll = \"16e4e860-d6b8-5056-a518-93e88b6392ae\"" > preferences/Project.toml
171+
```
172+
173+
and remove `UCX_jll` from `Project.toml` in the current directory.
174+
175+
```sh
176+
> JULIA_LOAD_PATH="@:./preferences" julia --project=.
177+
julia> using UCX
178+
ERROR: InitError: could not load library "/home/vchuravy/builds/ucx/lib/libucp.so"
179+
```

0 commit comments

Comments
 (0)