@@ -92,7 +92,7 @@ This work is being carried out by Well-Typed LLP thanks to investment from the
92
92
Sovereign Tech Fund. (For more information, read our [ blog post announcing the
93
93
project] ( https://www.well-typed.com/blog/2023/10/sovereign-tech-fund-invests-in-cabal/ ) .)
94
94
It has previously been discussed at [ Cabal issue #9292 ] ( https://github.com/haskell/cabal/issues/9292 ) ,
95
- and is now being [ discussed at this pull request] ( https://github.com/haskellfoundation/tech-proposals/pull/60 ) .
95
+ and [ tech-proposals pull request # 60 ] ( https://github.com/haskellfoundation/tech-proposals/pull/60 ) .
96
96
97
97
## Background
98
98
@@ -418,7 +418,7 @@ uniform mechanism.
418
418
419
419
## High-level design of ` build-type: Hooks `
420
420
421
- We propose to augment ` Cabal ` with a new build-type, ` Hooks ` .
421
+ We propose to augment ` Cabal ` with a new build-type, ` Hooks ` .
422
422
To implement a package with the ` Hooks ` build-type, the user needs to provide
423
423
a ` SetupHooks.hs ` file which specifies the hooks using a Haskell API.
424
424
@@ -544,15 +544,15 @@ while, in theory, a package using `build-type: Custom` can implement its `Setup`
544
544
script without depending on ` Cabal ` , we saw that this flexibility was unused in
545
545
practice, as ` Setup ` scripts end up being defined in terms of ` UserHooks ` .
546
546
This usage pattern incurs a corresponding dependency on the ` Cabal ` library in
547
- ` setup-depends ` , in much the same way as we propose here for ` Cabal-hooks ` .
547
+ ` setup-depends ` , in much the same way as we propose here for ` Cabal-hooks ` .
548
548
An additional benefit of the separate ` Cabal-hooks ` library is that it makes it
549
549
possible to evolve the Hooks API without requiring a version bump of the
550
550
` Cabal ` library.
551
551
552
552
In practice, we expect the initial versions of ` Cabal-hooks ` to mostly
553
553
re-export ` Cabal ` datatypes, as it is these types (such as ` LocalBuildInfo ` )
554
554
that get passed back-and-forth between the build system and the hooks in our
555
- current design (see e.g. [ § Configure hooks] ( #configure-hooks ) ).
555
+ current design (see e.g. [ § Configure hooks] ( #configure-hooks ) ).
556
556
This design choice does introduce some coupling between the versions of
557
557
` Cabal-hooks ` and ` Cabal ` (but see [ § Decoupling ` Cabal-hooks ` ] ( #decoupling-Cabal-hooks ) ).
558
558
At any rate, this design makes the situation no worse than with ` Custom `
@@ -561,7 +561,7 @@ using an older version of `Cabal-hooks`), but it gives more options to the build
561
561
e.g. where serialisation of hook inputs/outputs is used, the serialisation
562
562
format can be controlled by the build tool, and is not necessarily fixed by ` Cabal-hooks ` .
563
563
Indeed, we could imagine a build tool being compiled against multiple ` Cabal-hooks `
564
- versions.
564
+ versions.
565
565
The version compatibility problem exists in
566
566
` cabal-install ` already: even where communication happens via the ` Setup.hs `
567
567
command line interface, there is already a need for ` cabal-install ` to adapt to
@@ -726,7 +726,7 @@ From the build tool's perspective, the global configuration phase goes as follow
726
726
- Run the `preConfPackageHook` , which has the opportunity to modify the
727
727
initially decided global configuration (with a `BuildOptions ` that overrides
728
728
those stored in the passed in `LocalBuildConfig `, and `ConfiguredProgs ` that
729
- get added to the `ProgramDb `).
729
+ get added to the `ProgramDb `).
730
730
After this point, the `LocalBuildConfig ` can no longer be modified.
731
731
732
732
- Use the `LocalBuildConfig ` in order to perform the global package
@@ -828,7 +828,7 @@ from `Cabal`,making it less likely that an internal change in Cabal would end up
828
828
breaking the `Hooks ` defined by package authors. However , one would need to
829
829
ensure this interface is general enough in order to avoid locking out Hooks
830
830
authors, e. g. if `Cabal ` adds a new field to `Component ` without updating the
831
- corresponding `ComponentDiff ` type in order to make it modifiable by hook authors.
831
+ corresponding `ComponentDiff ` type in order to make it modifiable by hook authors.
832
832
If we end up with a design in which `Cabal `'s version of the `Component ` type
833
833
is necessarily separate from the type in the hooks API , we may want to reconsider
834
834
this alternative.
@@ -841,15 +841,15 @@ build hooks in the old `UserHooks`, but updated to the per-component world.
841
841
This included monolithic pre and post hooks for each component, plus the existing
842
842
" hooked pre-processors" abstraction. This had the advantage that it would be easy
843
843
for package authors to port their `Setup.hs` scripts to the new design, and it was
844
- a relatively minimal change in the Cabal codebase.
844
+ a relatively minimal change in the Cabal codebase.
845
845
Many Cabal contributors share a long term goal to move the Cabal design towards
846
846
one based on a build graph with fine- grained dependencies. From this perspective,
847
847
the critique was that the initial proposal was too conservative a change, and that
848
848
we should take this opportunity of making a significant API change to establish a
849
849
new API that would not hold back the move towards finer- grained dependencies.
850
850
Another critique was that the original `UserHooks ` design was somewhat ad- hoc,
851
851
since it used both monolithic hooks and hooked pre- processors to provide
852
- finer- grained dependencies for a modest subset of use cases.
852
+ finer- grained dependencies for a modest subset of use cases.
853
853
On the other hand, there is a very large design space for finer- grained
854
854
dependencies, and so picking a point in the design space is not simple.
855
855
Another disadvantage is that it will of course be more work for package authors
@@ -866,7 +866,7 @@ possible to build higher level patterns on top, using Haskell's usual powers of
866
866
abstraction to generate the lower level rules. Crucially , the design allows the
867
867
rules to be used across an IPC interface, which is necessary for build tools
868
868
like `cabal- install` or HLS to be able to interrogate and invoke them
869
- (see e. g. the future work discussed in [§ Hooks integration](# hooks- integration)).
869
+ (see e. g. the future work discussed in [§ Hooks integration](# hooks- integration)).
870
870
871
871
The full details of the design of pre- build hooks are provided in
872
872
[§ Pre - build hooks](# pre- build- hooks).
@@ -886,7 +886,7 @@ data BuildHooks
886
886
, postBuildComponentHook :: Maybe PostBuildComponentHook }
887
887
```
888
888
889
- Build hooks cannot change the configuration of the package.
889
+ Build hooks cannot change the configuration of the package.
890
890
There are deliberately no package- level build hooks, only component- level hooks.
891
891
This avoids introducing unnecessary synchronisation points when multiple
892
892
packages/ components are being built in parallel.
@@ -1026,7 +1026,7 @@ data TentativeRule = TentativeRule
1026
1026
That is, rules are specified by a function that takes in an environment
1027
1027
(which in practice consists of information known to ` Cabal ` after configuring,
1028
1028
e.g. ` LocalBuildInfo ` , ` ComponentLocalBuildInfo ` ) and returns an ` IO ` action
1029
- that computes a list of rules.
1029
+ that computes a list of rules.
1030
1030
1031
1031
### Proposed design of rules
1032
1032
@@ -1157,7 +1157,7 @@ declare
1157
1157
This design seems sufficient for common use cases, such as GHC's build system.
1158
1158
Indeed, as explained in [ Hadrian] ( #hadrian ) , its Make build system only required
1159
1159
second-layer expansion (i.e. ` $$$$ ` ) for rules that invoke ` ghc -M ` in some way,
1160
- not any further layers.
1160
+ not any further layers.
1161
1161
More generally, it is preferable to output a set of rules that are at a rather
1162
1162
low-level, so that these can be readily consumed by build tools, rather than
1163
1163
requiring the build tool to do additional work to resolve dependencies.
@@ -1172,7 +1172,7 @@ data RuleOutput = RuleOutput { outputOfRule :: RuleId, outputIndex :: Int }
1172
1172
```
1173
1173
1174
1174
In particular, a rule that depends on the output of another rule must depend
1175
- directly on the rule, rather than the file that that rule outputs.
1175
+ directly on the rule, rather than the file that that rule outputs.
1176
1176
This ensures that dependencies are resolved upfront rather than when running
1177
1177
the rules. This ensures that any complexity in the structure of the rules exists
1178
1178
within the program generating the rules rather than in the build tool consuming
@@ -1207,7 +1207,7 @@ type Location = (FilePath, FilePath)
1207
1207
That is, each rule can be thought of as a pure function that takes in the
1208
1208
contents of the files at the input locations (the `dependencies` of the rule),
1209
1209
and outputs the contents of the files at the output locations (the `results`
1210
- of the rule).
1210
+ of the rule).
1211
1211
The logic that computes all pre- build rules is responsible for computing such
1212
1212
resolved locations, for example by searching the Cabal search directories.
1213
1213
However , there are certain restrictions on the filepaths used for results of
@@ -1228,7 +1228,7 @@ one would thus:
1228
1228
the input/ output locations and additional flags).
1229
1229
1230
1230
This results in one rule for each `. y` file, which will get re- run whenever the
1231
- associated `. y` file is modified.
1231
+ associated `. y` file is modified.
1232
1232
The rules need to be re- computed whenever a `. y` file gets added/ removed, or when
1233
1233
a `. hs` file with the same module name as a `. y` file gets added/ removed; we can
1234
1234
declare this by using the `MonitorFileOrDir ` functionality.
@@ -1301,7 +1301,7 @@ See [§ API overview](#api-overview) for an illustration of such an implementati
1301
1301
This design means that we ** do not ** re- run the entire computation of rules
1302
1302
each time a `. chs` file is modified. Instead , we re- run the dependency
1303
1303
computation of the modified `. chs` file (as its imports list may have changed),
1304
- which allows us to update the build graph.
1304
+ which allows us to update the build graph.
1305
1305
This ensures the dependencies of this rule remain up to date, ensuring correct
1306
1306
recompilation checking. Without this mechanism for declaring additional dynamic
1307
1307
dependencies, we would be forced to re- run the entire computation of rules each
@@ -1376,7 +1376,7 @@ Justification:
1376
1376
mechanism for which there isn't a one- to- one mapping between modules
1377
1377
declared in the `. cabal` file and source files on disk.
1378
1378
- (O2 ) is clear: if we change the environment, we need to re- compute
1379
- the rules (as the rules are specified by a function from an environment).
1379
+ the rules (as the rules are specified by a function from an environment).
1380
1380
Note that this covers the event of the package configuration changing
1381
1381
(e. g. after `cabal configure` has been re- run).
1382
1382
- (N , S1 ) are clear.
@@ -1397,7 +1397,7 @@ registerRule :: ShortText -> Rule -> RulesM RuleId
1397
1397
```
1398
1398
1399
1399
The `ShortText ` argument is a user- given name for the rule. Different rules
1400
- defined within a package are required to have different name.
1400
+ defined within a package are required to have different name.
1401
1401
These `RuleId `s are used by the build system to determine when a rule needs to
1402
1402
be re- run. This means that users will in practice want to ensure persistence of
1403
1403
rules names across computations of rules. For example, if two successive
@@ -1406,7 +1406,7 @@ registered using the same name; not doing so will cause the rule to necessarily
1406
1406
be re- run the second time around, as described in [§ Rule demand](# rule- demand).
1407
1407
1408
1408
Note that rules do not know their own names: instead, one must register rules
1409
- with the API , which returns identifiers which are opaque to the user.
1409
+ with the API , which returns identifiers which are opaque to the user.
1410
1410
This leads to a more declarative and functional style for package authors
1411
1411
declaring pre- build rules. By having the identifier partly determined by the user
1412
1412
(in the form of the `ShortText ` arguments), we also ensure that these identifiers
@@ -1426,7 +1426,7 @@ addRuleMonitors :: [ MonitorFileOrDir ] -> RulesM ()
1426
1426
```
1427
1427
1428
1428
where `MonitorFileOrDir ` is some datatype, such as the one that exists in
1429
- `cabal- install` today, that specifies what one wants to monitor.
1429
+ `cabal- install` today, that specifies what one wants to monitor.
1430
1430
Specifically , `MonitorFileOrDir ` should at least support monitoring:
1431
1431
1432
1432
1 . The existence of a file or directory.
@@ -1532,9 +1532,9 @@ c2HsRules buildEnvt = mdo
1532
1532
Note how we use the ` static ` keyword in the definition of ` c2HsPreBuildRules ` .
1533
1533
Here, the Hooks API uses static pointers in order to tag rules by the package that
1534
1534
defines them, in order to allow combining the ` Rules ` declared by two different
1535
- libraries, as described in [ § Composing ` SetupHooks ` ] ( #composing-setuphooks ) .
1535
+ libraries, as described in [ § Composing ` SetupHooks ` ] ( #composing-setuphooks ) .
1536
1536
The user-provided names for rules (` "r1" ` , ` "r2" ` , ` "r3" ` above) are expected
1537
- to be unique (within the scope of the label).
1537
+ to be unique (within the scope of the label).
1538
1538
(NB: we don't use ` static ` for each individual identifier, as these are often
1539
1539
dynamically generated based on the result of an ` IO ` action, as above.)
1540
1540
@@ -1693,7 +1693,7 @@ data PreConfPackageOutputs
1693
1693
1694
1694
The configured programs returned by the package- wide pre- configure hook will
1695
1695
then be used to extend `Cabal `'s `ProgramDb `, which will then get stored in
1696
- `Cabal `'s `LocalBuildInfo ` datatype and passed to subsequent hooks.
1696
+ `Cabal `'s `LocalBuildInfo ` datatype and passed to subsequent hooks.
1697
1697
1698
1698
Note that we require hook authors configure the programs themselves (using
1699
1699
functions provided by the hooks API ). This is justified by the fact that
@@ -1768,9 +1768,9 @@ second.
1768
1768
1769
1769
Instead of ` Cabal-hooks ` re-exporting datatypes from ` Cabal ` , one could imagine
1770
1770
defining datatypes in ` Cabal-hooks ` instead; then ` Cabal-hooks ` would not depend
1771
- on ` Cabal ` .
1771
+ on ` Cabal ` .
1772
1772
This would completely encapsulate the Hooks API, which would no longer be tied
1773
- to a particular ` Cabal ` version.
1773
+ to a particular ` Cabal ` version.
1774
1774
Here are two conceivable ways in which this change might then impact ` Cabal ` :
1775
1775
1776
1776
1 . ` Cabal ` itself depends on ` Cabal-hooks ` . This is attractive from a
@@ -1898,7 +1898,7 @@ a rule which generates the dependency of the second rule.
1898
1898
1899
1899
In the first example, the dependency between the rules is specified directly
1900
1900
in the code, while in the second, it remains implicit (because a dependency of
1901
- the second rule matches an output of the first rule).
1901
+ the second rule matches an output of the first rule).
1902
1902
The first style is more functional:
1903
1903
1904
1904
- the data flow of the computation of rules reflects the data flow of the
@@ -1911,7 +1911,7 @@ The first style is more functional:
1911
1911
1912
1912
As noted in [ Pre-build hooks] ( #pre-build-hooks ) , rules are described at a
1913
1913
low-level with explicit inputs and outputs. For example, this framework does
1914
- not include "rule patterns" (generating ` *.hs ` from ` *.y ` ).
1914
+ not include "rule patterns" (generating ` *.hs ` from ` *.y ` ).
1915
1915
Moreover, filepaths are specified entirely explicitly: the rules themselves
1916
1916
are responsible for searching for input files in the input directory structure.
1917
1917
@@ -1927,13 +1927,13 @@ framework (e.g. there would be different types for locations of dependencies,
1927
1927
and for locations of results, before they get resolved by the build system).
1928
1928
We opted to keep the rules API as low-level as possible, following the approach
1929
1929
taken by the [ ` ninja ` build system] ( #ninja ) , which is designed around a
1930
- low-level syntax of rules being generated by a higher-level framework.
1930
+ low-level syntax of rules being generated by a higher-level framework.
1931
1931
1932
1932
### Making other hooks fine-grained
1933
1933
1934
1934
Note that we do not currently propose to use the same design of fine-grained
1935
1935
rules for other hooks, e.g. the hooks into the configure phase or the install
1936
- hooks.
1936
+ hooks.
1937
1937
The downside of this choice is that we do not track fine-grained dependency
1938
1938
information that would let us know when to re-run these hooks.
1939
1939
However, it is not clear that there is much demand for it to do so. Thus it may
@@ -2044,13 +2044,13 @@ via IPC:
2044
2044
- by emulating the classic ` Setup.hs ` CLI,
2045
2045
- in "one shot" CLI style: build ` SetupHooks.hs ` against a stub executable
2046
2046
that implements a CLI to expose all the hooks in a simple "invoke then terminate"
2047
- way, i.e. not as a long running process.
2047
+ way, i.e. not as a long running process.
2048
2048
This choice might be appropriate for non-interactive CLI tools like
2049
2049
` cabal-install ` or ` stack ` .
2050
2050
- in interactive/server style: build ` SetupHooks.hs ` against a stub executable
2051
2051
that communicates over pipes to be able to invoke hooks on command, in a
2052
2052
long-running process style. This would minimise latency at the cost of some
2053
- memory.
2053
+ memory.
2054
2054
This choice might be appropriate for an IDE such as HLS, as well as for
2055
2055
use with GHCi.
2056
2056
@@ -2061,7 +2061,7 @@ external hooks executable it has compiled, and deserialising the output data
2061
2061
from the executable to obtain the outputs (` PreConfPackageOutputs ` ).
2062
2062
2063
2063
In server style, one would avoid the cost of having to pass this data over
2064
- and over, as one would only need to pass what has changed in the meantime.
2064
+ and over, as one would only need to pass what has changed in the meantime.
2065
2065
2066
2066
On top of these options, it would also be possible to directly link against the
2067
2067
hooks, or to dynamically load them into an existing process, to further minimise
@@ -2070,24 +2070,20 @@ in practice.
2070
2070
2071
2071
# References
2072
2072
2073
- <a id="carte" href=https://www.microsoft.com/en-us/research/uploads/prod/2018/03/build-systems.pdf>
2074
- [ Build Systems à la Carte] </a >
2075
- Andrey Mokhov, Neil Mitchell, Simon Peyton Jones: <b >Build Systems à la Carte</b >
2076
- (2018).
2077
-
2078
- <a id="cloud-haskell" href=https://www.microsoft.com/en-us/research/wp-content/uploads/2016/07/remote.pdf>
2079
- [ Towards Haskell in the Cloud] </a >
2080
- Jeff Epstein, Andrew P. Black, Simon Peyton Jones: <b >Towards Haskell in the Cloud</b >
2081
- (2011).
2082
-
2083
- <a id="delivery" href=https://www.cs.ox.ac.uk/jeremy.gibbons/publications/delivery.pdf>
2084
- [ Free Delivery] </a >
2085
- Jeremy Gibbons: <b >Free Delivery</b > (2016).
2086
-
2087
- <a id="hadrian" href=https://www.microsoft.com/en-us/research/wp-content/uploads/2016/03/hadrian.pdf>
2088
- [ Hadrian] </a >
2089
- Andrey Mokhov, Neil Mitchell, Simon Peyton Jones, Simon Marlow: <b >Non-recursive Make Considered Harmful</b >
2090
- (2016).
2091
-
2092
- <a id="ninja" href=https://ninja-build.org>[Ninja ] </a >
2093
- The ninja build system.
2073
+ * <a id="carte" href=https://www.microsoft.com/en-us/research/uploads/prod/2018/03/build-systems.pdf>
2074
+ [ Build Systems à la Carte] </a >
2075
+ Andrey Mokhov, Neil Mitchell, Simon Peyton Jones: <b >Build Systems à la Carte</b >
2076
+ (2018).
2077
+ * <a id="cloud-haskell" href=https://www.microsoft.com/en-us/research/wp-content/uploads/2016/07/remote.pdf>
2078
+ [ Towards Haskell in the Cloud] </a >
2079
+ Jeff Epstein, Andrew P. Black, Simon Peyton Jones: <b >Towards Haskell in the Cloud</b >
2080
+ (2011).
2081
+ * <a id="delivery" href=https://www.cs.ox.ac.uk/jeremy.gibbons/publications/delivery.pdf>
2082
+ [ Free Delivery] </a >
2083
+ Jeremy Gibbons: <b >Free Delivery</b > (2016).
2084
+ * <a id="hadrian" href=https://www.microsoft.com/en-us/research/wp-content/uploads/2016/03/hadrian.pdf>
2085
+ [ Hadrian] </a >
2086
+ Andrey Mokhov, Neil Mitchell, Simon Peyton Jones, Simon Marlow: <b >Non-recursive Make Considered Harmful</b >
2087
+ (2016).
2088
+ * <a id="ninja" href=https://ninja-build.org>[Ninja ] </a >
2089
+ The ninja build system.
0 commit comments