Incremental compilation of this repo is broken

[katrinafyi]

Compiling the compiler leads to a full re-compile every time, even if nothing has changed.

Compiler version

bisected to 9e88707 as the first bad commit, and 0a7f843 is the preceding good commit

• started bisecting from good commit (i arbitrarily went back 100 commits) 1922c0f
• still bad at main as of time of writing d84dc5c

if it matters, the sbt and java versions are:

[info] welcome to sbt 1.11.0 (N/A Java 17.0.15)

openjdk version "17.0.15" 2025-04-15
OpenJDK Runtime Environment (build 17.0.15+6-suse-1.1-x8664)
OpenJDK 64-Bit Server VM (build 17.0.15+6-suse-1.1-x8664, mixed mode, sharing)

Minimized code

check out this repo and try to compile twice with no changes in between.

sbt compile compile

Output

second compilation reports sources are being compiled again and still takes large amounts of time

[info] compiling 36 Scala sources to /home/x/progs/scala3/compiler/target/scala-3.7.3-RC2/classes ...
[info] compiling 44 Scala sources to /home/x/progs/scala3/compiler/target/scala-3.7.3-RC2/classes ...
[info] compiling 46 Scala sources to /home/x/progs/scala3/compiler/target/scala-3.7.3-RC2/classes ...
[info] compiling 598 Scala sources and 8 Java sources to /home/x/progs/scala3/compiler/target/scala-3.7.3-RC2/classes ...
[info] compiling 4 Java sources to /home/x/progs/scala3/sbt-bridge/src/target/classes ...
[success] Total time: 64 s (0:01:04.0), completed 3 Sept 2025, 12:22:46 pm
[info] compiling 36 Scala sources to /home/x/progs/scala3/compiler/target/scala-3.7.3-RC2/classes ...
[info] compiling 44 Scala sources to /home/x/progs/scala3/compiler/target/scala-3.7.3-RC2/classes ...
[info] compiling 46 Scala sources to /home/x/progs/scala3/compiler/target/scala-3.7.3-RC2/classes ...
[info] compiling 598 Scala sources and 8 Java sources to /home/x/progs/scala3/compiler/target/scala-3.7.3-RC2/classes ...
[success] Total time: 38 s, completed 3 Sept 2025, 12:23:25 pm

Expectation

with a successful incremental compilation, the second compile should not mention any compilations and should be much faster.

[info] Executing in batch mode. For better performance use sbt's shell
[info] compiling 181 Scala sources to /home/x/progs/scala3/compiler/target/scala-3.7.3-RC2/classes ...
[info] compiling 164 Scala sources to /home/x/progs/scala3/compiler/target/scala-3.7.3-RC2/classes ...
[info] compiling 100 Scala sources to /home/x/progs/scala3/compiler/target/scala-3.7.3-RC2/classes ...
[info] compiling 4 Java sources to /home/x/progs/scala3/sbt-bridge/src/target/classes ...
[success] Total time: 55 s, completed 3 Sept 2025, 12:05:37 pm
[success] Total time: 2 s, completed 3 Sept 2025, 12:05:39 pm

[Gedochao]

cc @hamzaremmal @noti0na1

[hamzaremmal]

First, I'm surprised that the Develocity cache didn't kick in when you have a fresh clone (on fbbae09) and run sbt "compile;compile". That's what I would expect and that's what happened when I've done it.

hamzaremmal@macbook scala3 % sbt "compile;compile"
[info] welcome to sbt 1.11.5 (Oracle Corporation Java 17.0.7)
[info] loading settings for project scala3-build-build from build.sbt...
[info] loading project definition from /Users/hamzaremmal/Desktop/scala/scala3/project/project
[info] loading settings for project scala3-build from build.sbt, plugins.sbt...
[info] loading project definition from /Users/hamzaremmal/Desktop/scala/scala3/project
[info] loading settings for project scala3 from build.sbt...
[info] resolving key references (61391 settings) ...
git version 2.39.5 (Apple Git-154)
[info] set current project to scala3 (in build file:/Users/hamzaremmal/Desktop/scala/scala3/)
[success] Total time: 5 s, completed Sep 3, 2025, 11:09:40 AM
[success] Total time: 4 s, completed Sep 3, 2025, 11:09:44 AM
[info] Publishing Build Scan...
[info] https://develocity.scala-lang.org/s/xofoeiwqisp2m

Second, I've made sure that the bytecode manipulation we do on the new stdlib preserves the incremental compilation aspect (See 61de57d). I would very much appreciate having more information, and more specifically, is this still happening on the HEAD of main (fbbae09)?

[katrinafyi]

Seems to be fixed as of fbbae09, which I see is fixing another issue with the bisected commit. I guess the incremental compilation problem was another consequence of that issue.

I don't know anything about Devolocity cache and I don't see those messages in my sbt log - is this something which should "just work"?

[Gedochao]

is this something which should "just work"?

Ideally it should, yep.

[hamzaremmal]

I don't know anything about Devolocity cache and I don't see those messages in my sbt log - is this something which should "just work"?

Yes, it should just work out of the box but we do require to have a clean state. @katrinafyi Have you tried running sbt "clean;compile;compile"?

[katrinafyi]

Running it now, I still don't see the messages but there is a develocityCachePull task and it does seem to be suspiciously fast.

But I had done sbt clean compile compile while I was bisecting (on a different computer) and it did keep rebuilding. Maybe it's an OS issue, or broken by this issue, or maybe my state was sufficiently unclean to prevent it? What does "clean state" mean?

As far as I can tell, I've always been building the compiler from source - I got used to seeing "building 536 scala sources". Most of the time, I am building 3.3 if that makes a difference.

[hamzaremmal]

Most of the time, I am building 3.3 if that makes a difference.

Many many things changed since 3.3.x. I wouldn't be surprised if something doesn't work properly there. Also, we do not have any caching enabled in this branch and I'm not planning to put the efforts into it since 3.3.x EOL is close.

Running it now, I still don't see the messages but there is a develocityCachePull task and it does seem to be suspiciously fast.

It's the cache, so it's working :) (cc @dotta @Duhemm)

But I had done sbt clean compile compile while I was bisecting (on a different computer) and it did keep rebuilding. Maybe it's an OS issue, or broken by this issue, or maybe my state was sufficiently unclean to prevent it? What does "clean state" mean?

If you were building on main, then yes, it might be an issue. If you were building on 3.3.x, then I think this will be a "won't fix".
As what I understand a "clean state" just means we need to run "clean" before the task to benefit from the cache (cc @Duhemm for confirmation).

[katrinafyi]

I was building on main while bisecting, yes. On my computer, building on d84dc5c (main right before the fix commit) is not cached with sbt clean compile compile.

That said, it does seem to be working as of the latest main. I've only recently stated doing compiler work, so maybe I have just been unlucky and all of my work has been during the period while this issue existed ;-;

[katrinafyi]

I'm happy for this issue to be closed as you see fit. The presenting issue has been fixed. By a fun coincidence, the fix was within hours of filing the bug :-)