Is it being tested with real-world monorepos, or is it currently only intended for small repositories?

[develar]

Hi,

I’m very interested in GitButler, as I’m somewhat of a “core” developer and often make 20+ small commits across very different areas. Virtual Branches are exactly what I need to be able to create a separate PR for each logical set of changes.

However, GitButler currently feels slow, even for relatively small changes. For example, the “update” operation is noticeably slow (3 minutes), even with a low number of incoming commits (e.g., ~60 commits on a Sunday morning at my company).
(I provided a CPU trace for the issue about slow move).

Additionally, there are no visible progress indicators, so I often resort to monitoring CPU usage to guess when an operation has completed.

I’d like to understand whether these performance issues are:
• due to deep architectural limitations that would take months of work to resolve,
• or are they caused by relatively straightforward bugs or bottlenecks that could be fixed more easily?

Thanks in advance — I’d be happy to help test improvements or share more details.

Processing blobs: 9320142                        
Processing trees: 26881666                        
Processing commits: 2081465                        
Matching commits to trees: 2081465                        
Processing annotated tags: 0                        
Processing references: 12441                        
| Name                         | Value     | Level of concern               |
| ---------------------------- | --------- | ------------------------------ |
| Overall repository size      |           |                                |
| * Commits                    |           |                                |
|   * Count                    |  2.08 M   | ****                           |
|   * Total size               |   842 MiB | ***                            |
| * Trees                      |           |                                |
|   * Count                    |  26.9 M   | *****************              |
|   * Total size               |  30.2 GiB | ****************               |
|   * Total tree entries       |   706 M   | **************                 |
| * Blobs                      |           |                                |
|   * Count                    |  9.32 M   | ******                         |
|   * Total size               |   186 GiB | *******************            |
|                              |           |                                |
| Biggest objects              |           |                                |
| * Commits                    |           |                                |
|   * Maximum size         [1] |  63.7 KiB | *                              |
|   * Maximum parents      [2] |    15     | *                              |
| * Trees                      |           |                                |
|   * Maximum entries      [3] |  35.2 k   | !!!!!!!!!!!!!!!!!!!!!!!!!!!!!! |
| * Blobs                      |           |                                |
|   * Maximum size         [4] |   391 MiB | !!!!!!!!!!!!!!!!!!!!!!!!!!!!!! |
|                              |           |                                |
| History structure            |           |                                |
| * Maximum history depth      |   707 k   | *                              |
|                              |           |                                |
| Biggest checkouts            |           |                                |
| * Number of directories  [5] |   257 k   | !!!!!!!!!!!!!!!!!!!!!!!!!!!!!! |
| * Maximum path depth     [6] |    23     | **                             |
| * Maximum path length    [7] |   342 B   | ***                            |
| * Number of files        [8] |  1.19 M   | ***********************        |
| * Total size of files    [9] |  9.22 GiB | *********                      |

[1]  52e480831620289f4a72730e845231ff37d577d3
[2]  b747a1304a11fb5d9ed8d45feb9e90d625be8e4d
[3]  514ef99dca2cff3c0b79d2f51b862ea90de1016c (497a904f3c61e3aec51d4bb4491508bbc6aa8819:Psi.Features/test/data/Daemon/CSharp/RoslynNullable/RoslynTests)
[4]  dcc299919bcee8f0715d83af5565f97acb1c6705 (2f549e67942e6318b68c86c04478294de825e292:dotMemory/Test/Data/Raw snapshot/ssc02DCF.tmp.0002)
[5]  59798108123ac869d04657184e3ca7c6b3961edf (e5357eacc109e249bc049061e5091c19c62054a9^{tree})
[6]  a790c4b58d9a483326025d3ddb05a911bcca6f39 (refs/remotes/origin/252.21735^{tree})
[7]  b30bb2fd6d88114df3181204f81e3f1fb12eecdc (bb160734c89b12b4f6fdab0e74f38a242008908c^{tree})
[8]  24d5b7092f839058e28710b2333d351904066ead (88e924f4bdad918dcf19a771bc0a37282f8675fa^{tree})
[9]  2d974fb930ef6a9c1d34dc8fe3a1ba72b86fd2b7 (322725ec925128db3f2d496b7d075c16dd939728^{tree})

[Byron]

It's great to have you here and I'd love to continue working with you to sort out these kind of issues.

The biggest repositories I test on currently are GitLab, and the linux kernel occasionally. These are already slow enough to have a lot to work on, GitLab gets impossible to work with from time to time, but WebKit would otherwise be my final boss.

Generally, I expect GB to be as fast or faster than Git, while also communicating progress better than Git does.
Right now, there is no operation that actually implements progress, and I don't think that there is any design for that, but @PavelLaptev would know for sure. However, for large repositories that would be a must as operations will take time.

With that said…

• due to deep architectural limitations that would take months of work to resolve,

…I can say that it's likely going to take a while to resolve these issues and a rewrite of the core functionality is currently in progress. Whenever but_ prefixed methods show up in Instruments runs then it's already new code which would be expected to perform much better, but even there will be portions that have been ported over quickly and another pass will be needed to sort out additional performance (and maybe even correctness) issues.

With that said, if you have an operation where mostly but_ functions are called, it might already make sense to look into the traces. Something we could also consider is to have a notion of communicating progress earlier than planned. Now might be a good time to establish this, and it would be interesting how @krlvi thinks about that as well.

[develar]

Maybe there are some possible workarounds, like configuring which branches to ignore — for example, if the convention userName/branchName is used, we could check only the branches that match the current user.

It’s strange that even simple actions like “remove file from a commit” or commit are slow. The complexity of such tasks shouldn’t depend on the size of the monorepo, but rather on the number of applied branches and the size of the changed files — at least, that’s my understanding.

[Byron]

Removing a file from a commit should already be mostly new code, and I'd be keen to get another instruments save file if you find the time. (I look at these to see if something can be fixed right away, and store them for later reference otherwise. The time for performance optimisation will come.)