Building llama.cpp via flutter on windows

[danemadsen]

I'm having an issue when trying to build llama.cpp as part of the windows build of my flutter project Maid: https://github.com/MaidFoundation/Maid

If i build the project separately from llama.cpp it builds fine and if i build llama.cpp (and butler) separately from Maid they both build fine, its just when i try to combine the build process to have dll's build with maid that i get the error.

I know from reading the error it looks like a permissions issue which would usually be rectified by running in an administrator CMD or power-shell on windows, but that doesn't work any way Ive tried it.

The project builds fine on Linux and Android and the CMake file for Linux is logically very similar to windows so i don't really see whats causing the issue.

Below is the error:

[   +5 ms]   CMake Error at llama/cmake_install.cmake:41 (file):
[        ]     file cannot create directory:
[        ]     C:/maid/build/windows/x64/$<TARGET_FILE_DIR:maid>/lib.  Maybe need
[        ]     administrative privileges.
[        ]   Call Stack (most recent call first):
[        ]     cmake_install.cmake:310 (include)
[   +5 ms] C:\Program Files\Microsoft Visual Studio\2022\Community\MSBuild\Microsoft\VC\v170\Microsoft.CppCommon.targets(161,5): error MSB3073: The command "setlocal [C:\maid\build\windows\x64\INSTALL.vcxproj]
[        ] C:\Program Files\Microsoft Visual Studio\2022\Community\MSBuild\Microsoft\VC\v170\Microsoft.CppCommon.targets(161,5): error MSB3073: "C:\Program Files\Microsoft Visual Studio\2022\Community\Common7\IDE\CommonExtensions\Microsoft\CMake\CMake\bin\cmake.exe" -DBUILD_TYPE=Release -P cmake_install.cmake [C:\maid\build\windows\x64\INSTALL.vcxproj]
[        ] C:\Program Files\Microsoft Visual Studio\2022\Community\MSBuild\Microsoft\VC\v170\Microsoft.CppCommon.targets(161,5): error MSB3073: if %errorlevel% neq 0 goto :cmEnd [C:\maid\build\windows\x64\INSTALL.vcxproj]
[        ] C:\Program Files\Microsoft Visual Studio\2022\Community\MSBuild\Microsoft\VC\v170\Microsoft.CppCommon.targets(161,5): error MSB3073: :cmEnd [C:\maid\build\windows\x64\INSTALL.vcxproj]
[        ] C:\Program Files\Microsoft Visual Studio\2022\Community\MSBuild\Microsoft\VC\v170\Microsoft.CppCommon.targets(161,5): error MSB3073: endlocal & call :cmErrorLevel %errorlevel% & goto :cmDone [C:\maid\build\windows\x64\INSTALL.vcxproj]
[        ] C:\Program Files\Microsoft Visual Studio\2022\Community\MSBuild\Microsoft\VC\v170\Microsoft.CppCommon.targets(161,5): error MSB3073: :cmErrorLevel [C:\maid\build\windows\x64\INSTALL.vcxproj]
[        ] C:\Program Files\Microsoft Visual Studio\2022\Community\MSBuild\Microsoft\VC\v170\Microsoft.CppCommon.targets(161,5): error MSB3073: exit /b %1 [C:\maid\build\windows\x64\INSTALL.vcxproj]
[        ] C:\Program Files\Microsoft Visual Studio\2022\Community\MSBuild\Microsoft\VC\v170\Microsoft.CppCommon.targets(161,5): error MSB3073: :cmDone [C:\maid\build\windows\x64\INSTALL.vcxproj]
[        ] C:\Program Files\Microsoft Visual Studio\2022\Community\MSBuild\Microsoft\VC\v170\Microsoft.CppCommon.targets(161,5): error MSB3073: if %errorlevel% neq 0 goto :VCEnd [C:\maid\build\windows\x64\INSTALL.vcxproj]
[        ] C:\Program Files\Microsoft Visual Studio\2022\Community\MSBuild\Microsoft\VC\v170\Microsoft.CppCommon.targets(161,5): error MSB3073: :VCEnd" exited with code 1. [C:\maid\build\windows\x64\INSTALL.vcxproj]
[        ] Done Building Project "C:\maid\build\windows\x64\INSTALL.vcxproj" (default targets) -- FAILED.
[        ] Build FAILED.
[        ] "C:\maid\build\windows\x64\INSTALL.vcxproj" (default target) (1) ->
[        ] "C:\maid\build\windows\x64\ALL_BUILD.vcxproj" (default target) (3) ->
[        ] "C:\maid\build\windows\x64\llama\common\common.vcxproj" (default target) (5) ->
[        ] "C:\maid\build\windows\x64\llama\llama.vcxproj" (default target) (6) ->
[        ] (ClCompile target) ->
[        ]   C:\maid\src\llama.cpp\llama.cpp(1055,31): warning C4305: 'initializing': truncation from 'double' to 'float' [C:\maid\build\windows\x64\llama\llama.vcxproj]
[        ]   C:\maid\src\llama.cpp\llama.cpp(1205,40): warning C4566: character represented by universal-character-name '\u0120' cannot be represented in the current code page (1252) [C:\maid\build\windows\x64\llama\llama.vcxproj]
[        ]   C:\maid\src\llama.cpp\llama.cpp(1206,40): warning C4566: character represented by universal-character-name '\u010A' cannot be represented in the current code page (1252) [C:\maid\build\windows\x64\llama\llama.vcxproj]
[        ]   C:\maid\src\llama.cpp\llama.cpp(1207,40): warning C4566: character represented by universal-character-name '\u0120' cannot be represented in the current code page (1252) [C:\maid\build\windows\x64\llama\llama.vcxproj]
[        ]   C:\maid\src\llama.cpp\llama.cpp(1208,40): warning C4566: character represented by universal-character-name '\u010A' cannot be represented in the current code page (1252) [C:\maid\build\windows\x64\llama\llama.vcxproj]
[        ]   C:\maid\src\llama.cpp\llama.cpp(2244,60): warning C4566: character represented by universal-character-name '\u010A' cannot be represented in the current code page (1252) [C:\maid\build\windows\x64\llama\llama.vcxproj]
[        ]   C:\maid\src\llama.cpp\llama.cpp(9753,28): warning C4146: unary minus operator applied to unsigned type, result still unsigned [C:\maid\build\windows\x64\llama\llama.vcxproj]
[        ]   C:\maid\src\llama.cpp\llama.cpp(9783,28): warning C4146: unary minus operator applied to unsigned type, result still unsigned [C:\maid\build\windows\x64\llama\llama.vcxproj]
[        ] "C:\maid\build\windows\x64\INSTALL.vcxproj" (default target) (1) ->
[        ] "C:\maid\build\windows\x64\ALL_BUILD.vcxproj" (default target) (3) ->
[        ] "C:\maid\build\windows\x64\llama\common\common.vcxproj" (default target) (5) ->
[        ]   C:\maid\src\llama.cpp\common\console.cpp(253,30): warning C4267: 'initializing': conversion from 'size_t' to 'DWORD', possible loss of data [C:\maid\build\windows\x64\llama\common\common.vcxproj]
[        ]   C:\maid\src\llama.cpp\common\console.cpp(407,28): warning C4267: 'initializing': conversion from 'size_t' to 'int', possible loss of data [C:\maid\build\windows\x64\llama\common\common.vcxproj]
[        ]   C:\maid\src\llama.cpp\common\common.cpp(785): warning C4715: 'gpt_random_prompt': not all control paths return a value [C:\maid\build\windows\x64\llama\common\common.vcxproj]
[        ] "C:\maid\build\windows\x64\INSTALL.vcxproj" (default target) (1) ->
[        ] "C:\maid\build\windows\x64\ALL_BUILD.vcxproj" (default target) (3) ->
[        ] "C:\maid\build\windows\x64\butler\butler.vcxproj" (default target) (8) ->
[        ]   C:\maid\src\llama.cpp\common\log.h(379,19): warning C4996: 'fopen': This function or variable may be unsafe. Consider using fopen_s instead. To disable deprecation, use _CRT_SECURE_NO_WARNINGS. See online help for details. [C:\maid\build\windows\x64\butler\butler.vcxproj]
[        ]   C:\maid\src\llama.cpp\common\log.h(387,97): warning C4996: 'strerror': This function or variable may be unsafe. Consider using strerror_s instead. To disable deprecation, use _CRT_SECURE_NO_WARNINGS. See online help for details. [C:\maid\build\windows\x64\butler\butler.vcxproj]
[        ] "C:\maid\build\windows\x64\INSTALL.vcxproj" (default target) (1) ->
[        ] (PostBuildEvent target) ->
[        ]   C:\Program Files\Microsoft Visual Studio\2022\Community\MSBuild\Microsoft\VC\v170\Microsoft.CppCommon.targets(161,5): error MSB3073: The command "setlocal [C:\maid\build\windows\x64\INSTALL.vcxproj]
[        ] C:\Program Files\Microsoft Visual Studio\2022\Community\MSBuild\Microsoft\VC\v170\Microsoft.CppCommon.targets(161,5): error MSB3073: "C:\Program Files\Microsoft Visual Studio\2022\Community\Common7\IDE\CommonExtensions\Microsoft\CMake\CMake\bin\cmake.exe" -DBUILD_TYPE=Release -P cmake_install.cmake [C:\maid\build\windows\x64\INSTALL.vcxproj]
[        ] C:\Program Files\Microsoft Visual Studio\2022\Community\MSBuild\Microsoft\VC\v170\Microsoft.CppCommon.targets(161,5): error MSB3073: if %errorlevel% neq 0 goto :cmEnd [C:\maid\build\windows\x64\INSTALL.vcxproj]
[        ] C:\Program Files\Microsoft Visual Studio\2022\Community\MSBuild\Microsoft\VC\v170\Microsoft.CppCommon.targets(161,5): error MSB3073: :cmEnd [C:\maid\build\windows\x64\INSTALL.vcxproj]
[        ] C:\Program Files\Microsoft Visual Studio\2022\Community\MSBuild\Microsoft\VC\v170\Microsoft.CppCommon.targets(161,5): error MSB3073: endlocal & call :cmErrorLevel %errorlevel% & goto :cmDone [C:\maid\build\windows\x64\INSTALL.vcxproj]
[        ] C:\Program Files\Microsoft Visual Studio\2022\Community\MSBuild\Microsoft\VC\v170\Microsoft.CppCommon.targets(161,5): error MSB3073: :cmErrorLevel [C:\maid\build\windows\x64\INSTALL.vcxproj]
[        ] C:\Program Files\Microsoft Visual Studio\2022\Community\MSBuild\Microsoft\VC\v170\Microsoft.CppCommon.targets(161,5): error MSB3073: exit /b %1 [C:\maid\build\windows\x64\INSTALL.vcxproj]
[        ] C:\Program Files\Microsoft Visual Studio\2022\Community\MSBuild\Microsoft\VC\v170\Microsoft.CppCommon.targets(161,5): error MSB3073: :cmDone [C:\maid\build\windows\x64\INSTALL.vcxproj]
[        ] C:\Program Files\Microsoft Visual Studio\2022\Community\MSBuild\Microsoft\VC\v170\Microsoft.CppCommon.targets(161,5): error MSB3073: if %errorlevel% neq 0 goto :VCEnd [C:\maid\build\windows\x64\INSTALL.vcxproj]
[        ] C:\Program Files\Microsoft Visual Studio\2022\Community\MSBuild\Microsoft\VC\v170\Microsoft.CppCommon.targets(161,5): error MSB3073: :VCEnd" exited with code 1. [C:\maid\build\windows\x64\INSTALL.vcxproj]
[        ]     13 Warning(s)
[        ]     1 Error(s)
[        ] Time Elapsed 00:00:16.92
[  +16 ms] Building Windows application... (completed in 19.3s)
[        ] "flutter windows" took 19,857ms.
[   +2 ms] Build process failed.
[        ]
           #0      throwToolExit (package:flutter_tools/src/base/common.dart:10:3)
           #1      _runBuild (package:flutter_tools/src/windows/build_windows.dart:253:5)
           <asynchronous suspension>
           #2      buildWindows (package:flutter_tools/src/windows/build_windows.dart:106:5)
           <asynchronous suspension>
           #3      BuildWindowsCommand.runCommand (package:flutter_tools/src/commands/build_windows.dart:56:5)
           <asynchronous suspension>
           #4      FlutterCommand.run.<anonymous closure> (package:flutter_tools/src/runner/flutter_command.dart:1350:27)
           <asynchronous suspension>
           #5      AppContext.run.<anonymous closure> (package:flutter_tools/src/base/context.dart:150:19)
           <asynchronous suspension>
           #6      CommandRunner.runCommand (package:args/command_runner.dart:212:13)
           <asynchronous suspension>
           #7      FlutterCommandRunner.runCommand.<anonymous closure> (package:flutter_tools/src/runner/flutter_command_runner.dart:348:9)
           <asynchronous suspension>
           #8      AppContext.run.<anonymous closure> (package:flutter_tools/src/base/context.dart:150:19)
           <asynchronous suspension>
           #9      FlutterCommandRunner.runCommand (package:flutter_tools/src/runner/flutter_command_runner.dart:294:5)
           <asynchronous suspension>
           #10     run.<anonymous closure>.<anonymous closure> (package:flutter_tools/runner.dart:112:9)
           <asynchronous suspension>
           #11     AppContext.run.<anonymous closure> (package:flutter_tools/src/base/context.dart:150:19)
           <asynchronous suspension>
           #12     main (package:flutter_tools/executable.dart:90:3)
           <asynchronous suspension>


[ +267 ms] ensureAnalyticsSent: 263ms
[        ] Running 0 shutdown hooks
[        ] Shutdown hooks complete
[        ] exiting with code 1
PS C:\maid> flutter update
Could not find a command named "update".


Run 'flutter -h' (or 'flutter <command> -h') for available flutter commands and options.
PS C:\maid> flutter upgrade
Flutter is already up to date on channel beta
Flutter 3.16.0-0.2.pre • channel beta • https://github.com/flutter/flutter.git
Framework • revision fe6553b689 (6 days ago) • 2023-10-11 19:07:11 -0700
Engine • revision 249cc9b86c
Tools • Dart 3.2.0 (build 3.2.0-210.2.beta) • DevTools 2.28.1

[jpohhhh]

Hey Dane, was this the fix? Mobile-Artificial-Intelligence/maid@f6854f9

If so, it's sort of a shame, I got an Dart llama.cpp library completely abstracted into a plugin with Metal / optimizations on Android / etc., but it looks like it requires altering the depending app's CMakeLists.txt. That's not too bad to have as a manual step beyond pub add, but frustrating.

[danemadsen]

@jpohhhh yes that was A fix. Ive changed it since then though.

now all that's needed is adding the sub-directory to the CMakeList
https://github.com/Mobile-Artificial-Intelligence/maid/blob/main/windows/CMakeLists.txt#L107

I'm not sure there's a way to build llama.cpp as part of a dart plugin.

[jpohhhh]

Got it done: Telosnex/fllama@708074a
note the only changes needed are the ones mentioned below. that commit also removes a couple attempts to get it working that didn't work.

For posterity:

• Generally, this means you're running into [Windows] Adding desktop library with add_subdirectory causes build process to fail flutter/flutter#95530
• add_subdirectory is often used when depending on another CMake project. Stuart's generally correct, it's not the...most rigorous...approach. Regardless, it's commonly used enough that you should be able to make it work, eventually.
• here, we do that via adding EXCLUDE_FROM_ALL to src/CMakeLists.txt: i.e. the general CMakeLists that builds our plugin_name.c code
• then, in windows/CMakeLists.txt, add llama to the bundled libraries:

set(pluginname_bundled_libraries
  # Defined in ../src/CMakeLists.txt.
  # This can be changed to accommodate different builds.
  "$<TARGET_FILE:llama>"
  "$<TARGET_FILE:fllama>"
  PARENT_SCOPE
)

• n.b. this took a lot longer to figure out than I expected because there's so many CMakeLists.txt involved. I had convinced myself there was no CMakeLists.txt per-platform in the FFI plugin. But there are. i.e. both src/CMakeLists.txt (general) and windows...linux...etc/CMakeLists.txt exist