Precompile Issue and Test Failure

[gizmaa]

I have a functional Mosquitto broker running on a Raspberry Pi. It has been tested directly via the command line but fails in Julia.

(@v1.10) pkg> add https://github.com/JuliaMessaging/MQTT.jl
     Cloning git-repo `https://github.com/JuliaMessaging/MQTT.jl`
    Updating git-repo `https://github.com/JuliaMessaging/MQTT.jl`
   Resolving package versions...
    Updating `C:\Users\JuliaUser\.julia\environments\v1.10\Project.toml`
  [ebefff21] + MQTT v0.1.0 `https://github.com/JuliaMessaging/MQTT.jl#main`
    Updating `C:\Users\JuliaUser\.julia\environments\v1.10\Manifest.toml`
  [ebefff21] + MQTT v0.1.0 `https://github.com/JuliaMessaging/MQTT.jl#main`
Precompiling project...
  1 dependency successfully precompiled in 2 seconds. 59 already precompiled.

julia> using MQTT
[ Info: Precompiling MQTTClientExt [7cf2be68-6216-550b-92cb-fb9fb1a7cdcd]
ERROR: LoadError: ArgumentError: Package MQTTClient [985f35cc-2c3d-4943-b8c1-f0931d5f0959] is required but does not seem to be installed:
 - Run `Pkg.instantiate()` to install all recorded dependencies.

Stacktrace:
  [1] _require(pkg::Base.PkgId, env::String)
    @ Base .\loading.jl:1920
  [2] __require_prelocked(uuidkey::Base.PkgId, env::String)
    @ Base .\loading.jl:1806
  [3] #invoke_in_world#3
    @ Base .\essentials.jl:921 [inlined]
  [4] invoke_in_world
    @ Base .\essentials.jl:918 [inlined]
  [5] _require_prelocked(uuidkey::Base.PkgId, env::String)
    @ Base .\loading.jl:1797
  [6] macro expansion
    @ Base .\loading.jl:1784 [inlined]
  [7] macro expansion
    @ Base .\lock.jl:267 [inlined]
  [8] __require(into::Module, mod::Symbol)
    @ Base .\loading.jl:1747
  [9] #invoke_in_world#3
    @ Base .\essentials.jl:921 [inlined]
 [10] invoke_in_world
    @ Base .\essentials.jl:918 [inlined]
 [11] require(into::Module, mod::Symbol)
    @ Base .\loading.jl:1740
 [12] include
    @ Base .\Base.jl:495 [inlined]
 [13] include_package_for_output(pkg::Base.PkgId, input::String, depot_path::Vector{String}, dl_load_path::Vector{String}, load_path::Vector{String}, concrete_deps::Vector{Pair{Base.PkgId, UInt128}}, source::Nothing)
    @ Base .\loading.jl:2216
 [14] top-level scope
    @ stdin:3
in expression starting at C:\Users\JuliaUser\.julia\packages\MQTT\mFE2O\ext\MQTTClientExt.jl:1
in expression starting at stdin:3
┌ Error: Error during loading of extension MQTTClientExt of MQTT, use `Base.retry_load_extensions()` to retry.
│   exception =
│    1-element ExceptionStack:
│    Failed to precompile MQTTClientExt [7cf2be68-6216-550b-92cb-fb9fb1a7cdcd] to "C:\\Users\\JuliaUser\\.julia\\compiled\\v1.10\\MQTTClientExt\\jl_CDDF.tmp".
│    Stacktrace:
│      [1] error(s::String)
│        @ Base .\error.jl:35
│      [2] compilecache(pkg::Base.PkgId, path::String, internal_stderr::IO, internal_stdout::IO, keep_loaded_modules::Bool)
│        @ Base .\loading.jl:2462
│      [3] compilecache(pkg::Base.PkgId, path::String, internal_stderr::IO, internal_stdout::IO, keep_loaded_modules::Bool)
│        @ Base .\loading.jl:2334 [inlined]
│      [4] (::Base.var"#968#969"{Base.PkgId})()
│        @ Base .\loading.jl:1968
│      [5] mkpidlock(f::Base.var"#968#969"{Base.PkgId}, at::String, pid::Int32; kwopts::@Kwargs{stale_age::Int64, wait::Bool})
│        @ FileWatching.Pidfile C:\Users\JuliaUser\AppData\Local\Programs\Julia\v1.10.0\share\julia\stdlib\v1.10\FileWatching\src\pidfile.jl:93
│      [6] #mkpidlock#6
│        @ FileWatching.Pidfile C:\Users\JuliaUser\AppData\Local\Programs\Julia\v1.10.0\share\julia\stdlib\v1.10\FileWatching\src\pidfile.jl:88 [inlined]
│      [7] trymkpidlock(::Function, ::Vararg{Any}; kwargs::@Kwargs{stale_age::Int64})
│        @ FileWatching.Pidfile C:\Users\JuliaUser\AppData\Local\Programs\Julia\v1.10.0\share\julia\stdlib\v1.10\FileWatching\src\pidfile.jl:111
│      [8] #invokelatest#2
│        @ Base .\essentials.jl:889 [inlined]
│      [9] invokelatest
│        @ Base .\essentials.jl:884 [inlined]
│     [10] maybe_cachefile_lock(f::Base.var"#968#969"{Base.PkgId}, pkg::Base.PkgId, srcpath::String; stale_age::Int64)
│        @ Base .\loading.jl:2977
│     [11] maybe_cachefile_lock
│        @ Base .\loading.jl:2974 [inlined]
│     [12] _require(pkg::Base.PkgId, env::Nothing)
│        @ Base .\loading.jl:1964
│     [13] __require_prelocked(uuidkey::Base.PkgId, env::Nothing)
│        @ Base .\loading.jl:1806
│     [14] #invoke_in_world#3
│        @ .\essentials.jl:921 [inlined]
│     [15] invoke_in_world
│        @ .\essentials.jl:918 [inlined]
│     [16] _require_prelocked
│        @ .\loading.jl:1797 [inlined]
│     [17] _require_prelocked
│        @ .\loading.jl:1796 [inlined]
│     [18] run_extension_callbacks(extid::Base.ExtensionId)
│        @ Base .\loading.jl:1289
│     [19] run_extension_callbacks(pkgid::Base.PkgId)
│        @ Base .\loading.jl:1324
│     [20] run_package_callbacks(modkey::Base.PkgId)
│        @ Base .\loading.jl:1158
│     [21] __require_prelocked(uuidkey::Base.PkgId, env::String)
│        @ Base .\loading.jl:1813
│     [22] #invoke_in_world#3
│        @ Base .\essentials.jl:921 [inlined]
│     [23] invoke_in_world
│        @ Base .\essentials.jl:918 [inlined]
│     [24] _require_prelocked(uuidkey::Base.PkgId, env::String)
│        @ Base .\loading.jl:1797
│     [25] macro expansion
│        @ Base .\loading.jl:1784 [inlined]
│     [26] macro expansion
│        @ Base .\lock.jl:267 [inlined]
│     [27] __require(into::Module, mod::Symbol)
│        @ Base .\loading.jl:1747
│     [28] #invoke_in_world#3
│        @ Base .\essentials.jl:921 [inlined]
│     [29] invoke_in_world
│        @ Base .\essentials.jl:918 [inlined]
│     [30] require(into::Module, mod::Symbol)
│        @ Base .\loading.jl:1740
│     [31] eval
│        @ .\boot.jl:385 [inlined]
│     [32] eval_user_input(ast::Any, backend::REPL.REPLBackend, mod::Module)
│        @ REPL C:\Users\JuliaUser\AppData\Local\Programs\Julia\v1.10.0\share\julia\stdlib\v1.10\REPL\src\REPL.jl:150
│     [33] repl_backend_loop(backend::REPL.REPLBackend, get_module::Function)
│        @ REPL C:\Users\JuliaUser\AppData\Local\Programs\Julia\v1.10.0\share\julia\stdlib\v1.10\REPL\src\REPL.jl:246
│     [34] start_repl_backend(backend::REPL.REPLBackend, consumer::Any; get_module::Function)
│        @ REPL C:\Users\JuliaUser\AppData\Local\Programs\Julia\v1.10.0\share\julia\stdlib\v1.10\REPL\src\REPL.jl:231
│     [35] run_repl(repl::REPL.AbstractREPL, consumer::Any; backend_on_current_task::Bool, backend::Any)
│        @ REPL C:\Users\JuliaUser\AppData\Local\Programs\Julia\v1.10.0\share\julia\stdlib\v1.10\REPL\src\REPL.jl:389
│     [36] run_repl(repl::REPL.AbstractREPL, consumer::Any)
│        @ REPL C:\Users\JuliaUser\AppData\Local\Programs\Julia\v1.10.0\share\julia\stdlib\v1.10\REPL\src\REPL.jl:375
│     [37] (::Base.var"#1013#1015"{Bool, Bool, Bool})(REPL::Module)
│        @ Base .\client.jl:432
│     [38] #invokelatest#2
│        @ Base .\essentials.jl:887 [inlined]
│     [39] invokelatest
│        @ Base .\essentials.jl:884 [inlined]
│     [40] run_main_repl(interactive::Bool, quiet::Bool, banner::Bool, history_file::Bool, color_set::Bool)
│        @ Base .\client.jl:416
│     [41] exec_options(opts::Base.JLOptions)
│        @ Base .\client.jl:333
│     [42] _start()
│        @ Base .\client.jl:552
└ @ Base loading.jl:1295

julia> using MQTTClient
 │ Package MQTTClient not found, but a package named MQTTClient is available from a registry.
 │ Install package?
 │   (@v1.10) pkg> add MQTTClient
 └ (y/n/o) [y]: y
   Resolving package versions...
    Updating `C:\Users\JuliaUser\.julia\environments\v1.10\Project.toml`
  [985f35cc] + MQTTClient v0.3.1
    Updating `C:\Users\JuliaUser\.julia\environments\v1.10\Manifest.toml`
  [985f35cc] + MQTTClient v0.3.1
  [8ba89e20] + Distributed
Precompiling project...
  ✗ MQTTClient → PrecompileMQTT
  1 dependency successfully precompiled in 5 seconds. 62 already precompiled.

Restart Julia

julia> using MQTT, MQTTClient
Precompiling PrecompileMQTT
        Info Given PrecompileMQTT was explicitly requested, output will be shown live
ERROR: LoadError: ArgumentError: could not listen on path /tmp/testmqtt.sock
Stacktrace:
  [1] listen(path::String)
    @ Sockets C:\Users\JuliaUser\AppData\Local\Programs\Julia\v1.10.0\share\julia\stdlib\v1.10\Sockets\src\PipeServer.jl:79
  [2] MockMQTTBroker(args::String)
    @ MQTTClient C:\Users\JuliaUser\.julia\packages\MQTTClient\yZP6s\src\utils.jl:160
  [3] macro expansion
    @ C:\Users\JuliaUser\.julia\packages\MQTTClient\yZP6s\ext\PrecompileMQTT.jl:124 [inlined]
  [4] macro expansion
    @ C:\Users\JuliaUser\.julia\packages\PrecompileTools\L8A3n\src\workloads.jl:78 [inlined]
  [5] macro expansion
    @ C:\Users\JuliaUser\.julia\packages\MQTTClient\yZP6s\ext\PrecompileMQTT.jl:21 [inlined]
  [6] macro expansion
    @ C:\Users\JuliaUser\.julia\packages\PrecompileTools\L8A3n\src\workloads.jl:140 [inlined]
  [7] top-level scope
    @ C:\Users\JuliaUser\.julia\packages\MQTTClient\yZP6s\ext\PrecompileMQTT.jl:13
  [8] include
    @ Base .\Base.jl:495 [inlined]
  [9] include_package_for_output(pkg::Base.PkgId, input::String, depot_path::Vector{String}, dl_load_path::Vector{String}, load_path::Vector{String}, concrete_deps::Vector{Pair{Base.PkgId, UInt128}}, source::Nothing)
    @ Base .\loading.jl:2216
 [10] top-level scope
    @ stdin:3
in expression starting at C:\Users\JuliaUser\.julia\packages\MQTTClient\yZP6s\ext\PrecompileMQTT.jl:2
in expression starting at stdin:3
  ✗ MQTTClient → PrecompileMQTT
  0 dependencies successfully precompiled in 8 seconds. 4 already precompiled.
[ Info: Precompiling PrecompileMQTT [d95a94f8-d074-5308-be8f-46211dc27fa2]
ERROR: LoadError: ArgumentError: could not listen on path /tmp/testmqtt.sock
Stacktrace:
  [1] listen(path::String)
    @ Sockets C:\Users\JuliaUser\AppData\Local\Programs\Julia\v1.10.0\share\julia\stdlib\v1.10\Sockets\src\PipeServer.jl:79
  [2] MockMQTTBroker(args::String)
    @ MQTTClient C:\Users\JuliaUser\.julia\packages\MQTTClient\yZP6s\src\utils.jl:160
  [3] macro expansion
    @ C:\Users\JuliaUser\.julia\packages\MQTTClient\yZP6s\ext\PrecompileMQTT.jl:124 [inlined]
  [4] macro expansion
    @ C:\Users\JuliaUser\.julia\packages\PrecompileTools\L8A3n\src\workloads.jl:78 [inlined]
  [5] macro expansion
    @ C:\Users\JuliaUser\.julia\packages\MQTTClient\yZP6s\ext\PrecompileMQTT.jl:21 [inlined]
  [6] macro expansion
    @ C:\Users\JuliaUser\.julia\packages\PrecompileTools\L8A3n\src\workloads.jl:140 [inlined]
  [7] top-level scope
    @ C:\Users\JuliaUser\.julia\packages\MQTTClient\yZP6s\ext\PrecompileMQTT.jl:13
  [8] include
    @ Base .\Base.jl:495 [inlined]
  [9] include_package_for_output(pkg::Base.PkgId, input::String, depot_path::Vector{String}, dl_load_path::Vector{String}, load_path::Vector{String}, concrete_deps::Vector{Pair{Base.PkgId, UInt128}}, source::Nothing)
    @ Base .\loading.jl:2216
 [10] top-level scope
    @ stdin:3
in expression starting at C:\Users\JuliaUser\.julia\packages\MQTTClient\yZP6s\ext\PrecompileMQTT.jl:2
in expression starting at stdin:3
┌ Error: Error during loading of extension PrecompileMQTT of MQTTClient, use `Base.retry_load_extensions()` to retry.
│   exception =
│    1-element ExceptionStack:
│    Failed to precompile PrecompileMQTT [d95a94f8-d074-5308-be8f-46211dc27fa2] to "C:\\Users\\JuliaUser\\.julia\\compiled\\v1.10\\PrecompileMQTT\\jl_4E37.tmp".
│    Stacktrace:
│      [1] error(s::String)
│        @ Base .\error.jl:35
│      [2] compilecache(pkg::Base.PkgId, path::String, internal_stderr::IO, internal_stdout::IO, keep_loaded_modules::Bool)
│        @ Base .\loading.jl:2462
│      [3] compilecache(pkg::Base.PkgId, path::String, internal_stderr::IO, internal_stdout::IO, keep_loaded_modules::Bool)
│        @ Base .\loading.jl:2334 [inlined]
│      [4] (::Base.var"#968#969"{Base.PkgId})()
│        @ Base .\loading.jl:1968
│      [5] mkpidlock(f::Base.var"#968#969"{Base.PkgId}, at::String, pid::Int32; kwopts::@Kwargs{stale_age::Int64, wait::Bool})
│        @ FileWatching.Pidfile C:\Users\JuliaUser\AppData\Local\Programs\Julia\v1.10.0\share\julia\stdlib\v1.10\FileWatching\src\pidfile.jl:93
│      [6] #mkpidlock#6
│        @ FileWatching.Pidfile C:\Users\JuliaUser\AppData\Local\Programs\Julia\v1.10.0\share\julia\stdlib\v1.10\FileWatching\src\pidfile.jl:88 [inlined]
│      [7] trymkpidlock(::Function, ::Vararg{Any}; kwargs::@Kwargs{stale_age::Int64})
│        @ FileWatching.Pidfile C:\Users\JuliaUser\AppData\Local\Programs\Julia\v1.10.0\share\julia\stdlib\v1.10\FileWatching\src\pidfile.jl:111
│      [8] #invokelatest#2
│        @ Base .\essentials.jl:889 [inlined]
│      [9] invokelatest
│        @ Base .\essentials.jl:884 [inlined]
│     [10] maybe_cachefile_lock(f::Base.var"#968#969"{Base.PkgId}, pkg::Base.PkgId, srcpath::String; stale_age::Int64)
│        @ Base .\loading.jl:2977
│     [11] maybe_cachefile_lock
│        @ Base .\loading.jl:2974 [inlined]
│     [12] _require(pkg::Base.PkgId, env::Nothing)
│        @ Base .\loading.jl:1964
│     [13] __require_prelocked(uuidkey::Base.PkgId, env::Nothing)
│        @ Base .\loading.jl:1806
│     [14] #invoke_in_world#3
│        @ .\essentials.jl:921 [inlined]
│     [15] invoke_in_world
│        @ .\essentials.jl:918 [inlined]
│     [16] _require_prelocked
│        @ .\loading.jl:1797 [inlined]
│     [17] _require_prelocked
│        @ .\loading.jl:1796 [inlined]
│     [18] run_extension_callbacks(extid::Base.ExtensionId)
│        @ Base .\loading.jl:1289
│     [19] run_extension_callbacks(pkgid::Base.PkgId)
│        @ Base .\loading.jl:1324
│     [20] run_package_callbacks(modkey::Base.PkgId)
│        @ Base .\loading.jl:1158
│     [21] __require_prelocked(uuidkey::Base.PkgId, env::String)
│        @ Base .\loading.jl:1813
│     [22] #invoke_in_world#3
│        @ Base .\essentials.jl:921 [inlined]
│     [23] invoke_in_world
│        @ Base .\essentials.jl:918 [inlined]
│     [24] _require_prelocked(uuidkey::Base.PkgId, env::String)
│        @ Base .\loading.jl:1797
│     [25] macro expansion
│        @ Base .\loading.jl:1784 [inlined]
│     [26] macro expansion
│        @ Base .\lock.jl:267 [inlined]
│     [27] __require(into::Module, mod::Symbol)
│        @ Base .\loading.jl:1747
│     [28] #invoke_in_world#3
│        @ Base .\essentials.jl:921 [inlined]
│     [29] invoke_in_world
│        @ Base .\essentials.jl:918 [inlined]
│     [30] require(into::Module, mod::Symbol)
│        @ Base .\loading.jl:1740
│     [31] eval
│        @ .\boot.jl:385 [inlined]
│     [32] eval_user_input(ast::Any, backend::REPL.REPLBackend, mod::Module)
│        @ REPL C:\Users\JuliaUser\AppData\Local\Programs\Julia\v1.10.0\share\julia\stdlib\v1.10\REPL\src\REPL.jl:150
│     [33] repl_backend_loop(backend::REPL.REPLBackend, get_module::Function)
│        @ REPL C:\Users\JuliaUser\AppData\Local\Programs\Julia\v1.10.0\share\julia\stdlib\v1.10\REPL\src\REPL.jl:246
│     [34] start_repl_backend(backend::REPL.REPLBackend, consumer::Any; get_module::Function)
│        @ REPL C:\Users\JuliaUser\AppData\Local\Programs\Julia\v1.10.0\share\julia\stdlib\v1.10\REPL\src\REPL.jl:231
│     [35] run_repl(repl::REPL.AbstractREPL, consumer::Any; backend_on_current_task::Bool, backend::Any)
│        @ REPL C:\Users\JuliaUser\AppData\Local\Programs\Julia\v1.10.0\share\julia\stdlib\v1.10\REPL\src\REPL.jl:389
│     [36] run_repl(repl::REPL.AbstractREPL, consumer::Any)
│        @ REPL C:\Users\JuliaUser\AppData\Local\Programs\Julia\v1.10.0\share\julia\stdlib\v1.10\REPL\src\REPL.jl:375
│     [37] (::Base.var"#1013#1015"{Bool, Bool, Bool})(REPL::Module)
│        @ Base .\client.jl:432
│     [38] #invokelatest#2
│        @ Base .\essentials.jl:887 [inlined]
│     [39] invokelatest
│        @ Base .\essentials.jl:884 [inlined]
│     [40] run_main_repl(interactive::Bool, quiet::Bool, banner::Bool, history_file::Bool, color_set::Bool)
│        @ Base .\client.jl:416
│     [41] exec_options(opts::Base.JLOptions)
│        @ Base .\client.jl:333
│     [42] _start()
│        @ Base .\client.jl:552
└ @ Base loading.jl:1295

julia> using MQTT, MQTTClient

julia> broker = "192.168.178.60";port = 1883
1883

julia> mqttconnection = MQTT.MQTTConnection(MQTTClient.MakeConnection(broker, port))
MQTTClientExt.MQTTClientConfig(MQTTClient[state: ready, read_loop: ready, write_loop: ready, keep_alive: ready]
, Connection(Protocol: MQTTClient.TCP(ip"192.168.178.60", 1883), Client ID: wghpYMP2))

julia> subscribe!(mqttconnection, "Arduino/temperature/Au", EXACTLY_ONCE) do (topic, payload)
           println("MQTT[$topic]: $payload")
       end

Publishing something on the Arduino/temperature/Au topic doesn't return anything in Julia.

Windows 11 Pro 23H2
Julia Version 1.10.0
Commit 3120989f39 (2023-12-25 18:01 UTC)
Build Info:
Official https://julialang.org/ release
Platform Info:
OS: Windows (x86_64-w64-mingw32)
CPU: 8 × Intel(R) Core(TM) i5-10210U CPU @ 1.60GHz
WORD_SIZE: 64
LIBM: libopenlibm
LLVM: libLLVM-15.0.7 (ORCJIT, skylake)
Threads: 2 on 8 virtual cores

[divbyzerofordummies]

The problem does not lie within this package, but the MQTTClient library it uses as a backend and tries to precompile.

If you are running on windows, maybe your error is linked to my issue.
There is something in the precompile instructions Julia uses to speed up runtime speed that fails.

I have already provided a pull request, but it has not been merged yet.

Maybe you could get things to run if you do not add MQTTClient as your backend, but rather AWSCRT; I have not used that, though, so I could not say.

[NickMcSweeney]

@gizmaa is this still an issue? I think that the PR on MQTTClient fixed this for windows. If it is I can move this to an issue on MQTTClient, otherwise I will close this.

[gizmaa]

Even after an update it doesn't seem to work but you are right, it looks like an MQTTClient problem.