libpython error during installation

[sibyjackgrove]

@jlapeyre Thank you for this package. I just installed it following the instructions. But I am getting the following error when I do import diffeq_julia

  [8dfed614] + Test
  [cf7118a7] + UUIDs
  [4ec0a83e] + Unicode
ERROR: InitError: Incompatible `libpython` detected.
`libpython` for C:\Users\splathottam\.conda\envs\diffeq_julia\python.exe is:
    C:\Users\splathottam\.conda\envs\diffeq_julia\python37.dll
`libpython` for C:\Users\splathottam\.conda\envs\pvder\python.exe is:
    C:\Users\splathottam\.conda\envs\pvder\python37.dll
PyCall.jl only supports loading Python environment using
the same `libpython`.

I am installing this in the diffeq_julia Conda environment. But for some reason, the libpython in another Conda environment is getting picked up. Could you suggest a fix?

[jlapeyre]

I am installing this in the diffeq_julia Conda environment.

I don't know which environment this is. Did you just create it ?
I developed these things without using Conda at all.

Something like this should work.

• Clone this repo

• python -m venv ./venv

• source ./venv/bin/activate

• pip install -r requirements.txt

• pip install -e .

• pip install ipython

• Start ipython and do import diffeq_julia

I have see something about the error you mention, in PyCall or pyjulia. I'll look into it.

[sibyjackgrove]

@jlapeyre Yes, diffeq_julia is just a Python 3.7 environment I created using Conda. I generally use conda to create virtual environments. But I will try the method you showed.

[jlapeyre]

I think the solution may involve setting this environment variable PYCALL_JL_RUNTIME_PYTHON in the right place.
This should be set to what you get from which python.
You might try setting that environment variable before launching python.

[jlapeyre]

@sibyjackgrove . I just push a commit to julia_project that sets the above environment variable early. I wonder if this solves your original issue. It would be nice if this supports both conda and venv.

[jlapeyre]

I should point out that julia_project should does not require installing the package diffeq_julia in dev mode, i.e. with -e ..
You can also install it in the normal way.

For example, you can download diffeq_julia, then make an environment variable on the same level, not under it.

• git clone https://github.com/jlapeyre/diffeq_julia
• mkdir diffeq_env
• cd diffeq_env
• python -m venv ./venv
• source ./venv/bin/activate
• pip install -r ../diffeq_julia/requirements.txt
• pip install ../diffeq_julia

I should try to get this to work via conda as well.

[jlapeyre]

just push a commit to julia_project that sets the above environment variable early.
I have a feeling this won't work. I am setting it on the Python side, and it might be too late for Julia to see it.... Trying something else.

[sibyjackgrove]

@jlapeyre Yes, I reinstalled julia_project after pulling the latest commit. However, I am getting the same error when I do diffeq_julia.compile_diffeq_julia(). It is setting the PYCALL_JL_RUNTIME_PYTHON to the correct Python path for the environment. But as you said, Julia doesn't seem to be seeing it.

[jlapeyre]

Arg! I pushed a commit, then a change, then reverted with a new commit.
I fear that you got the bad commit in the middle. Can you try again, please. Sorry!

[jlapeyre]

I am getting the same error when I do diffeq_julia.compile_diffeq_julia().

Oh, wait. I did not know that this is happening when you are compiling. This is late in the process.

But, you apparently have no problem importing diffeq_julia to begin? It is only when compiling that you get the error ?

[jlapeyre]

I see what may be an important difference with python -m venv, no libpython is copied into your virtual environment. The libpython that created the venv (e.g. from your system installation) is still used. So different envs might use the same libpython. But, Conda seems to make a copy of libpython for each virtual env.

[jlapeyre]

PyCall insists that users rebuild PyCall to use it with a different library. Even if it is a bit for bit copy in a different location. I can look for a way around this, but won't be able to get to it this week.

Using venv, has a smaller proliferation of paths to libpython is a workarund.

[jlapeyre]

Here is an issue for this:
jlapeyre/julia_project#2

PyCall.jl README says flatly that conda virtual environments are not supported:

https://github.com/JuliaPy/PyCall.jl#python-virtual-environments

It doesn't say why, but guess it's because the libpython is always copied to an environment.

[sibyjackgrove]

@jlapeyre Thank you for looking into this further. So, from what I understand it won't work with Conda environments at this time?

[jlapeyre]

Yes. My best understanding is julia_project and diffeq_julia will not work with Conda environments because they are not supported by the underlying tool, PyCall. They are supported by venv.

We may be able to find a way to support Conda environments. But, that will take some work.