ImportError('Error importing numpy: you should not try to import numpy from\n its source directory; please exit the numpy source tree, and relaunch\n your python interpreter from there.')

[ZKBL]

my code

use pyo3::types::PyList;
use which::which;
fn main() {
    let path = which("python").unwrap();
    println!("{:?}", path);
    pyo3::Python::with_gil(|py| {
        let sys = py.import("sys").unwrap();
        let path: &PyList = sys.getattr("path").unwrap().extract().unwrap();
        path.append("".to_string()).unwrap();
        for i in path.iter() {
            println!("{:?}", i);
        }
        py.run("import cv2;import numpy as np;", None, None)
            .unwrap();
    })
}

run the command

PYO3_PYTHON=~/ImageRust/.venv/bin/python cargo r --bin test-path
outpu is

warning: file `/home/green/ImageRust/tests/test-path.rs` found to be present in multiple build targets:
  * `bin` target `test-path`
  * `integration-test` target `test-path`
   Compiling pyo3-build-config v0.20.0
   Compiling pyo3-ffi v0.20.0
   Compiling pyo3 v0.20.0
   Compiling image v0.1.0 (/home/green/ImageRust)
   Compiling numpy v0.20.0
   Compiling pyo3-asyncio v0.20.0
    Finished dev [unoptimized + debuginfo] target(s) in 4.90s
     Running `target/debug/test-path`
"/home/green/ImageRust/.venv/bin/python"
'/home/green/.rye/py/cpython@3.11.6/install/lib/python311.zip'
'/home/green/.rye/py/cpython@3.11.6/install/lib/python3.11'
'/home/green/.rye/py/cpython@3.11.6/install/lib/python3.11/lib-dynload'
'/home/green/ImageRust/.venv/lib/python3.11/site-packages'
'/home/green/ImageRust/python'
''
OpenCV bindings requires "numpy" package.
Install it via command:
    pip install numpy
thread 'main' panicked at tests/test-path.rs:14:14:
called `Result::unwrap()` on an `Err` value: PyErr { type: <class 'ImportError'>, value: ImportError('Error importing numpy: you should not try to import numpy from\n        its source directory; please exit the numpy source tree, and relaunch\n        your python interpreter from there.'), traceback: Some(<traceback object at 0x7feb79f56c80>) }
note: run with `RUST_BACKTRACE=1` environment variable to display a backtrace

The above is the case when the PYTHONHOME and PYTHONPATH environment variables are not set.

---

When I set PYTHHOME and PYTHPATH, the output looks like this

warning: file `/home/green/ImageRust/tests/test-path.rs` found to be present in multiple build targets:
  * `bin` target `test-path`
  * `integration-test` target `test-path`
    Finished dev [unoptimized + debuginfo] target(s) in 0.13s
     Running `target/debug/test-path`
"/home/green/ImageRust/.venv/bin/python"
Python path configuration:
  PYTHONHOME = '/home/green/ImageRust/.venv/bin/python'
  PYTHONPATH = '/home/green/ImageRust/.venv/lib/python3.11/site-packages'
  program name = 'python3'
  isolated = 0
  environment = 1
  user site = 1
  safe_path = 0
  import site = 1
  is in build tree = 0
  stdlib dir = '/home/green/ImageRust/.venv/bin/python/lib/python3.11'
  sys._base_executable = '/home/green/ImageRust/.venv/bin/python3'
  sys.base_prefix = '/home/green/ImageRust/.venv/bin/python'
  sys.base_exec_prefix = '/home/green/ImageRust/.venv/bin/python'
  sys.platlibdir = 'lib'
  sys.executable = '/home/green/ImageRust/.venv/bin/python3'
  sys.prefix = '/home/green/ImageRust/.venv/bin/python'
  sys.exec_prefix = '/home/green/ImageRust/.venv/bin/python'
  sys.path = [
    '/home/green/ImageRust/.venv/lib/python3.11/site-packages',
    '/home/green/ImageRust/.venv/bin/python/lib/python311.zip',
    '/home/green/ImageRust/.venv/bin/python/lib/python3.11',
    '/home/green/ImageRust/.venv/bin/python/lib/python3.11/lib-dynload',
  ]
Fatal Python error: init_fs_encoding: failed to get the Python codec of the filesystem encoding
Python runtime state: core initialized
ModuleNotFoundError: No module named 'encodings'

Current thread 0x00007f6ccb7e3800 (most recent call first):
  <no Python frame>

My Environmental Information

• OS version: Ubuntu 22.04.3
• rust version: 1.74.0
• pyo3 version: 0.20.0
• python version: 3.11.6

[cguentherTUChemnitz]

I run over and over into the same problem as soon as i try to use nix flakes or devenv.nix to manage the development environment:

devenv.nix:

{ pkgs, lib, config, inputs, ... }: let
in {
  languages.python = {
    enable = true;
    venv.enable = true;
    venv.requirements = ''
      numpy
    '';
  };

  languages.rust = {
    enable = true;
    channel = "stable";
  };
}

rust test:

    #[test]
    fn numpy_import() {
        Python::with_gil(|py| {
            py.import_bound("numpy").unwrap();
        })
    }

result

called `Result::unwrap()` on an `Err` value: PyErr { type: <class 'ImportError'>, value: ImportError('Error importing numpy: you should not try to import numpy from\n        its source directory; please exit the numpy source tree, and relaunch\n        your python interpreter from there.'), traceback: Some(<traceback object at 0x7ffff6f1af40>) }

This is totally unclear what happens here. I can confirm that i run in the test command inside the venv. The venv is able to import numpy correctly when i just run it directly from the interactive python session: python -c "import numpy", but an cargo test fails in the same environment.

[epiloguess]

same problem here，did you found out what happened ？

[cguentherTUChemnitz]

I worked around at the moment by using pythonPackages.numpy or pythonPackages.numpy_2 instead of the pip installed package. Nevertheless, the problem seems to hit just some python packages. In my case, all other python packages can be imported from the pip installation, just numpy is not working there

[kirito41dd]

My solution on Windows:

pip uninstall numpy
pip install numpy==1.26.4