Rename Refactor does not work for packages with __init__ and PYTHONPATH

[bschnurr]

@schperplata commented on Thu Dec 17 2020

Environment data

• VS Code version: 1.52.1
• Extension version (available under the Extensions sidebar): 2020.12.424452561
• OS and version: WIN 10
• Python version (& distribution if applicable, e.g. Anaconda): 3.6, 64 bit
• Type of virtual environment used (N/A | venv | virtualenv | conda | ...): N/A
• Relevant/affected Python packages and their versions: rope, latest version
• Relevant/affected Python-related VS Code extensions and their versions:
• Value of the python.languageServer setting: Pylance

Expected behaviour

Refactor works in both ways regardess of how module import is implemented.

Actual behaviour

We have a following setup:

1. Project script(s) on a location A

2. Packages (internal libraries) on a locations B, C, D, ...
   Each of these packages have its own __init__.py file, where relevant modules are imported. User use these libs as B.<something>, B.<somethingElse>

3. We have VSC workspace on a location A, but we open it with a batch file, that sets PYTHONPATH so all libs (B, C, D) are available to a project on A.

The problem is that refactoring does not work in both ways, meaning:

• we can navigate to func definition, for example B.something.func() and refactor it there. This works, all relevant locations in VSC workspace are fixed, including scripts in A that use B.something.func()
• however, we can't refactor B.something.func() directly from script from A. VSC just show No results and that is it.

I assume this is not as expected, since, for example,
refactorTest.zip

PyCharm seems to handle this scenario somehow.

Steps to reproduce:

• Open appended workspace via: SET_PYHON_PATH_OPEN_VSC.bat
• Navigate to proj/app.py -> useLib(). Refactor lib.m1.func1() func1 to whatever. Will fail.
• Navigate to func1() definition with F12. Refactor it to whatetever, will succeed.

Logs

No log statement is generated. However, GUI pop-up is issued, showing only: No results.

System Info

Item	Value
CPUs	Intel(R) Core(TM) i7-6820HQ CPU @ 2.70GHz (8 x 2712)
GPU Status	2d_canvas: enabled flash_3d: enabled flash_stage3d: enabled flash_stage3d_baseline: enabled gpu_compositing: enabled multiple_raster_threads: enabled_on oop_rasterization: disabled_off opengl: enabled_on protected_video_decode: unavailable_off rasterization: enabled skia_renderer: disabled_off_ok video_decode: enabled vulkan: disabled_off webgl: enabled webgl2: enabled
Load (avg)	undefined
Memory (System)	15.92GB (5.24GB free)
Process Argv	--crash-reporter-id 919e0ed9-46f3-4ad4-9e28-f35755bf8266
Screen Reader	no
VM	0%

A/B Experiments

vsliv368cf:30146710
vsreu685:30147344
openlogontheside:30221877
python763:30178808
python383cf:30185419
vspor879:30202332
vspor708:30202333
vspor363:30204092
python504:30227505
vswsl492:30211401
wsl2prompt:30224612
pythonvsdeb440:30237827
unusedpromptcf:30224611
folderexplorer:30224614
openfilemenu:30224647
pythonvsded773cf:30236630
pythonvspyt859:30237612

---

@brettcannon commented on Thu Dec 17 2020

Thanks for the bug report, but unfortunately we need some more information from you in order to triage this issue appropriately. Please edit your opening comment by providing a filled-in copy of our issue template. Once you are done please leave a comment to that effect.

---

@schperplata commented on Sat Dec 19 2020

@brettcannon Done.
However, this really wasn't necessary and almost make me close the issue and walk away. Don't bother the users that are willing to take time and report issues, especially like this one, that have everything prepared (including example). I've reported it directly from VSC and there were no info missing - only reformatting was necessary. For example, VSC Python extension version was written in a default output generated with VSC, but not in a "Environment data" section as your template demands. Maybe I am wrong, but I don't think any info was missing - correct me if I am wrong. I know VSC is free and I am truly gratefull for such a great tool, but let's just assume we are all very busy and try to save our time where possible, that is, including reporting issue.

---

@brettcannon commented on Mon Dec 21 2020

@schperplata actually the absolutely critical bit was what language server you were using which you did not specify.

And while we do appreciate you taking the time to report issues, do also note that our issue volume isn't small, so we need to be somewhat aggressive in closing issues as the vast majority of people who skip our template never give us the info we need. It's easier for us to reopen issues for those that do take the time to fill in the info than it is to leave things open and wait for a reply (which is what used to happen to us).

Regardless, Pylance handles refactorings so I am transferring your issue to them.

---

@bschnurr commented on Mon Dec 21 2020

Can you clarify which commands you are using when you say "refactor"? (maybe a pic)

I have a feeling you are using Rope commands and not Pylance.

---

@schperplata commented on Tue Dec 22 2020

By refactor I mean pressing F2 on a function/class/variable name. I always thought that underlying engine was indeed rope library.


In case of a function name refactor, it only show No results.

---

@brettcannon commented on Wed Dec 23 2020

@schperplata sorry for the back-and-forth on this. We plan to move to using Jedi for refactoring (#11333) which will then cause us to drop support for Rope (#10440). I will leave this issue open, though, as a reminder of the bug for anyone else who comes across it.

---

@komodovaran commented on Sat Dec 26 2020

Is there a workaround in the meantime? Use a different language server altogether and set up Pyright separately? Or can another plugin be added on top of current setup to patch it up?

---

@brettcannon commented on Mon Jan 04 2021

@komodovaran I would use Pylance.

---

@bschnurr commented on Wed Jan 06 2021

Sorry. I was mistaken. There is a Rename in Pylance and the issue is most likely due to lazy loading. We'll take the bug back

[jakebailey]

I think this issue predated our support for PYTHONPATH; can anyone on this thread retry to see if things work as expected now?

@schperplata @komodovaran

[schperplata]

I didn't test the exact scenario as described above, since we have already moved to venv (without batch files and PYTHONPATH) and I have quite a busy schedule ATM. However, A quick test showed the same No results popup: just try to refactor (F2) any function from a library located in site-packages folder.

[jakebailey]

I'm moderately confused... Are you saying that you are trying to refactor a method coming out of site-packages? We treat that as library code and bail if we see that you are trying to modify stuff like that, as we want site-packages to be treated as read-only. I'm not aware of tooling that chooses to let someone modify library code like that, but maybe I'm wrong and some older tooling did... That strikes me as dangerous.

[schperplata]

Yeah, absolutely true. My bad. I "tested" it that way because adding some directory to PYTHONPATH effectively behave the same way as if package was placed in a site-packages, at least from user perspective. So, feel free to ignore the "test" above.
I've checked now in the same way as original issue report (refactor sources that are added to VSC wks via PYTHONPATH) and the behaviour is the same: No results.

[heejaechang]

currently, we don't support refactoring across multiple workspaces nor refactoring on library files (files not under workspace root, in another words, files that don't match "include" - https://github.com/microsoft/pyright/blob/main/docs/configuration.md#main-pyright-config-options - regardless auto/explicit include)

we probably need decision on this. regardless this is enhancement request.

[heejaechang]

multi workspace support is dup of #159