Handle editable installs with .pth files #5457

Describe the Bug

In #2646 we tried to add support for .pth files that provide editable package installs (where the .pth file is a link in PYTHONPATH to some code that's actually part of the project).

This is a valuable feature, but also seems to be quite nontrivial, I think because project structure is already complex and prone to bugs, and the .pth files make this worse. They behave like symlinks, so they have all sorts of complications like being able to be re-pointed, and then they link into directories where behaviors are already hard to keep track of.

I set AI agents on trying to make the original PR work, and wound up with a stack of over 30 diffs handling various edge cases. This to me is an architecture smell - we cannot build a reliable type checker on a pile of edge cases. As a result, my instinct is that we need to carefully model project structure, work out how editable installs need to work, possibly refactor quite a bit of existing code, and implement it in a reasonable amount of logic.

I suspect this is going to be a hard thing for an external contributor to do. I'm assigning myself an @connernilsen to claim this issue - if you are interested, feel free to reach out but be aware this might be highly nontrivial.

Sandbox Link

No response

(Only applicable for extension issues) IDE Information

No response