Fix symlinked directories not being watched

[auwi-nordic]

Closes #35173 and #27263

Code was written by AI agent. Test was reviewed and accepted as is. Fix was reviewed thoroughly and some fixes was done to improve code quality.

Release Notes:

• Fixed: Symlinked directories not not being watched

[probably-neb]

Hey, thanks for the contribution, and sorry for the delay in reviewing.

I'm a bit concerned about the performance implications of this fix, iterating all workspace entries and taking the first one that matches does not seem to be to be a robust or performant solution.
Could we instead just keep a hash map around of canonical path -> symlink path? Given that this has never worked I assume this is an infrequent case, so the memory overhead is worth it to avoid having to scan the full worktree, and potentially select the wrong path in the case of nested symlinks, or multiple symlinks pointing to the same directory

[auwi-nordic]

@probably-neb I've updated the pull requests with some improvements now

1. Did a rebase on main and fixed conflicts
2. Performed an AI assisted review of the code.. the issue you bring up was actually discovered by the AI as well so yeah, probably good to address
3. Implemented the performance improvement. A canonical_path_to_symlink hash map field was added to LocalSnapshot to track the canonical path mapping.
4. Simplified the code a bit
5. Simplified some comments / assertion messages

While testing this fix again, I realized it's not updating symlinks within the project. I don't remember why I only implemented fix for symlinks pointing outside the project. And the issue this PR claims to close doesn't say anything about whether the symlinks point inside or outside. So I added fix for the inside case as well.

[auwi-nordic]

BTW, I also did some manual checking that the fix interacts nicely with #51382 , so when a symlinked directory is expanded in project tree, file changes are also visible in the index. E.g. a newly added file in a symlinked directory is searched by "Find in Folder"

[auwi-nordic]

Did some more reviewing and added some more cleanup, plus another test "test_symlinked_dir_removal_cleans_up_routing"

Now I'm wondering if the three tests added by this PR should be combined into one. I could do that if it's preferable.

Otherwise I think I'm done with this round of improvements.

I actually ran one more review, and some potential issues came up, but I think at this point the issues are nitpicks

1. Consider longest-prefix matching in the routing block or document the nested symlink limitation (If symlinks are nested — e.g., /real/sub and /real are both canonical keys — an event for /real/sub/foo.rs could match either entry. With HashMap, which one "wins" is effectively random.)
2. Use .log_err() on the canonicalize call instead of silently discarding failures.
3. Optionally, move the canonicalize call outside the state lock scope in scan_dir to reduce lock contention.

[probably-neb]

Hey @auwi-nordic, thank you for being so responsive to the review comments, and taking the time to try and solve this. I just realized that #50746 (which I am also assigned to review) is solving the same problem. The solution in the other PR is very similar, but it canonicalizes events at the background layer instead of the worktree snapshot layer. I think this is a better layer to solve the problem at, so I'm going to close this PR and merge that one. Apologies for not realizing the conflict sooner

[auwi-nordic]

No worries @probably-neb , just happy it's getting resolved