Skip to content

Use os.Open for Windows realpath to handle long paths #862

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 7 commits into from
May 14, 2025

Conversation

jakebailey
Copy link
Member

@jakebailey jakebailey commented May 12, 2025

My hand-written open function doesn't handle long paths, relying on the SEH bit Go normally sets. But, that bit is undocumented and seemingly isn't approved for using in releases, and isn't set in artifacts built by the Go fork used by Microsoft.

Rather than use CreateFile directly, use os.Open instead, which will handle long paths. I didn't do this initially because I thought it didn't work on directories, but it actually has worked since https://go.dev/cl/405275 a long time ago.

I don't have a test for this because I can't seem to make a link with long enough paths in the tests... ☹️

cc @qmuntal

Copy link
Contributor

@Copilot Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull Request Overview

This PR updates the Windows realpath implementation to use os.Open instead of a custom openMetadata function that relies on undocumented SEH bits.

  • Replaces CreateFile-based openMetadata with os.Open.
  • Updates resource clean-up using f.Close() and converts the file descriptor with windows.Handle.
Comments suppressed due to low confidence (1)

internal/vfs/osvfs/realpath_windows.go:14

  • [nitpick] Consider renaming the variable 'f' to a more descriptive name such as 'file' to improve code clarity.
f, err := os.Open(path)

@qmuntal
Copy link
Member

qmuntal commented May 13, 2025

The slow down comes from the fact that os.Open opens the file with the GENERIC_READ access, while you were previously using FILE_ANY_ACCESS(0). Note that the name is misleading, as passing 0 to the desired access is equivalent to just passing FILE_READ_ATTRIBUTES. When CreateFileW sees that the only desired access is FILE_READ_ATTRIBUTES, then it can open the file much faster, as it doesn't have to prepare it for doing I/O.

The problem is that the "just reading the attributes" access right does not map well with any Unix concept, so using just this access is not well supported by the os, it being Unix-centric. On the other hand, you can stretch the implementation by calling os.OpenFile(name, 3, 0) instead of os.Open(name). That would do exactly what you want: the access passed to CreateFileW will be 0, as the access mode 3 doesn't match any of the mapped accesses in here: https://github.com/golang/go/blob/a2fbb50322e716f75e9c4707afd2de725a95e14b/src/syscall/syscall_windows.go#L373-L381. On my computer, this trick opens files 3 times faster.

@jakebailey jakebailey force-pushed the jabaile/windows-realpath-longpath branch from 70d420a to c1f52a4 Compare May 13, 2025 19:45
@jakebailey
Copy link
Member Author

Hm, unfortunately opening with flag=3 causes tests to fail due to EISDIR, which I assume is because FILE_FLAG_BACKUP_SEMANTICS isn't getting set.

@jakebailey
Copy link
Member Author

Maybe I'll just use my helper for len(path) < 248 and then use plain os.Open for anything longer.

@qmuntal
Copy link
Member

qmuntal commented May 14, 2025

Maybe I'll just use my helper for len(path) < 248 and then use plain os.Open for anything longer.

That will only work for absolute paths. You can have a relative path that is just one character long and still hit the long path limit if the working directory is 247 characters or more.

Do you normally pass relative paths to realpath? If they are a minority, then you can always convert them to absolute before doing the length check, although that conversion is not free.

@jakebailey
Copy link
Member Author

They won't be relative; this FS abstraction is always absolute.

@jakebailey jakebailey added this pull request to the merge queue May 14, 2025
Merged via the queue into main with commit d1b4a91 May 14, 2025
23 checks passed
@jakebailey jakebailey deleted the jabaile/windows-realpath-longpath branch May 21, 2025 18:49
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

3 participants