Ignore some extra chars in no-combining search #2929
Merged
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
dae
reviewed
Jan 4, 2024
rslib/src/text.rs
Outdated
| .nfkd() | ||
| .filter(|c| !is_combining_mark(*c)) | ||
| .collect::<String>() | ||
| .into() | ||
| .collect::<String>(); |
There was a problem hiding this comment.
Use of phf is nice! I think the step below could be done more efficiently though, by using .map() here to transform the unwanted characters, instead of modifying the string below. You should be able to look up each character instead of having to iterate over the phf, too.
There was a problem hiding this comment.
(might be more efficient to do this in a 'for char in ... { output.push(char) or output.push_str(repl) }' than using flat_map)
dae
reviewed
Jan 5, 2024
|
Thanks Abdo! |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Closes #2926
I just took the translation list in WordPress page and kept only letters that don't change after NFKD normalization.