NOCLAU GUIDE
Invisible Unicode Characters Guide
Learn to find and remove invisible Unicode characters.
Invisible text problems are easiest to solve when you stop treating every blank-looking mark as the same thing. A paragraph may contain ordinary spaces, a zero-width separator, a no-break space, and a directional mark, each doing a different job. This guide gives you a way to classify a suspicious character, reproduce the problem with a short sample, and decide whether a change is appropriate. The goal is a usable document with a clear editing history, not a report showing zero findings at any cost.
Scroll the table sideways to see every column.
| Finding | What to investigate | Conservative behavior |
|---|---|---|
| U+200B | Unexpected separator inside a value | Removed |
| U+00A0 | Whether a nonbreaking gap is intentional | Preserved |
| U+200D | Emoji or script joining | Preserved |
| U+202E | Directional context in a suitable editor | Reported; preserved |
Start with a symptom you can reproduce
A common investigation begins with two product names that look identical but fail an exact comparison. Keep both values in an untouched note. Then reduce the failing example to the smallest useful pair: perhaps a single name copied from a page and the same name typed manually. Do not begin by cleaning a whole customer list. A small pair makes it possible to see whether an invisible mark actually explains the mismatch.
Write down the destination and the operation that fails. Searching within a document, matching a spreadsheet cell, and submitting a username are different operations. One system may ignore certain characters while another compares the sequence exactly. If the problem disappears when you change fonts, you may be investigating rendering rather than hidden content. A useful report records both the appearance and the observed behavior.
Separate spacing from formatting
An ordinary space takes up visible width. A no-break space can look much the same while expressing a different wrapping preference. A zero-width space can be present between letters without creating a visible gap. A joiner can participate in a sequence whose appearance changes when it is removed. These distinctions matter because replacing everything with an ordinary space is as destructive as deleting everything.
Use the detector to obtain a code point, not just a count. In noclau, U+200B is reported as a zero width space, while U+00A0 is reported as a no-break space. A control or directional mark receives a different category. The table shows a few useful distinctions. It is a practical selection from the scanner, not a complete inventory of every invisible Unicode character.
Work through a small constructed example
Imagine the identifier AC[U+200B]42, where the bracketed label represents one actual zero-width character rather than literal bracket text. The intended value is AC42. Inspection should identify the extra mark after C. Conservative cleanup removes that mark and leaves AC42. Review the diff and rerun the exact comparison in the original destination. If it now succeeds, you have evidence that this character explained this particular mismatch.
Now consider the label New[U+00A0]York. Conservative cleanup leaves that space alone. Turning on Normalize spaces changes it to an ordinary space while preserving the two words. The same cleaning setting is therefore not appropriate for both examples. Before applying the second transformation to a layout, check whether keeping New and York on the same line was intentional.
Understand what noclau reports
The current scanner checks selected zero-width and formatting characters, a set of unusual spaces, three em-dash variants, and selected control and directional marks. It does not enumerate every code point in a document. Ordinary spaces and tabs can be visualized with the preview controls but are not counted as special findings. Newlines are preserved. A clean report means no characters in the supported categories were found.
The total count can be larger than the visible issue list, which displays the first thirty findings. If a long document produces hundreds of results, inspect smaller sections instead of assuming the visible badges cover the whole document. Positions are counted in Unicode code points and displayed starting at one. A visible emoji can occupy several code points, so its visual width does not predict the next reported position.
Choose a limited edit and preserve the original
For an isolated accidental separator in plain prose, conservative cleanup is a reasonable first experiment. It removes zero-width spaces, BOM characters, and soft hyphens while preserving joiners, unusual spaces, and dashes. That is a narrower choice, but it is not a guarantee of correctness for every language or document. Even a soft hyphen may have been placed deliberately for typography.
Custom removal can also remove joiners. Before using it, check for connected scripts, emoji sequences, and intentionally controlled wrapping. Directional and control findings are reported for manual review rather than automatically deleted. If you cannot explain what a character does in its surrounding text, retain it until you can inspect the context in an appropriate editor or obtain a language review.
Verify the result where it will be used
The cleaned text area is your proposed result, not the final acceptance test. Copy it into the same application where the problem occurred. Repeat the exact search, comparison, or submission that failed. Then check nearby words, punctuation, paragraph boundaries, and any links. A successful comparison does not prove that the rest of the document remained semantically correct.
If the destination reintroduces the mark, cleaning repeatedly only hides the source of the problem. Compare a typed sample, a paste from the original source, and a paste from noclau. That sequence helps locate the change in the copying path. Keep a short record of the selected rules so another editor can repeat your work without guessing what was removed.
Know when another tool is needed
If two values still differ after inspection, investigate leading spaces, line endings, letter case, accents represented by different sequences, or visually similar letters. noclau does not perform general Unicode normalization or compare confusable characters. A broken font, rich-text style, or hyperlink target may also be responsible. Those problems need a different check rather than more aggressive invisible-character removal.
Finally, a hidden character does not establish who wrote a passage or which application created it. The examples here are constructed editing cases. Use the findings to repair a concrete formatting issue, preserve required attribution, and describe the actual change. That produces a more useful result than calling every unfamiliar character a watermark.
References and further reading
Examples on this page are constructed demonstrations, not measurements of how often a platform inserts characters.