NOCLAU GUIDE
Unicode Control Characters Guide
Identify Unicode control and formatting characters.
Control characters are a reason to investigate text carefully, not a reason to erase every nonprinting character. Some support ordinary document structure, while others influence direction or reflect data copied from another system. This guide explains how to investigate noclau's control findings, preserve context, and hand off a useful example when the issue needs a specialized editor. It also explains a deliberate limit of the cleaner: control and directional marks are reported but left unchanged by the automatic cleaning rules.
Scroll the table sideways to see every column.
| Text context | Automatic behavior | Next check |
|---|---|---|
| Ordinary tabs and newlines | Preserved; not special findings | Review document structure |
| Directional marks | Reported and preserved | Check language and display order |
| Control in a restricted identifier | Reported and preserved | Check field requirements before manual edit |
| Source code | No security certification | Use an appropriate editor and project checks |
Begin with the operation that behaves strangely
Record whether the problem is display order, a rejected field, a broken import, or unexpected spacing. These symptoms require different tests. A number appearing on the wrong side of mixed-language punctuation may require direction-aware layout review. A data import rejecting a control code may require a check of the receiving format. A cleaner cannot infer either requirement from a character name alone.
Keep a short sample containing the text before and after the finding. Removing all surrounding words can destroy the context that made the behavior visible. If the original contains private information, create an invented sample with the same direction changes and punctuation. Include a description of what the destination should display, because a screenshot without an expected result is often ambiguous.
Understand the scanner's coverage
noclau reports selected C0 and C1 controls, DEL, and several directional marks and formatting controls. The scanner intentionally excludes ordinary tab, line feed, and carriage return from its control findings. You can visualize tabs in the preview, and line breaks remain in the input and result. This distinction prevents normal multiline text from being described as a collection of errors.
Coverage is selective rather than exhaustive. The report does not list every Unicode property or perform a complete security assessment. A finding may use the general label Control / directional mark instead of an individual formal name. The code point is the more precise identifier to include in a bug report. If the report is empty, continue investigating a reproducible problem rather than assuming every aspect of the text is valid.
Distinguish storage order from display order
Text is stored as a sequence, but multilingual display can place parts of that sequence in a different visual order. The Unicode bidirectional algorithm describes how direction affects presentation. Explicit marks and controls can participate in that process. An unfamiliar mark in mixed Arabic or Hebrew and English text may have a useful purpose, so its presence is not enough to classify it as unwanted.
For your editing workflow, compare the logical sequence shown by an appropriate editor with the rendered text in the actual destination. Copying the same sample into several applications may reveal different handling, but does not by itself tell you which result is intended. Ask a reader familiar with the language to check meaningful punctuation and number placement before approving a change.
Use a constructed example without deleting evidence
Consider an invented field with the sequence A[U+202E]BC. The bracket label stands for a directional control. noclau can report U+202E, but automatic cleaning leaves it in the result. That unchanged output is intentional: the utility avoids silently rewriting the direction of an arbitrary text fragment. Do not interpret a zero-removal result as a failure to inspect the character.
If the receiving field is documented to require a simple Latin identifier, an editor with explicit code-point selection can help remove the control from a copy. Retest that field with the intended value ABC. For a paragraph containing mixed writing directions, the same deletion may be inappropriate. Write down the field's requirements and use them to justify the edit instead of generalizing from a single example.
Treat source code as a separate review task
A general prose cleaner is not a source-code security tool. If directional controls appear in program text, preserve the original and inspect the file with an editor that makes those controls visible. Review the actual syntax and use the project's normal tests and security checks. A text tool does not determine whether displayed code matches a reader's interpretation or whether an identifier is deceptive.
Avoid copying confidential source or credentials into support messages. A tiny invented sequence with the same code point is usually enough to report a display problem. Also avoid bulk cleanup of strings, fixtures, or language-specific examples that intentionally contain controls. Those values may be test data, and changing them can remove the very behavior a test is supposed to exercise.
Review imports and structured data deliberately
For CSV, tab-separated values, or configuration files, identify which characters are delimiters and which belong to field content. A tab may separate columns, a newline may separate records, and a quoted field may contain a legitimate line break. noclau preserves these ordinary characters, but it does not parse the file format or validate its structure.
If an import fails, isolate one record and compare its expected columns before and after any manual edit. Check escaping, quotation marks, and the receiving application's encoding expectations. A rejected import is not automatically a Unicode-control problem. When a parser gives a line or column number, keep that information alongside noclau's code-point position rather than assuming the two numbering systems are identical.
Build an acceptance check for a manual change
A useful acceptance check has three parts: the intended string or layout, the exact code-point change, and the operation that now succeeds. For a form field, submit the edited test value. For a paragraph, review the punctuation and reading order in the publishing system. For a data file, validate a small record before importing a larger set.
Keep the original and edited examples together until the result is accepted. If the change does not solve the symptom, restore the original rather than stacking more speculative edits. Reporting and preserving a control is often the correct outcome. The purpose of inspection is to make the next decision informed, not to force every document into the same simplified character set.
References and further reading
Examples on this page are constructed demonstrations, not measurements of how often a platform inserts characters.