Remove Duplicate Lines: Examples & Matching Rules
Short answer
Remove Duplicate Lines keeps the first occurrence of each line and removes later repetitions without sorting. By default case and surrounding spaces matter. Optional controls ignore case, trim surrounding spaces or remove empty lines.
Formula
Output = first occurrence of each unique line in input order
Remove Duplicate Lines keeps the first occurrence of each line and removes later repetitions without sorting. By default case and surrounding spaces matter. Optional controls ignore case, trim surrounding spaces or remove empty lines.
Worked example
- Your text
- apple banana apple
- Cleaned text: apple banana
- Removed lines: 1
Clean lists without changing their order
Repeated entries commonly appear when combining lists, copying spreadsheet columns or merging notes. This tool treats each line as one entry. In apple, banana, apple on three separate lines, the second apple is removed and banana stays second.
Understand the three controls
Case sensitive distinguishes “London” from “london”. Disable it only when capitalization is irrelevant. Trim surrounding spaces treats “apple” and “ apple ” as the same and writes the kept line without those outer spaces. Interior spaces are not changed. Remove empty lines discards lines containing only spaces or tabs.
The removed-line total includes both duplicate lines and empty lines discarded by the empty-line option. With no options changed, a whitespace-only line and a truly empty line can be different entries.
Practical examples
For a mailing list of case-insensitive labels, disable Case sensitive and enable trimming. For programming identifiers or case-sensitive file paths, retain case sensitivity. For addresses or CSV rows containing quoted line breaks, use a format-aware editor: a physical line is not necessarily a complete record.
Verify before replacing your original
Check the removed count, scan the first and last entries and retain the original file. Windows, Unix and older carriage-return line endings are accepted; output uses newline separators. The tool does not validate emails, normalize spelling, sort entries or detect approximate duplicates.
Find & Replace can standardize a phrase first. Text Diff Checker shows what changed. Word Counter counts the resulting text.
Comparison
| Scenario | Input | Result |
|---|---|---|
| Matching behavior | apple banana apple | Apple and apple remain separate by default; disabling Case sensitive keeps only the first spelling. |
Common mistakes
- Expecting alphabetical sorting when the original order is preserved.
- Ignoring case for identifiers where capitalization matters.
- Trimming spaces that are meaningful in source code.
Checklist
- Keep a copy of the original text.
- Check the options and result before copying.
- Use smaller sections for long text.

Michael Torres
Co-Founder & Chief Editor, Calculators PercentageCo-Founder & Chief EditorMichael Torres writes about personal finance, loans, and savings for the financial blog The Money Leads (opens in a new tab).
Plain-text processing only. Browser and application counting rules may differ; verify important edits against your original.
