Remove Duplicate Lines

Remove repeated lines while preserving the first occurrence and original order.

18 / 50,000 characters · Text is processed on this device.

Result

Removed lines: 1

Remove Duplicate Lines: Examples & Matching Rules

Short answer

Remove Duplicate Lines keeps the first occurrence of each line and removes later repetitions without sorting. By default case and surrounding spaces matter. Optional controls ignore case, trim surrounding spaces or remove empty lines.

Formula

Output = first occurrence of each unique line in input order

Remove Duplicate Lines keeps the first occurrence of each line and removes later repetitions without sorting. By default case and surrounding spaces matter. Optional controls ignore case, trim surrounding spaces or remove empty lines.

Worked example

Your text
apple banana apple
  • Cleaned text: apple banana
  • Removed lines: 1

Clean lists without changing their order

Repeated entries commonly appear when combining lists, copying spreadsheet columns or merging notes. This tool treats each line as one entry. In apple, banana, apple on three separate lines, the second apple is removed and banana stays second.

Understand the three controls

Case sensitive distinguishes “London” from “london”. Disable it only when capitalization is irrelevant. Trim surrounding spaces treats “apple” and “ apple ” as the same and writes the kept line without those outer spaces. Interior spaces are not changed. Remove empty lines discards lines containing only spaces or tabs.

The removed-line total includes both duplicate lines and empty lines discarded by the empty-line option. With no options changed, a whitespace-only line and a truly empty line can be different entries.

Practical examples

For a mailing list of case-insensitive labels, disable Case sensitive and enable trimming. For programming identifiers or case-sensitive file paths, retain case sensitivity. For addresses or CSV rows containing quoted line breaks, use a format-aware editor: a physical line is not necessarily a complete record.

Verify before replacing your original

Check the removed count, scan the first and last entries and retain the original file. Windows, Unix and older carriage-return line endings are accepted; output uses newline separators. The tool does not validate emails, normalize spelling, sort entries or detect approximate duplicates.

Find & Replace can standardize a phrase first. Text Diff Checker shows what changed. Word Counter counts the resulting text.

Comparison

ScenarioInputResult
Matching behaviorapple banana appleApple and apple remain separate by default; disabling Case sensitive keeps only the first spelling.

Common mistakes

  • Expecting alphabetical sorting when the original order is preserved.
  • Ignoring case for identifiers where capitalization matters.
  • Trimming spaces that are meaningful in source code.

Checklist

  • Keep a copy of the original text.
  • Check the options and result before copying.
  • Use smaller sections for long text.
Michael Torres

Michael Torres

Co-Founder & Chief Editor

Michael Torres writes about personal finance, loans, and savings for the financial blog The Money Leads (opens in a new tab).

Plain-text processing only. Browser and application counting rules may differ; verify important edits against your original.

Frequently Asked Questions

Is the original order preserved?

Yes. The first matching line stays in its original position.

Are uppercase and lowercase separate?

Yes by default. Disable Case sensitive to merge them.

Can spaces be ignored?

Enable Trim surrounding spaces. This trims the saved output lines as well as their comparison keys.

Are blank lines removed?

Repeated identical blank lines are deduplicated. Enable Remove empty lines to remove all whitespace-only lines.

Does this remove duplicate words inside a sentence?

No. It compares complete lines, not individual words.

Can I download the clean list?

Yes. Download text saves the result as a UTF-8 text file on your device.

Sources & references

Formulas are tested against hand-checked answers. How we check our calculators.

Last updated: November 15, 2026