Remove duplicate lines: keep the first of each line, or only unique or duplicate lines
0Lines in
0Removed
0Remaining
This tool removes duplicate lines from any list. Paste your text, and every line that has already appeared is dropped, so each line is kept once, in the order of its first appearance. It is useful for email lists, keyword lists, log files, URLs, CSV columns and anything else with one item per line.
Choose how lines are compared. Case-sensitive decides whether Apple and apple match. Trim whitespace ignores spaces and tabs around each line, and Ignore blank lines leaves empty lines alone. Two other modes go beyond plain deduplication: Only unique lines drops every line that repeats, and Only duplicates shows just the lines that appear more than once.
The panel under the text shows how many lines went in, how many were removed and how many remain, plus the most repeated lines with their counts. Everything runs in your browser, in a background thread, so a list of a million lines does not freeze the page. Your text is never sent anywhere or saved.
It reads your text one line at a time. The first time a line appears it is kept, and every later copy of that line is dropped. The lines that stay keep the order they had in your text.
With case-sensitive on, Apple and apple are different lines. Turn it off and they count as the same, and the first one is kept. Trim whitespace ignores spaces and tabs at the start and end of each line when comparing, and also removes them from the lines in the result.
Removing duplicates keeps one copy of every line. Only unique lines is stricter: a line that appears more than once is dropped completely, including its first copy, so you are left with the lines that never repeat.
Choose Only duplicates. The result lists each repeated line once, in the order it first appeared. The panel under the text also shows the most repeated lines with how many times each one appears.
Blank lines are left alone. They are never treated as duplicates, so your spacing and paragraph breaks stay where they are. Without it, blank lines count like any other line and only the first one is kept.
Yes. The work runs in a background thread, so the page stays responsive. A list of a million lines is processed in a fraction of a second on most computers. Very long results are shown in part in the box to keep the page fast, and Copy and Download always give you the full result.
No. Everything happens in your browser and your text is never uploaded or stored. Only the option choices are kept in the page address.