How to remove duplicate lines
Each line is compared with the lines before it. The first line for each comparison value stays in Output text, and later matches are skipped. Case sensitive controls capitalization, while Trim spaces before comparing controls whether surrounding whitespace changes the comparison.
- Paste multiline text into Input text, or select Upload for a text file.
- Choose Case sensitive when capitalization should distinguish two lines.
- Leave Trim spaces before comparing selected when spaces at the start or end should not affect matching.
- Review the cleaned text in Output text.
- Select Copy to place the result on your clipboard, or select Download to save it.
How the line comparison works
The tool checks lines in their original order. With the default settings, it trims each line for comparison and compares the result in lowercase. The line written to Output text keeps its original capitalization and surrounding whitespace.
North
north
South
NORTH
SouthWith Case sensitive off and Trim spaces before comparing on, the output is:
North
SouthCase-sensitive duplicate line removal
Turn on Case sensitive when North and NORTH must remain separate. The comparison uses the trimmed line in its original case, so those two values have different keys.
Turn off Trim spaces before comparing when surrounding whitespace is part of the value. When it stays selected, North and North share a comparison key, but the first line's original spacing remains in Output text. Whitespace-only lines compare as empty after trimming.
What happens when you upload a text file
Select Upload for a .txt, .csv, .log, .md, or text/plain file. The page processes the text in the browser, and its scripts contain no request that sends the text or file contents elsewhere. If the file cannot be read, an error appears below the input.
Line breaks work as separators for Windows, Unix, and older Mac-style text. Output text uses newline characters between retained lines. The tool keeps line order and the wording of each retained line.
Choose the tool for the text you need to change
This page compares complete lines. For repeats that occur inside each line, use Remove Duplicate Words. Choose Remove Empty Lines when blank or whitespace-only lines are the problem.
If you need to join separate lines with a separator, use Merge Lines. The Text Cleanup & Filter Tools category contains these related text operations.
Frequently asked questions
How to remove duplicate lines from a list?
Paste the list into Input text, set Case sensitive and Trim spaces before comparing as needed, then read the retained lines in Output text.
How does a case-sensitive duplicate line remover work?
Case sensitive makes capitalization part of the comparison, so North and NORTH are different lines.
Does removing duplicate lines keep the first line?
Yes, the first line for each comparison value stays in Output text with its original spacing and capitalization.
Can I remove duplicate lines from a text file?
Yes, select Upload and choose a supported text file such as .txt, .csv, .log, or .md.