0 lines · 0 characters
0 lines · 0 characters · 0 removed, 0 kept, 0 line(s) had duplicates
Delete duplicate lines and keep the original order
This tool removes duplicate lines from text and keeps each line once, in its original order. Paste a list or open a file, choose how lines are compared, and copy or download the cleaned result.
Last updated:
How it works
Paste or open a list
Paste lines, open a .txt, .csv or .log file, or load one from a URL.
Decide what counts as a duplicate
Match case or ignore it, ignore spaces at the ends of lines or inside them, and choose whether the first or the last copy survives.
Check and save
The result updates as you type. The table under it lists every removed line with its line number, and the most repeated lines with their counts.
Features
Three ways to deduplicate
Remove repeats anywhere in the text, only adjacent repeats (like the Unix uniq command), or every line that occurs more than once so that only unique lines are left.
Keep first or last occurrence
Keeping the last copy suits logs and settings files, where a later line overrides an earlier one.
Comparison you can trust
Case-insensitive matching uses full case folding, so STRASSE and straße are one line. Unicode normalization treats an é typed as one character and as e plus an accent as equal.
Blank lines on your terms
Deduplicate blank lines like any other line, keep them all to preserve paragraphs, or drop them.
Limits & privacy
Pasted text and opened files can be up to 200 MB. Text up to 256 KB is processed on every keystroke; longer text is processed in a background worker, and files above 8 MB are streamed without loading them into the editor. With “Last occurrence” or “Remove all copies” the whole list is held in memory.
Line endings (LF, CRLF or CR) and a final line break are preserved.
Nothing is uploaded: the text never leaves your browser. See the privacy page.
FAQ
How do I remove duplicate lines from a text file?
Open the file or paste its content. Duplicates disappear at once; press Download to save the cleaned file.
Does it change the order of my lines?
No. Lines stay in their original order; only repeats are removed. If you also want the list sorted, use Sort Lines with its Remove duplicates option.
How do I remove duplicates but ignore upper and lower case?
Turn Case sensitive off. "Apple", "apple" and "APPLE" then count as one line, and the copy you chose to keep is written exactly as it was typed.
Why are two lines that look the same not removed?
They differ in something invisible: a trailing space, a tab, a no-break space or a different accent encoding. Turn on the space options, or clean the text with Remove Extra Spaces first.
How can I keep only lines that are not repeated?
Set Duplicates to "Remove all copies". Every line that occurs twice or more is removed entirely – handy for finding entries that appear in only one of two merged lists.
Can it handle a file with millions of lines?
Yes. Files up to 200 MB are read in 4 MB chunks in a Web Worker, so the page stays responsive. The result is downloaded as a file and previewed on the page.