Remove Duplicate Lines
Clean up lists of emails, keywords, URLs or log entries by removing repeated lines while keeping everything in its original order.
Your text stays in this browser tab. Nothing is uploaded, saved or logged.
How the Remove Duplicate Lines works
Lines are compared one by one. By default the first copy of each line is kept and later copies are removed, so the order of the remaining lines never changes. “Only lines that appear once” keeps lines with no duplicates at all, and “Only lines that are repeated” keeps a single copy of each line that appears more than once.
Comparison is case-insensitive unless you turn on “Case sensitive”, and spaces at the start and end of a line are ignored when comparing (the kept line is left as typed). Accented letters written in different but equivalent Unicode forms, such as é as one character or as e plus an accent, are treated as the same.
Empty lines are kept by default so the layout of your list is not disturbed. Turn that option off to treat blank lines like any other line and keep only the first. Windows, Mac and Linux line endings are all recognised and the output uses standard line breaks.
How to use it
- Paste your list, one item per line.
- Choose which lines to keep and whether case matters.
- Copy the result or download it as a text file.
Example
Case and surrounding spaces are ignored, so only three distinct lines remain.
Input
apple Banana apple banana cherry Apple cherry
Result
apple Banana cherry
Frequently asked questions
Does it sort the lines?
No. The first occurrences stay in the order they appeared. To sort afterwards, use Sort Lines Alphabetically.
Can I see only the lines that were duplicated?
Yes. Choose “Only lines that are repeated” to get one copy of each line that appears more than once. “Only lines that appear once” gives you the reverse.
How large a list can it handle?
It runs in your browser and handles hundreds of thousands of lines comfortably. Very large files depend on your device’s memory.