Skip to content
Allin
Sign up · Sign in

Remove duplicate words

Keep only the first occurrence of each word in your text.

Related tools

All Transform & clean tools →
Remove duplicate linesDelete repeated lines while keeping the original order of the first occurrences.Alphabetize wordsSort all the words in your text into alphabetical order.Duplicate each lineRepeat every line a chosen number of times.Find duplicate linesShow only the lines that appear more than once in your text.Remove repeated charactersCollapse runs of the same character into one (helllo → helo).Remove accentsStrip accents and diacritics from text, turning é into e and ü into u.Remove blank linesDelete all empty or whitespace-only lines from your text.Remove emojisStrip all emojis and pictographs from your text.

Open Remove duplicate words and you get an answer straight away, with no account to create. It sits under Transform & clean in our catalogue, alongside Remove duplicate lines and Alphabetize words.

How to use it

  1. Open the tool — no signup or install needed.
  2. Enter your input or adjust the available options.
  3. Get your result instantly, then copy or download it.

Frequently asked questions

What is Remove duplicate words?

Keep only the first occurrence of each word in your text.

When would I actually use this?

Cleaning up a paste before it goes anywhere: stripping duplicates from a list, normalising case, removing accents for a filename, or sorting lines.

What is the most common mistake?

Sorting text as if it were ASCII. Accented letters land after Z in a byte sort, which puts "Zurich" before "Émile" and produces a list no reader will accept.

How is Remove duplicate words different from Remove duplicate lines?

They sit next to each other but answer different questions: Remove duplicate lines is the one to open when you need it to delete repeated lines while keeping the original order of the first occurrences. Pick whichever matches what you're starting from — both are free.

Is there a tool for the next step?

Alphabetize words is the closest one after this: Sort all the words in your text into alphabetical order.

What else is worth having open alongside it?

Duplicate each line and Find duplicate lines — they come up in the same task often enough to be worth a second tab.

Where do the figures come from?

Transformations use the browser's Unicode-aware string handling, so case changes and sorting respect the locale rather than byte order. Nothing is truncated: what comes out is the whole of what went in.

Further reading

All guides →
GuideFind and Replace: The Regex Features That Bite, Demonstrated One by OneGreedy against lazy on the same string, the dot that skips newlines, $& and $$ in the replacement, a reused /g regex that silently skips a row, and why /i knows nothing about Turkish i. Every failure run in Node, with a count-then-replace routine that catches them.ExplainerFinding Duplicates in a List Without a SpreadsheetTwo lines that look identical are often not identical. Case, a trailing space, a no-break space and two different encodings of the same accented letter were each run through the duplicate finder, and it reported no duplicates for three of the four.ExplainerDetecting the Language of a Text, and Why Short Texts FailMeasured, not asserted: 90 short real phrases across six languages, none declined and 68 right — 76%, falling to 64% under sixteen letters. Four of the wrong answers came back at 100% confidence.How-toFiltering Lines by a Pattern Without a Command LineThis is grep for people who do not use grep, with one important difference: the match is a plain substring, so a real regular expression returns an empty box and no error. Every claim here was checked by running the tool.GuideCleaning Up a List Pasted from a Spreadsheet or a PDFA paste carries characters you cannot see: no-break spaces, soft hyphens, zero-width spaces, tabs and CRLF. Four cleanup tools were run against each of them, and they use three different definitions of whitespace.ExplainerRemoving Accents Breaks Search — Until You Do It on Both SidesFolding diacritics is a normalisation step, and normalisation only works when the same function runs on the index and on the query. NFC against NFD with the code points shown, and the letters — ø, ł, ß, œ, ı — that survive the strip untouched.