Fuzzy Duplicate Finder

1. Drop a file into the upload area or click browse files.
2. Choose the columns to compare. All discovered columns start selected; use Select all or Select none to change the selection.
3. Set Similarity method, adjust Similarity threshold, and enable Case sensitive when capitalization should affect the score.
4. Review each candidate group, choose the row marked Keep, and select field values in the optional merge controls.
5. Click Keep selected, remove rest or Not a duplicate. Then choose Export resolved dataset (CSV) or Export group report (JSON).

How similarity settings find near-duplicate records

Similarity method has character-level Levenshtein matching for typos, word-level Token Jaccard matching for reordered text, and Combined (Levenshtein + token overlap). Combined averages the two scores. Similarity threshold starts at 80% and accepts values between 40% and 99%.

Input
Name,Email
Alice Smith,[email protected]
Alicia Smith,[email protected]

Candidate group
similarity 92%
status: Awaiting review

With Case sensitive off, the tool trims values, reduces repeated whitespace to one space, and ignores letter case. When it is on, case differences affect the score. The comparison converts values to text.

Which files Fuzzy Duplicate Finder reads

Drop a CSV, TSV, JSON, JSONL, XLSX, or XLS file into the page. CSV and TSV data use the first row as column names. JSON accepts an array of records or an object containing a nested array of objects. JSONL reads each nonblank line as a JSON value. Spreadsheet input uses the first worksheet.

• Files with more than 1,500 rows show a warning because matching may take longer.
• Files above 6,000 rows are rejected before matching.
• At least one comparison column must remain selected. A pair with blank values in every selected column receives no match score.

Why duplicate groups can chain

The tool compares pairs of records and joins connected matches into groups. If row A matches row B, and row B matches row C, all three can appear in one candidate group. Review the whole group before choosing Keep selected, remove rest.

Both-empty values in a selected column are skipped for that pair. A resolved group keeps the selected base row and removes its other members in the exported dataset. The merge controls can take a field value from any member of the group. Not a duplicate leaves the rows in place, while Undo / re-review reopens the group.

Where duplicate record cleanup comes up

• A customer CSV contains Alice Smith and Alicia Smith with the same email address. Compare the name and email columns, inspect the group, and keep the preferred row.
• Product descriptions contain the same words in a different order. Token Jaccard can group those rows for review.
• JSONL records use inconsistent capitalization and spacing. Leave Case sensitive off, review the candidates, and export the resolved dataset as CSV.

When to use another dataset analysis tool

Use Fuzzy Duplicate Finder when similar text rows need a review decision. The parent Dataset Analyze Tools category includes Dataset Profiler for row, column, null, unique, duplicate, and data-quality summaries. Choose Correlation Matrix Generator for pairwise correlations between numeric columns, or Outlier Detector for unusually high or low numeric values.

Frequently asked questions