LINQ in Practice · Lesson 13

DistinctBy and explicit identity keys

Remove repeated entities using a key rather than whole-object equality.

How it works

Two records can describe the same business identity while containing different other values. The sample has two user records whose names differ only in case. Whole-record equality includes the Id, so it would treat those records as different. DistinctBy lets the query choose Name as the deduplication key instead.

StringComparer.OrdinalIgnoreCase makes the name comparison explicit and appropriate for this example's identifier policy. The retained records have identifiers one and three. An explicit OrderBy supplies presentation order before the identifiers are joined into output. Identity selection and display ordering are separate decisions, and keeping both visible makes the query easier to review.

Deduplication also needs a conflict policy. This example accepts the retained representative from the source enumeration. That is not automatically the newest, most complete, or most authoritative record. If two rows disagree about important data, silently keeping one may hide an import error. Decide whether duplicates should be rejected, merged, or ranked before selection.

Avoid normalizing identifiers by changing unrelated fields or relying on the device's current culture. A comparer can express case-insensitive membership without rewriting the stored display name. Human-language names may need a different policy, so document why these particular strings are treated as identifiers in the first place.

Practice by adding a second Sam record and a new user whose name contains surrounding spaces. Predict which rows are considered equal under the existing key function. Then build a separate normalized key using Trim and explain whether blank normalized names should be rejected before the distinct operation.

Finally, produce a duplicate report with GroupBy before removing anything. Show each repeated key and its original identifiers. Comparing that diagnostic pipeline with the final deduplicated output helps distinguish data cleanup from data loss, and it gives a caller evidence when the chosen representative is not the row they expected.

A working example

using System;
using System.Linq;
using System.Collections.Generic;
using System.Globalization;
using System.Text;
using System.Text.Json;
using System.Text.Json.Serialization;
using System.Threading;
using System.Threading.Tasks;
using System.Runtime.CompilerServices;

var users = new[] { new User(1, "Ada"), new User(2, "ada"), new User(3, "Sam") };
var unique = users.DistinctBy(user => user.Name, StringComparer.OrdinalIgnoreCase)
    .OrderBy(user => user.Id);
Console.WriteLine(string.Join(",", unique.Select(user => user.Id)));
record User(int Id, string Name);

Expected output

1,3
Try this
Select the identity key and its comparer deliberately, then order the displayed result.