Practical .NET · Lesson 26

A small import pipeline with explicit stages

Parse a limited row format, validate values, aggregate duplicates, and report results.

How it works

A useful import pipeline separates syntax, conversion, validation, accumulation, and presentation. Each stage answers a different question. Does the row have the required fields? Can the quantity become an integer? Is that quantity permitted? Which existing total should receive it? How should the final result be displayed? Keeping those questions visible makes rejected records easier to explain.

The format in this lesson is intentionally limited: one code, one comma, and one integer quantity, with no quoting or embedded commas. A general CSV format needs a proper parser that understands its quoting rules. A small explicit format is still useful for learning the pipeline, provided the example does not claim to handle a broader grammar than it implements.

Walk through the example. Five prepared rows enter the pipeline. Three valid rows contribute to two product totals, including a repeated A code whose quantities add together. One malformed row lacks the separator, and one row has a negative quantity. Both increment the rejected count without changing totals. The final report sorts codes ordinally and prints A with five, B with one, and two rejected records.

Watch the boundary. Updating the dictionary before completing every validation check can leave partial results from a rejected row. Keep mutation after validation. Also define whether trimming, case folding, duplicate codes, and zero quantities are allowed instead of relying on incidental behavior. These choices affect the import contract and should be visible in tests, not hidden as string-cleaning conveniences.

Practice independently. Extract a row parser that returns either a code-and-quantity value or a stable rejection reason. Add tests for an empty code, an extra comma, invalid numeric text, and a zero quantity. Preserve the existing aggregation behavior for repeated codes. Finally produce a rejection report containing original row positions, so someone correcting an import can locate the problem without guessing which occurrence was rejected.

A working example

using System;
using System.Collections.Generic;
using System.Linq;
using System.Globalization;
using System.Text;
using System.Text.Json;
using System.Text.Json.Serialization;
using System.Text.RegularExpressions;

var totals = new Dictionary<string, int>(StringComparer.Ordinal);
int rejected = 0;
foreach (string row in new[] { "A,2", "B,1", "A,3", "bad", "B,-1" })
{
    string[] fields = row.Split(',');
    if (fields.Length != 2 || fields[0].Length == 0 ||
        !int.TryParse(fields[1], NumberStyles.Integer, CultureInfo.InvariantCulture, out int quantity) || quantity < 0)
    {
        rejected++;
        continue;
    }
    totals.TryGetValue(fields[0], out int previous);
    totals[fields[0]] = previous + quantity;
}
foreach (var item in totals.OrderBy(x => x.Key, StringComparer.Ordinal))
    Console.WriteLine($"{item.Key}:{item.Value}");
Console.WriteLine($"rejected:{rejected}");

Expected output

A:5
B:1
rejected:2
Try this
Finish parsing and validation before mutating totals, and state the row format’s limits.