Decoding failure is part of the input contract
An encoding maps valid byte sequences to text, but an importer must also choose what happens when bytes are invalid. Replacement fallback permits decoding to continue with a replacement character, while exception fallback rejects the operation. Neither policy should be discovered accidentally after corrupted text has reached a model. The lesson feeds the same malformed two-byte sequence to two real UTF-8 encoding configurations so the difference is visible without filesystem settings or network variability. It tests a boundary before any record parsing occurs: whether the bytes can be accepted as the text representation the application claims to consume.
Follow the invalid prefix through both decoders
The input begins with hexadecimal C3, which starts a two-byte UTF-8 sequence, but the following byte 28 is not a valid continuation. The default Encoding.UTF8 decoder uses replacement fallback, so the resulting string contains U+FFFD and the first print is True. The sample then constructs new UTF8Encoding(false, true), where the second argument requests an exception for invalid bytes. Decoding the same input throws DecoderFallbackException, and its catch prints invalid UTF-8. The boolean answers a targeted replacement-character question; it does not claim that the default decoder has validated a business record.
Encoding and decoding have different invalid inputs
Decoding starts with bytes and may encounter invalid byte sequences. Encoding starts with a .NET string and may encounter invalid text, such as an unpaired UTF-16 surrogate, depending on the chosen encoding and fallback policy. Do not confuse a decoder exception with an encoder exception or assume that a previously created string could only have originated from valid UTF-8. The false preamble option in this constructor controls emitting a UTF-8 byte-order mark when obtaining a preamble; it is not what activates strict decoding. The explicit true invalid-byte option is the relevant switch for this experiment.
A valid multibyte character can cross a buffer boundary
Malformed input and incomplete input during a streaming operation are not automatically the same thing. A valid character may begin near the end of one chunk and finish in the next. Calling a stateless GetString separately on each arbitrary chunk can treat those pieces incorrectly. A Decoder or a correctly configured StreamReader preserves decoding state across reads and can distinguish an unfinished prefix from invalid final input when the sequence ends. This sample decodes a complete supplied byte array, so its invalid continuation is unambiguous. A streaming adaptation must preserve state rather than merely repeating the whole-array call.
Replacement text should not silently become canonical data
Replacement can be useful when displaying damaged human-readable documents, but accepting it in identifiers, signatures or import keys changes the application's data. A strict importer may reject invalid UTF-8 before applying domain rules, while a document viewer may report damage and continue. Choose that policy at the boundary and preserve a useful error context without dumping sensitive input into logs. Strict decoding alone still says nothing about allowed names, numeric ranges, duplicate records or authorization. It confirms one representation property, after which the parser and domain validator must apply their own independently stated rules.
Compare a valid character with an incomplete sequence
First replace the two input bytes with C3 A9, a complete UTF-8 encoding of é. Predict the replacement-character test and confirm that strict decoding returns a character rather than entering the catch. Then use only C3 as the complete supplied input. Explain why the final missing continuation is now an error even though the same first byte could be valid if another chunk arrived later. For an additional investigation, feed those two valid bytes to a stateful decoder in separate calls, marking only the final call as the end of input.
- Hint for C3 A9: neither decoder needs replacement; the Contains check becomes False.
- Hint for a lone final C3: strict decoding rejects the incomplete sequence; streaming code must distinguish final input from a temporary chunk boundary.
Keep representation validation visible in the pipeline
A review should identify the raw byte source, the chosen encoding, the invalid-input policy and the point where decoding is considered complete. The original program makes all four explicit, so the two output lines can be predicted from the same malformed bytes. When integrating this knowledge into an importer, place strict decoding before converting text into records and decide how failures reach the caller. Do not catch every exception and return an empty successful document, because that conflates invalid representation with valid absence of data. Preserve a distinct failure path and test it with known malformed sequences.
- Select replacement or exception fallback deliberately for the application boundary.
- Use a stateful decoder when arbitrary byte chunks can split characters.
- Treat successful text decoding as one validation stage rather than a complete trust decision.
A working example
using System;
using System.Linq;
using System.Collections.Generic;
using System.Globalization;
using System.Text;
using System.Text.Json;
using System.IO;
using System.IO.Compression;
using System.Buffers.Binary;
using System.Net;
using System.Net.Http;
using System.Net.Http.Headers;
using System.Security.Cryptography;
using System.Threading;
using System.Threading.Tasks;
byte[] invalid = { 0xC3, 0x28 };
string replaced = Encoding.UTF8.GetString(invalid); Console.WriteLine(replaced.Contains('\uFFFD'));
try { new UTF8Encoding(false, true).GetString(invalid); } catch (DecoderFallbackException) { Console.WriteLine("invalid UTF-8"); }
Expected output
True invalid UTF-8
Replacement decoding and strict validation answer different questions about the original bytes.