Text decoding adds a contract above the byte stream
A byte stream does not inherently contain characters, lines or an encoding name. StreamReader interprets those bytes using an encoding and supplies character-oriented operations such as ReadLine. This lesson constructs a known byte sequence in memory: a UTF-8 byte order mark followed by the UTF-8 bytes for rune and a newline. The reader receives an explicitly constructed UTF-8 encoding and enables byte-order-mark detection. That makes the experiment about decoding choices rather than machine defaults or an external file's undocumented contents. The stream remains a byte source even while the reader presents a character interface.
Observe the line and the detected encoding
The first three input bytes are EF BB BF, the UTF-8 preamble. Because detection is enabled at the beginning of the input, the reader treats that preamble as an encoding marker rather than exposing it as the first character of the returned line. ReadLine then returns rune without its line terminator, producing the first printed line. Inspecting CurrentEncoding.WebName after that read prints utf-8. The ordering matters: the first read is what gives detection an opportunity to examine the stream. Merely constructing the reader is not evidence that all input-dependent decoding decisions have already taken place.
BOM detection and a fixed-encoding protocol are different policies
Enabling detection allows supported preambles to influence the selected encoding. That is appropriate when a document contract explicitly permits such markers, but a protocol that requires UTF-8 regardless of input should choose its detection policy deliberately. An encoding object supplied to the constructor is the starting choice, not a promise that enabled detection will never select something else. The sample does not inspect malformed input after a preamble or claim a general strict-validation policy for every detected encoding. A stricter importer should define its accepted encoding and marker rules together, then test invalid bytes under that exact configuration.
Do not turn byte position into a character index
A reader buffers bytes and decoded characters for efficiency. Reading one line can therefore advance the underlying stream farther than the visible character count suggests. The stream's byte position is not a reliable cursor for restarting at a chosen character, especially when UTF-8 characters occupy different numbers of bytes. Mixing direct stream reads with an active buffered reader can skip or duplicate data from the consumer's perspective. The example avoids that problem by giving the reader exclusive access during its scope and printing semantic observations rather than internal buffer positions. Seek and restart only with a carefully defined reader-reset strategy.
Own the reader separately from the byte source
The constructor's leaveOpen: true means disposing this reader does not dispose the memory stream supplied by the caller. It does not rewind the stream or undo buffering. The outer owner still disposes its stream at the end of the sample. No filesystem path is involved, and the printed encoding name is a framework encoding identity rather than a guess about a file extension. This separation lets a test control the exact byte sequence while testing the real character API. It also makes clear that line parsing, encoding selection and resource lifetime are three independent concerns.
Change the marker policy and the character width
First set only detectEncodingFromByteOrderMarks to false while leaving the UTF-8 preamble in the input. Examine the first returned character's numeric value rather than relying on its visibility in the console. Explain why the line can look similar while containing an extra character. For the second exercise, restore detection and replace the plain text with `λ
`. Predict the returned line and compare the number of UTF-8 payload bytes with the string's character count. These experiments separate marker handling from the variable byte width of encoded text.
- Hint for disabled detection: the leading UTF-8 preamble decodes as U+FEFF within the returned string.
- Hint for lambda: its UTF-8 payload occupies two bytes before the newline; the returned line contains one UTF-16 code unit.
State the text boundary before adding parsing rules
A line reader solves the mechanical task of extracting text lines after decoding; it does not validate a business record or bound the length of an individual line. An importer still needs rules for empty lines, record separators, numeric culture and maximum accepted data. Before composing those rules, be able to explain this lesson's two outputs from the original bytes and constructor settings. Then ask what changes if the marker is absent, detection is disabled, or a different accepted encoding is selected. Those are explicit format decisions, not details that an extension or an operating system can safely decide for every consumer.
- Observe CurrentEncoding after a read when BOM detection matters.
- Do not mix direct byte reads with an active buffered text reader casually.
- Specify encoding, marker acceptance and ownership independently.
A working example
using System;
using System.Linq;
using System.Collections.Generic;
using System.Globalization;
using System.Text;
using System.Text.Json;
using System.IO;
using System.IO.Compression;
using System.Buffers.Binary;
using System.Net;
using System.Net.Http;
using System.Net.Http.Headers;
using System.Security.Cryptography;
using System.Threading;
using System.Threading.Tasks;
byte[] payload = new byte[] { 0xEF, 0xBB, 0xBF }.Concat(Encoding.UTF8.GetBytes("rune\n")).ToArray();
using var stream = new MemoryStream(payload);
using var reader = new StreamReader(stream, new UTF8Encoding(false, true), detectEncodingFromByteOrderMarks: true, bufferSize: 128, leaveOpen: true);
Console.WriteLine(reader.ReadLine()); Console.WriteLine(reader.CurrentEncoding.WebName);
Expected output
rune utf-8
The BOM is representation metadata; read before assuming CurrentEncoding has completed detection.