CLR & Concurrency · Lesson 24

Measure observations without inventing a benchmark

Keep actual timestamp/allocation observations distinct from a portable performance conclusion.

16 min guided lesson · Read, predict, then run

External .NET 10 SDK / desktop runtime. OS-thread experiments run only here; the .NET 9 browser companion has the narrower scope stated in the guide.

Read or download the complete SDK project, then follow its setup on your computer. The separate C# companion practice below runs in SharpRune’s .NET 9 console environment.

The runtime contract

Performance investigation needs observable measurements and a careful statement of what they cover. The portable program really samples a monotonic timestamp and the current thread's allocation counter around a tiny bounded workload, but its stable output checks only nonnegative observations and a checksum. It does not invent a fixed number of bytes or nanoseconds. The desktop harness warms a small calculation, records three actual timing and allocation samples on stderr, and keeps deterministic behavioral checks on stdout. This separates a measurement mechanism from a benchmark conclusion and from any claim that desktop results characterize SharpRune's WebAssembly environment.

Trace the portable program

The portable workload creates a byte buffer, stores seven, and samples elapsed ticks and allocated bytes before retaining the object through KeepAlive. The expected output is nonnegative:True and checksum:7; exact metric values remain runtime observations. The external workload fills one hundred integers and checks the same computed sum across three measured samples, printing equivalent:True and samples:3. Its stderr contains the actual per-sample ticks, bytes, and checksum. A common wrong answer quotes one tiny duration as a universal speed ranking, or treats current-thread allocation bytes as total live memory, retained heap size, or a cross-thread async allocation total.

Execute the actual desktop lab

The external harness records real measurements after a short bounded warmup and emits variable observations only on stderr. Its ordinary default project run is Debug, intentionally useful for demonstrating measurement mechanics but unsuitable as a serious optimized-code comparison. The repeated checksum comparison is a stability check, not an independent correctness oracle for the algorithm. The known sum from zero through ninety-nine is four thousand nine hundred fifty and should be checked separately, as the validation does. Keeping actual diagnostics alongside a direct expected answer prevents a stable wrong implementation from being mistaken for a meaningful benchmark result.

Pitfalls and boundary limits

Stopwatch ticks use Stopwatch.Frequency and are not automatically TimeSpan ticks or nanoseconds. Current-thread allocated bytes count managed allocations in that thread's scope, not native allocations, retained live objects, or total allocations after an asynchronous thread switch. JIT optimization, interpreter behavior, debug settings, instrumentation, and first-use initialization can change observations. KeepAlive makes the intended object lifetime visible but does not prove one universal allocation strategy. A value of zero elapsed ticks can be legitimate for a tiny workload at the available timer resolution. None of these facts justifies inventing a nonzero duration to make a report look convincing.

Practice one: isolate behavior

First change the portable stored byte to nine and predict checksum nine while the nonnegative observation remains a relationship check. Hint: the correctness value and measurement values answer different questions. Then convert an actual measured timestamp difference using the documented frequency or GetElapsedTime rather than assuming a tick duration. Record environment and build configuration with the result. Do not add an expected fixed byte count merely because one desktop run printed a familiar array size; another runtime or optimization strategy may legitimately produce a different observation.

Practice two: design and verify

For another exercise, compare two genuinely different implementations of the same bounded operation using independently expected outputs before measuring either. Run an appropriate Release benchmark without a debugger and retain multiple samples and the chosen workload. Hint: repeated execution of one implementation proves neither equivalence to another nor statistical confidence. Use a proper benchmarking tool when the question needs its isolation and analysis features; this console harness does not impersonate that tool. Keep desktop, interpreter, and WebAssembly results labeled separately, and include setup or allocation costs only when they belong to the operation being compared.

Review the evidence

Review performance evidence by stating the correctness oracle, measured scope, execution model, build configuration, timer unit, allocation scope, sample count, and inference being proposed. The portable output establishes a real tiny observation and checksum. The external diagnostics provide actual samples, not a universal ranking or a fabricated browser benchmark. A useful experiment can be small when its limits are explicit. Let measurements narrow a concrete question and guide the next test, while preserving the distinction between observing a number, explaining its causes, and claiming that a change improves the real application's workload.

External .NET 10 SDK / desktop runtime. OS-thread experiments run only here; the .NET 9 browser companion has the narrower scope stated in the guide.

Measure observations without inventing a benchmark — desktop runtime lab

Set up the SDK project

  1. Save both files at their exact paths in a fresh owned directory.
  2. Use a .NET 10 SDK, then run dotnet run --project RuntimeLab.csproj. This project intentionally uses C# 13 syntax.
  3. Workers and waits are bounded; a timeout or a violated invariant fails the process. No benchmark or NativeAOT publication is implied by an ordinary run.

Download complete SDK project ↗

Project files

Program.csDownload file ↗
using System;
using System.Linq;
using System.Collections.Generic;
using System.Collections.Concurrent;
using System.Reflection;
using System.Linq.Expressions;
using System.Threading;
using System.Threading.Tasks;

for (int warmup = 0; warmup < 10; warmup++) _ = Work(100);
int reference = Work(100);
bool equivalent = true;
for (int sample = 0; sample < 3; sample++)
{
    long before = GC.GetAllocatedBytesForCurrentThread();
    long start = System.Diagnostics.Stopwatch.GetTimestamp();
    int result = Work(100);
    long ticks = System.Diagnostics.Stopwatch.GetTimestamp() - start;
    long bytes = GC.GetAllocatedBytesForCurrentThread() - before;
    equivalent &= result == reference && ticks >= 0 && bytes >= 0;
    Console.Error.WriteLine($"sample:{sample};ticks:{ticks};bytes:{bytes};checksum:{result}");
}
Console.WriteLine($"equivalent:{equivalent}");
Console.WriteLine("samples:3");
static int Work(int count) { var values = new int[count]; for (int i = 0; i < values.Length; i++) values[i] = i; int total = 0; foreach (int value in values) total += value; GC.KeepAlive(values); return total; }
RuntimeLab.csprojDownload file ↗
<Project Sdk="Microsoft.NET.Sdk">
  <PropertyGroup>
    <OutputType>Exe</OutputType>
    <TargetFramework>net10.0</TargetFramework>
    <LangVersion>13.0</LangVersion>
    <ImplicitUsings>enable</ImplicitUsings>
    <Nullable>enable</Nullable>
    <PublishAot>false</PublishAot>
  </PropertyGroup>
</Project>

Expected result

Exact behavioral stdout on the stated desktop lab: equivalent:True samples:3 Variable diagnostics, where emitted, go to stderr and must not be treated as portable timing or allocation guarantees.

Use the listed SDK and dependencies for this project. Open C# practice runs the separate portable exercise.

C# companion practice

using System;
using System.Linq;
using System.Collections.Generic;
using System.Collections.Concurrent;
using System.Reflection;
using System.Linq.Expressions;
using System.Threading;
using System.Threading.Tasks;

long beforeBytes = GC.GetAllocatedBytesForCurrentThread();
long beforeTime = System.Diagnostics.Stopwatch.GetTimestamp();
var buffer = new byte[256];
buffer[0] = 7;
long elapsed = System.Diagnostics.Stopwatch.GetTimestamp() - beforeTime;
long bytes = GC.GetAllocatedBytesForCurrentThread() - beforeBytes;
GC.KeepAlive(buffer);
Console.WriteLine($"nonnegative:{elapsed >= 0 && bytes >= 0}");
Console.WriteLine($"checksum:{buffer[0]}");
Open C# practice

C# practice output

nonnegative:True
checksum:7
Try this
This is an actual bounded observation, not a fixed allocation size or a benchmark claim. The external harness prints measured samples on stderr after warmup.

Further reading

Interview questions for this lesson