Camera ISP → CVML Lab

RESEARCH / DEMONSTRATION — SYNTHETIC DATA. Take an image signal processor apart one stage at a time and watch what each stage costs, in image quality and in detection, on the same frame. A synthetic scene is rendered twice: once as the ideal image a perfect lens and a noiseless sensor would produce, and once as the radiance a real lens delivers — tinted by the illuminant, hazed, smeared by the pass, blurred by the point-spread function, and rotated into the sensor's own primaries. A configurable sensor then photographs it with shot noise, read noise, PRNU, DSNU and hot pixels, and nine ISP stages turn the RAW mosaic back into a picture: black level, demosaic, white balance, colour matrix, denoise, tonemap, local tone, sharpen, normalise. Every stage is separately switchable and separately timed. The ideal image is what makes this possible and is the reason the scene is synthetic — PSNR, SSIM and ΔE₀₀ are full-reference metrics, and you can never ask a field photograph what it should have looked like. Press Run to process and measure; press Filmstrip to see every intermediate stage as a thumbnail, including the mosaic, which is the only frame that looks like what the sensor actually recorded. Press Detect to run a detector over the processed pixels, match its boxes against ground truth by IoU, and score it. Then change one thing and press Run again. The point is what happens next: switch the denoiser to an unconditional blur and raise its radius, and SSIM improves for two steps while small-object recall collapses; switch white balance to gray-world on a frame that is four-fifths soil and foliage and watch a defensible estimator produce a strong colour cast that costs the colour-led detector every weed it had; set the colour matrix to identity and PSNR falls 10 dB while one detector gets BETTER. Three detectors are provided because which one you deploy changes which ISP is correct. LIMITS, STATED RATHER THAN DISCOVERED: THE SCENE IS PROCEDURAL AND SO IS EVERYTHING IN IT. Soil, three crop rows, eight broadleaf weeds and a test target of six colour patches, a flat noise field and a band-limited slanted edge. It is not a photograph, it is not derived from one, and it contains no imagery, data, model or parameter from any real imaging or agricultural vision system. The weed and crop reflectances are plausible broadband values chosen so the two classes differ in chroma more than in luminance, which is a modelling decision and not a measurement. THE DETECTOR IS A SURROGATE AND NO CONVOLUTIONAL NETWORK RUNS HERE. Its confidence is a deterministic logistic over four features measured from the processed pixels — a chromaticity posterior against frozen class prototypes, an excess-green vegetation index, figure-ground luma contrast and Laplacian texture energy — with per-model weights chosen to reproduce published CNN sensitivities. That is a modelled response, not a measurement of any network. Real inference needs a supplied .onnx model and `infer.runImage`, whose boxes go through this same IoU matching and these same metrics. Ground truth decides where a proposal is placed, as a region-proposal stage would, and then plays no further part in its score, which is the one property that makes the numbers mean anything; localisation quality is modelled by the proposal jitter rather than measured, so the IoU distribution is a property of the fixture. There is no non-maximum suppression over a dense anchor grid, so precision is limited by proposals landing on soil-to-crop boundaries whose chroma genuinely is weed-like — that ambiguity is in the scene, not in the detector. TIMINGS ARE SINGLE-THREADED JAVASCRIPT ON ONE CPU CORE. They are useful for ranking stages against each other and are not an estimate of what a GPU would do; the modelled roofline on the bench plate says exactly what it ignores. A single cold run is dominated by JIT warm-up, which is why Time reports a median over a warmed loop. MTF50 comes from ONE slanted edge at one orientation, so it measures the horizontal response only, and the discrete difference used to get the line spread function applies its own rolloff that ISO 12233 corrects for and this does not — it is consistent across settings at one resolution, which is what this compares, and is not a number to quote against a lens datasheet. It is in cycles per pixel and therefore depends on the sampling grid, so comparing it between the preview and measure resolutions is meaningless and the readout carries the resolution. SSIM is computed on Rec.709 luma with an 11×11 Gaussian window, so a pipeline that gets luminance right and colour wrong scores well here and badly on ΔE₀₀ — that is not a flaw in either. ΔE₀₀ is CIEDE2000 against the chart's declared reflectances, and a clipped patch is excluded rather than scored, because a blown patch has no colour left to be wrong about. Denoise runs AFTER demosaic, which is a real engineering choice and not a fact: many production pipelines denoise on the Bayer mosaic instead, and that is not measurable here. Sharpening runs after the tone curve, in display space. No real RAW file reader, no lens distortion or vignetting, no image sequences, no masks, and no training of anything. The whole lab lives in the track state at a couple of kilobytes; the pixels are runtime-only and are recomputed on load, while the MEASUREMENTS are read from state rather than recomputed, so two people opening the same published link see the same figures.