A hands-on project to build a simplified Image Signal Processing (ISP) pipeline from scratch. This project implements each pipeline stage manually using NumPy for pixel manipulation, with rawpy for RAW file decoding and matplotlib for display.
The pipeline processes a RAW .ARW (Sony) file through the following stages:
- Decode & Extract — Read the Bayer mosaic, CFA layout, black/white levels, white balance multipliers, and the camera-to-XYZ color matrix using
rawpy. - Linearize — Subtract black level and normalize to
[0, 1]using the white level. - Label Map — Tile the 2×2 CFA pattern to full image size and build per-channel boolean masks (R, G, B).
- Demosaic (Bilinear) — Interpolate missing color values at each pixel position using nearest-neighbor averaging via vectorized NumPy shifts.
- White Balance — Normalize camera white balance multipliers relative to green, then scale each RGB channel.
- Color Space Conversion — Apply a 3×3 Color Correction Matrix (CCM) to each pixel. Currently uses an identity matrix; cam-to-XYZ and XYZ-to-sRGB matrices are extracted but not yet composed.
- Gamma / Tone Mapping — Apply the sRGB transfer function (linear below 0.0031308, power curve above).
Color Correction Matrix is bypassed (np.eye(3)) intentionally.
The calibration matrices from rawpy are coupled to libraw's internal pipeline conventions. Dropping them into a custom pipeline without knowing libraw's exact ordering and scaling contract produces incorrect results (severe green cast — R row sums to -0.47, G to 1.51).
To fix this properly: either reverse-engineer libraw's conventions or derive an independent CCM from a color checker calibration target.
- Python 3.x
- NumPy
- rawpy
- matplotlib
Place a .ARW file at ./imgs/AKG02229.ARW and run:
python main.py- Learn the fundamentals of digital image processing.
- Gain hands-on experience with RAW image data.
- Understand and implement each ISP stage from scratch.
MIT License