A hands-on project to build a simplified Image Signal Processing (ISP) pipeline from scratch. This project implements each pipeline stage manually using NumPy for pixel manipulation, with rawpy for RAW file decoding and matplotlib for display.
The pipeline processes a RAW .ARW (Sony) file through the following stages:
- Decode & Extract — Read the Bayer mosaic, CFA layout, black/white levels, white balance multipliers, and the camera-to-XYZ color matrix using
rawpy. - Linearize — Subtract black level and normalize to
[0, 1]using the white level. - Label Map — Tile the 2×2 CFA pattern to full image size and build per-channel boolean masks (R, G, B).
- Demosaic (Bilinear) — Interpolate missing color values at each pixel position using nearest-neighbor averaging via vectorized NumPy shifts.
- White Balance — Normalize camera white balance multipliers relative to green, then scale each RGB channel.
- Color Space Conversion — Apply a 3×3 Color Correction Matrix (CCM) to each pixel. Currently uses an identity matrix; cam-to-XYZ and XYZ-to-sRGB matrices are extracted but not yet composed.
- Gamma / Tone Mapping — Apply the sRGB transfer function (linear below 0.0031308, power curve above).
- The Color Correction Matrix is bypassed (np.eye(3)) intentionally
- Why: the calibration matrices from rawpy are coupled to libraw's internal pipeline conventions, so dropping them into a custom pipeline withouth knowing libraw's exact ordering and scaling contract produces incorrect results (severe green cast).
- What would be needed to fix it: either reverse-engineer libraw's conventions or derive an independent Color Correction Matrix from a color check calibration target.
- Python 3.x
- NumPy
- rawpy
- matplotlib
Place a .ARW file at ./imgs/AKG02229.ARW and run:
python main.py- Learn the fundamentals of digital image processing.
- Gain hands-on experience with RAW image data.
- Understand and implement each ISP stage from scratch.
MIT License