HTML5 Green Screen Technique using <VIDEO> and <CANVAS>

by webdood on Sep.09, 2011, under Canvas, HTML5, Safari-Specific, Software Development

This technique allows you to composite a green-screen video over a web page.
The basic setup consists of a hidden <VIDEO> tag, another hidden <CANVAS> tag, where each frame of video is captured for further processing and a visible <CANVAS> tag where each processed frame is displayed.
When a frame of video is captured and placed into our “canvasProcessor”, each pixel is analyzed in turn. A pixel is represented by a CanvasPixelArray consisting of four values: (R)ed, (G)reen, (B)lue and (A)lpha.
When we find a pixel that fits in the range of “green”, we change that one to have a 0 (A)lpha value (and thus it becomes invisible).

Here is link to a working example.

<!DOCTYPE HTML><
<html>
<head>
  <title>Green Screen Test</title>
  <script type="text/javascript">
    greenScreenEngine = {
      _animator : null,
      _canvasProcessorContext : null,
      _canvasTargetContext : null,
      _height : 0,
      _width : 0,
      _video : null,
      handlerFor : {
        pause : function() {
          clearTimeout( this._animator );
        },
        play : function() {
          greenScreenEngine.parseAndProcessFrame();
        }
      },
      parseAndProcessFrame : function() {
        if (this._video.paused || this._video.ended) {
          return;
        }
        this._canvasProcessorContext.drawImage(this._video,0,0,this._width,this._height)
        var videoFrame     = this._canvasProcessorContext.getImageData(0,0,this._width,this._height),
            numberOfPixels = videoFrame.data.length / 4;
        for (var i=0;i 100 && b < 100) {      // Here we look for what we perceive to be a green pixel
            videoFrame.data[i * 4 + 3] = 0;        // If we find one, make its alpha channel transparent
          }
        }
        this._canvasTargetContext.putImageData(videoFrame,0,0);
        this._animator = setTimeout( function() { greenScreenEngine.parseAndProcessFrame() },0);
      },
      setCanvasProcessor : function( canvasProcessorID, scale ) {
        scale = (scale!==undefined) ? scale : 1;
        this._height = this._video.videoHeight * scale;
        this._width  = this._video.videoWidth * scale;
        var oCanvas = document.getElementById(canvasProcessorID);
        oCanvas.height = this._height;
        oCanvas.width  = this._width;
        this._canvasProcessorContext = oCanvas.getContext("2d");
        this._canvasProcessorContext.drawImage(this._video,0,0,this._width,this._height)
      },
      setCanvasTarget : function( canvasTargetID ) {
        var oCanvas = document.getElementById(canvasTargetID);
        oCanvas.height = this._height;
        oCanvas.width  = this._width;
        this._canvasTargetContext =  oCanvas.getContext("2d");
      },
      setSourceVideo : function( videoID ) {
        this._video = document.getElementById( videoID );
        this._video.addEventListener("play", greenScreenEngine.handlerFor.play)
        this._video.addEventListener("pause", greenScreenEngine.handlerFor.pause)
      }
    }
    function setup() {
      greenScreenEngine.setSourceVideo("video-green-screen");
      greenScreenEngine.setCanvasProcessor("canvas-processor",.5);
      greenScreenEngine.setCanvasTarget("canvas-target");
      greenScreenEngine._video.play();
    }
  </script>
</head>
<body>
  <div>
    <video id="video-green-screen" src="talkingHead.mp4" controls="true" tabindex="0" oncanplay="setup()" style="display:none;"></video>
  </div>
  <div>
    <canvas id="canvas-processor" style="display:none;"></canvas>
    <canvas id="canvas-target" style="-webkit-transform:rotate(90deg);position:absolute;bottom:79px;"></canvas>
  </div>
</body>
</html>>

Note - For the video I've used here, I had to play with the values a bit to find just the right range of values that mean "green." Also, the source video was rotated -90deg, so I had to apply a webkit-transform to it. To see how everything works, just comment out the display:none CSS.

Shannon Norrell

1 Comment for this entry

Shannan Rohde
January 10th, 2012 on 4:53 pm
I was outside on break and thought this up and I am glad there are others that are looking into it. I think it would be impressive to use this on an intro page. I wonder thos if there is a way that when people refresh the page it would do a step process, meaning refresh would play vid 2 => refresh 3 => vid 3 and so on and so forth? What are your thoughts. Feel free to email me

Reply