JD's tech website

What you don't know can, and will, hurt you

In some industries, displaying images with pixel perfect precision can be critical to quality. It might seem trivial to some, but in many ways, displaying pixel on a screen is still an exercise that remains to be mastered. There are many pitfalls but in this article we will focus on a little known fact that can be tricky to analyze and overcome.

One of the advantage of developing applications based on web technologies is the standardization of the environment you are working with. There are several vendors of web environments (also known as browsers) and they all have to respect a set of standards and rules that you can rely to be implemented consistently across platforms, whatever your OSes and whatever your hardware. It might be a little naive and such assumption might sometimes lead you to a false sense of security. But at least, you can rest assured that if a browser does not position your div properly, even though your code respects the HTML and CSS standards, then the browser is to be blamed.

However, not everything can be specified and indeed, not everything is. One important aspect of displaying images on a discreet grid, that compose our screens, is interpolation. When you have to display an image which resolution does not perfectly match the resolution of the part of the screen you are drawing it to, some measure of interpolation must be applied. Unfortunately, how browser interpolate images is not entirely specified by the folks at the W3C.

For example, let's look at the imageSmoothingEnabled option of the canvas. It says that you can enable or disable image smoothing. Fine, but what is image smoothing? Some sort of interpolation indeed. But what sort? Is it nearest neighbor? Bilinear? Bicubic? Not specified. And this is only for the general algorithm used, so let's forget about the detail and corner cases.

All this is to say that how browser interpolate images are implementation details. However, it can have a non negligible impact on how images are displayed. For example, let's take this image:

This image is 256 pixels by 256 pixels and due to the limitation of this blogging platform we can't display it in a canvas, but if you were to try to display it this way:

<script type="text/javascript">
  function main() {
    const canvas = document.getElementById('image');
    const context = canvas.getContext('2d');

    base_image = new Image();
    base_image.src = 'awesome-face.png';
    base_image.onload = function(){
      context.drawImage(base_image, 0.5, 0.5);
    }
  }

  window.onload = main;
</script>
<canvas id="image" width="257" height="257"></canvas>
  

You might be surprised at the result. Comparing the rendering of that code on firefox and chrome will show this:

Indeed, here the image and the canvas sizes do not match. In order to have the image centered in the canvas we set the top left corner of the image window to (0.5, 0.5) which is perfectly reasonable. You will expect your environment to perform the necessary interpolation so that the image starting at half the top left pixel will lead to an appropriate result. And it does. The problem is "appropriate result" here is under specified. The interpolation is an implementation detail and both browser, at the time of this writing, interprets these specifications differently.

Closer look

In order to understand what is going on, let's try to do something simpler. We will print a black rectangle of size 2x2 to a canvas of size 2x2. Then we will "zoom" on that canvas to a 256x256 canvas. See the code here. This will give you a canvas that display 4 blocks which are basically the 4 pixels of your initial 2 by 2 canvas.

As you can see, if we position that rectangle at the coordinate (0.5, 0.5), as we did with our awesome face earlier, the results in firefox and chrome are quite different.

If you are like me, you will find firefox interpolation more "correct". The top left pixel only is a quarter filled with black whereas the top right and bottom left are half filled and thus should be darker. But against the specifications, both representations are fine. Yet there are different. So a pixel perfect rendering, when interpolation is necessary, meaning every time the screen pixel grid does not exactly match the image grid, is impossible. And if you do not know that, it might hurt you.