CPU versus GPU with the Canvas Web API

Introduction

The Canvas API is a rich and performant API for drawing and manipulating 2D graphics in a Web browser. It is used with the <canvas> HTML element or an OffscreenCanvas. When rendering content to a canvas, the browser can choose to use either the CPU or the GPU. This post looks at how the browser makes this decision and the effect this has on performance.

Browser heuristics

When you create a canvas, the browser has to decide where it should exist. It can store the data for the canvas in main memory, invoking functions running on the CPU to render to it. Or it can create the canvas on the GPU, invoking GPU commands to draw to it. I use the term CPU canvas for the former and GPU canvas for the latter. Unlike a CPU canvas, a GPU canvas is hardware accelerated. This will generally result in better performance, but not always. As a result, the browser might include heuristics for deciding which approach to adopt. It may even change approach after the initial decision in response to how the canvas is being used. This file is an example of the heuristics used in an older version of the Blink browser engine in Chrome.

One simple heuristic is the size of the canvas: a very small or very large canvas might be better as a CPU canvas. Here is a discussion in the Blink developers group about how canvas size matters. The browser might also choose to create any new canvases as CPU canvases if there are already many GPU canvases.

Usages of getImageData and putImageData

The browser will be particularly interested if you use getImageData and putImageData. You can use these methods to operate on the underlying pixel data of a canvas. For example, you can change the brightness or contrast of the image. These manipulations are performed on the CPU and not the GPU. If you invoke getImageData on a GPU canvas then the browser needs to transfer the pixel data from the GPU to the CPU. This is sometimes termed a GPU readback and it is well known for being a slow operation. Similarly the browser has to copy the image data back to the GPU when you use putImageData. Thus the browser might decide to make the canvas a CPU canvas to avoid this data shuffling.

Sometimes you already know that a CPU canvas would be best for your use case. Currently you have to rely on getting one via browser heuristics, but you will soon be able to request it. This will be via a new 2D canvas context attribute called willReadFrequently.

The problem with heuristics

The browser's use of heuristics points to a fundamental limitation of the Canvas API: it is up to the browser whether a given canvas is a CPU canvas or a GPU canvas. You are relying on heuristics and they can change between browser versions. In one version, the browser might determine that a given canvas should be a GPU canvas. In another version, it might make the opposite determination. One reason that Figma gave for using WebGL rather than the Canvas API was not being able to guarantee GPU acceleration.

The performance difference between the two types of canvas can be significant. I used the JSBench.me Web site to test this. In this first test I try to get the browser to use a CPU canvas. In this second test I try to get the browser to use a GPU canvas. The test itself is identical. It copies a part of the canvas to itself:

context.drawImage(canvas, 0, 0, 960, 540, 0, 0, 1920, 1080);
// Force the drawImage call to be evaluated within this benchmark code:
createImageBitmap(canvas, 0, 0, 1, 1).then(() => deferred.resolve());

The difference is in the set-up functions. Both include the same initial code...

var canvas = document.createElement("canvas");
canvas.width = 1920;
canvas.height = 1080;
var context = canvas.getContext("2d");

... but the set-up for the CPU canvas test includes a line designed to force the use of a CPU canvas in Chrome v84:

context.getImageData(0, 0, 1, 1);

I get the following result when I run these tests in Chrome v84 on macOS v10.15:

This is a significant difference in performance, although the variability of the GPU canvas result is greater.

If you have a canvas that exists to do processing of image data using getImageData and putImageData then it should generally be a CPU canvas. (This avoids the overhead of GPU readbacks.) If you have a canvas that uses getImageData and putImageData sparingly then remove those usages. This is to try to prevent triggering a CPU canvas now or in the future. You can instead use a temporary canvas for those operations. When using getImageData, use drawImage to copy the source data to the temporary canvas. Then use getImageData on that temporary canvas. When using putImageData, use it to write to the temporary canvas. Then use drawImage to copy from the temporary canvas to the destination canvas. Also consider if WebGL would be a better choice for canvas rendering in your app.

Conclusion

The Canvas API is a performant and easy-to-use API for drawing 2D graphics in a Web browser, but it has caveats. It is currently not possible to guarantee getting a hardware accelerated canvas when the user's hardware supports it. This could lead to performance issues if the browser's heuristics change in the future. Web developers need to be aware of this, and should consider using WebGL instead.

Changelog

• 2020-08-24 Initial version
• 2020-08-25 Plain English improvements