Tracking.js & the computer vision power of JavaScript

There are various frameworks, methodologies, and standards for building websites and web applications. No matter which ones you’re following, you will always somehow end up in the browser and therefore with JavaScript.

With tracking.js, the browser got even more powerful. Here’s our first review of tracking.js and why JavaScript is on the winning team.

Browsers – the place to be.

Well, recently I’ve shown you the concept of designing in the browser and its advantages over traditional frameworks.

As mentioned in this post there are so much more advantages when going straight into the browser. Today, I want to share another great real life example why the browser is the platform of the future for us developers.

Tracking.js – computer vision power to JavaScript.

Just recently we at Usersnap stumbled upon tracking.js, which is a powerful JavaScript library bringing computer vision and therefore much more power to the browser.

Tracking.js is an open source computer vision library, with different computer vision algorithms on board. These algorithms can be used for features like facial detection or color tracking. And all of that in the browser while having a lightweight core.

Why is this huge?

You might wonder why we are so excited about yet another JavaScript framework?

With tracking.js, the days are gone where computer vision was limited to native apps and C/C++. You use tracking.js directly in your browser to detect faces and track colors! Moreover, it’s up to you, your creativity, and your browser.

The days of OpenCV are gone.

OpenCV stands for open computer vision and supports C++, C, Python and Java interfaces. It has been designed to focus on real-time application and is widely used for sophisticated use cases (for example facial recognition).

With tracking.js, all those processings are happening on the client-side. In the browser. The days of server-side processing are gone.

Real-time applications on the web.

As mentioned before real-time applications (such as facial recognition) have been limited on the web. Tracking.js brings that power of computer vision to the web.

With the increasing power of JavaScript engines, good results can be achieved when real-time apps are running in the browser.

Real-time face detection

So, you want to see some real life examples on what tracking.js can do? Face detection is one great example.

According to the hacker news thread announcing tracking.js, it’s implementation is based on Viola-Jones algorithm, which is one of the most used algorithms for solving the “face detection problem”. Fun fact this algorithm is also used by OpenCV and it allows you to track several types of objects just by changing the training data that is used as an input.

You can view the code here and, of course, you can try it yourself on trackingjs.com. Pretty nice, huh?

Color tracker

What about tracking the color of every item which appears in front of your laptop camera. In order to use a color tracker, you need to instantiate the constructor passing the colors to detect:

var colors = new tracking.ColorTracker(['magenta', 'cyan', 'yellow']);

Now you need to know when something is happening – here’s the code example for that:

colors.on('track', function(event) {
  if (event.data.length === 0) {
    // No colors were detected in this frame.
  } else {
    event.data.forEach(function(rect) {
      // rect.x, rect.y, rect.height, rect.width, rect.color
    });
  }
});

And you’re good to go.

tracking.track('#myVideo', colors);

The Color Tracker will now request access to your camera and track magenta, cyan and yellow colors that appear in front of your camera.

Just give it a try yourself.

Friend tagging

Another, yet interesting use case is friend tagging. At least since Facebook introduced their tagging feature friend tagging has become a household experience. With tracking.js, it now can be implemented in the browser.

You can find the example source code here.

Limitations of tracking.js?

Well, as with every new library, tracking.js have some limitations, e.g. it doesn’t have as many algorithms as OpenCV. Though backed by a huge JavaScript community it’s only a matter of time and we’re excited to see with which creative ideas other developers will come up in order to make use of tracking.js.

Wrapping it up.

With tracking.js, the browser just got a bit more powerful. No matter if you’re implementing a face detection for your web app or developing a browser game, or you’re just in search for a tagging feature, tracking.js might be your JavaScript library to go to.

The browser is definitely the platform of the future. With tracking.js mighty things can and will happen. And they will happen in your browser.

This article was brought to you by Usersnap – a customer feedback solution, used by software companies like Facebook, Google, and AddThis.