Creating interactive visualizations of large datasets using JavaScript

Crossfilter is a JavaScript library initially designed by Square to explore large multivariate datasets in a web browser. It basically allows you to create sorted indexes and feed them to a charting library like D3, and enable the user to filter by clicking and dragging, even when sifting through 200.000 rows in a 5MB file:

Time of Day
Arrival Delay (min.)
Distance (mi.)
Date

(go ahead, you can play with it. Click and drag on any of the charts to perform a live filtering of the dataset)

The process to create these charts is relatively easy:
1. Load the data.
2. Define the dimensions with crossfilter.
3. Create the charts with D3.
4. Write the event code to respond to clicks and filters.

The main challenge of using crossfilter is that you have to spend quite some time fighting engaging D3 to make the charts respond to filter changes and user clicks. These charts have to be wired together, so when you click on one, the others respond to the action. The JavaScript code for the chart above is about 300 lines and was made by clever guys who work at Square.

There’s also the question of reusability. How much code would we have to write –and maintain– if we wanted to adapt any of these charts to another visualization?

A lot!

This is the main problem with using D3.js as a charting library and doing everything by hand. The development and testing of a single non-trivial dashboard could take weeks, if not months.

You know what would be ideal? To have a charting library that allowed us to use large datasets with crossfilter and, at the same time, enabled us to create reusable charts that we could easily plug in wherever we want.

That’s exactly what dc.js does. Here I’m loading the same 5MB, 200.000 records file, and dealing with it in only 121 lines of un-optimized JavaScript.

Time of Day

Arrival Delay (min.)

Distance (mi.)

Date

(click and drag on any of the charts to filter the data)

This is the same set of charts, but done in a much simpler, faster and expressive way.

Take a look at the samples they have in their site, like this interactive Twitter dashboard or this stock picker.

dc.js performs the dirty work of linking the charts together, so when we click on a chart, the other charts in the group respond automatically, without writing any additional code. Magic!






Data + Design

data-design-cover

A team of over 50 people have collaborated in Data+Design, a book about preparing and visualizing information.

It’s a thorough but simple introduction to data collection and visualization, very thoroughly written and chock full of great advice.

bars

axis labels

 

labels in mobile charts

labels in mobile charts

It’s open-source too, published in its present form using O’Reilly’s Atlas e-publishing platform, which produces very clean, readable books.

Go and read it here.

 

How to create an embeddable timeline chart


The other day, web developer friend asked me how to create and insert a timeline into WordPress. He actually wanted to put an interactive timeline with links, images and video on the homepage of a news site.

I checked out several options out there and settled on TimelineJS. You can create timeline charts directly on their web site. Each story in the timeline has to be loaded as rows in a google spreadsheet. You can enrich each of the “steps” or events with links, videos and pictures and everything’s stupid-easy to use.

At the end you get an iframe embedding code that you can copy and paste to your site. But they even have a WordPress plugin for that. There’s even other developers who have created alternative tools based on TimelineJS that allow you to play with more settings.

So if you ever need a quick and dirty solution to build a timeline and tell a story, look no further.

 

Now, if you’re a developer and you want a proper, scalable and maintainable solution, that’s another story.

In the case of my friend, depending on an external site for such a crucial part of a website is not a good idea. What if these guys decide to abandon the product (hope not!)? What if their site goes off-line? My friend may lose his content temporarily or permanently. It’s not maintained by him, it’s not hosted by him, it’s not backed up by him. His content it’s not his.

So, the correct solution is to find a way to self-host a similar solution.

Fortunately and amazingly, TimelineJS is an open source tool and it’s on GitHub. If you’re a developer like my friend, you can download the source code, copy it to your server and that’s it. You’ll have a fledged timeline builder in minutes.

 

amCharts, a charting library for creating interactive web charts

amcharts
amCharts is a new JavaScript library for creating charts on the web. It handles many types of charts and it’s somewhat reminiscent of D3.js.

Its most exciting feature is a live editor, which removes the grunt work related to your typical JavaScript charting library and enables you to design, change and export your chart to HTML and include it in your website or dashboard. You can try it out for free here. Amazing work and amazing tool.

For me, the most important feature is the ability to show a chart scrollbar that enables you to zoom in and out of a section of the chart, in less than 5 minutes, like so:



How to build quick and free charts

apples-oranges_chartbuilder

Back in July 2013, the guys from released their Chartbuilder tool to the rest of the world.

Quartz is a very successful online magazine. For their charting needs they use Chartbuilder, a web tool they built to streamline their cumbersome Excel-based process and convert it to a three step copy+paste, copy+paste, publish workflow.

Author David Yanofksy:

(…) frustrated by the amount of my own time it was taking to produce even the simplest charts in our style, I set out to make a tool that would allow me to support the charting needs of the newsroom without consuming all of my time. It quickly turned into an application for the entire newsroom.

Back in July 2013 they released the tool to the rest of us. Chartbuilder can be downloaded and installed in your machine, but you can also use their hosted version here.