Internet Adoption : The Movie

Player_WDI_InetUsePlotly

In my last blog article I discussed the reusable Player component.  So I’ve decided to put it to work on a real WDI dataset consisting of 5,166 rows.  This example plays a heatmap of % users on the internet mapped across country.

Images link to live examples.

Tying the plot.ly chloropleth to the player was quite easy.  In this case, I didn’t even bother to wrap the component in the Dex component framework, though I may do so in the future for consistency.

Next, I threw the same dataset at the C3 based player discussed in the earlier blog.  I also expanded the player to support categorical x-axis.  By framing the data in different ways, we can create very different views of the data and gain deeper insight.

In this example, I frame the data by country.   The player plays the internet adoption across each country from 1989-2014.  Here, we can see Afghanistan’s slow but increased adoption of the internet.

Player_WDI_InetUseFramedByCountry

By simply reframing the datset by year, we can see all countries side by side.  The player cycles through time.  There are many labels, so the zoom and pan options of C3 come in handy here.  Same data, but quite a different view.

Player_WDI_InetUseFramedByYear

Anyway, that’s it for now, but I expect that there will be much more to come.

  • Pat
Posted in General | Leave a comment

A Reusable Javascript Player Control

Anyone who has read deeply into my blog knows that I am a huge fan of motion charts.  However, adding that extra interactive dimension tends to add more than it’s share of complexity.

So, with inspiration derived from multiple sources such as Hans Rosling, Mike Bostock, James Curley, Alex de Muller and lastly Steve Wittens.

My idea was, why is a motion chart such as this a special case?

MotionChart

I asked myself, isn’t it really two independent components interacting with one another?  In this case, a time slider driving a simple bubble chart.  Furthermore, why can’t we implement them independently and in a reusable manner so that we can simplify things?

So the thoughts mulled around in my head, almost dying until I saw Alex de Muller’s three dimensional animation:

 

VisJSGraph3D

The controls are simple and serve as a dimensional filter, in this case a time dimension. As you step through the graph, it changes over time.  However, I would also like to support numeric and categorical dimension in addition to time.

The next thing I would like to talk about is a term Mike Bostock introduced me to called Object Constancy.

ObjectConstancy

Object constancy is the practice of allowing the eye to follow the transitions of visual state changes.  It’s the realm of tweens, transitions and easing.  If an element enters or exits the stage, give it some flourish (a dramatic entrance and exit if you will) so that the viewer can better wrap their mind around the changes from frame to frame.  James Curley has been a prolithic producer of interesting visuals; genereously sharing them on bl.ocks.org.  James tends to pay great attention to transitions as this NHL example shows.

NHLTransitions

Finally, Steve Wittens of Mathbox fame, state:

“I’ve found that, when in doubt, adding an extra dimension always helps. The moment I finally realized that every implicit graph in N dimensions is really just a slice of an explicit one in N+1 dimensions, a ridiculous amount of things clicked together.”

So true…and lead me to think of a design where there might be multiple dimensional filters interacting in tandem.

The Player Component

So without further adieu, let me introduce the Player component.  In the coming weeks or months depending, I will github everything, sooner if there is avid interest.

The old deprecated project of mine DexCharts has evolved into an more general purpose wrapping framework for providing a consistent interface around different browser-based technologies.  It’s called dex.js.  So I implemented the Player in dex.js under the dex.ui module.

Here’s an example of the Player component interacting with C3 components also wrapped in the dex.js framework.

Player1Demo

In this example, I kept the dataset to 4000 points in order to respect those viewing the visuals on mobile devices.  However, it scales much higher on more powerful devices.  This one has 11,000 datapoints.

Player2Demo

The controls are simple and should be intuitive as they follow a media-player metaphor.

You can move to the first or last frame, step through the frames at a specified delay, or play the frames on a continuous loop.

All links are live so click the images for live versions.

C3 was an ideal choice due to Masayuki Tanaka’s strict adherence to the principles of re-usability and object constancy.

Within dex.js, this chart is pretty simple to describe.  Essentially, the core of this program is as follows:

var player = new dex.ui.jqueryui.Player({
  'parent' : '#toolbar',
  'csv' : csv,
  'delay' : 500
});

var linechart = new dex.charts.c3.LineChart({
  'parent'      : '#ChartArea',
  'resizable'  : true,
  'color'       : d3.scale.category20c(),
  'linktype' : "line",
  'csv' : player.attr("frames").frames[0]
});

linechart.subscribe(player, "new-frame", function(msg) {
  linechart.attr('csv', msg.data).update();
});

player.render();
linechart.render();

We create a player with a 500ms delay based on a simple csv dataset.

The player separates the csv into frames based on column zero of the csv, though this is configurable.

Next, we create a linechart which is initialized to the contents of the first frame.

Then we add a pub-sub event where the linechart listens for “new-frame” events from the player.  When received, it will update the csv data to the current frame.  If the dexjs wrapped visual.

Lastly, we render both and the fun begins.

Wrapping Up

I have much more to say on this topic and more demos to come.  I want to demonstrate some multi-dimensional players across multiple dimensions, and want to bring motion to more than just simple area/line/scatter/bubble charts.

For example, here is a motion-chord chart.  It’s pretty much the same code as the first examples only using a chord instead of a C3 chart.

It’s not quite as polished due to my Chord diagram’s lack of object constancy, but you get the idea of where I am going.

PlayerChordDemo

That’s it for now…

  • Pat
Posted in General | Leave a comment

Trumps Iowa Speech

Here is a link to Trump’s Iowa speech visualized.  Size is proportional to word frequency and adjacent words are connected to one another.

It uses vivagraph with custom svg text nodes.

TrumpIowaSpeech

Posted in General | Leave a comment

Graphing in 3D with Dex and Vis.js

Here is a video demonstating graphing in 3D and time via Dex and Vis.js.

 

This capability is available in Dex 0.8.0.7 which is also available here.

Posted in General | 1 Comment

More Cool Stuff

I’ve been busy, silently coding, having fun and discovering new and exciting visualization technologies both new and old.  Since the last post, I’ve released version 0.8.0.6 of Dex which has a bunch of bug fixes and some new capabilities.  I’ll hit some of of the highlights in this article.

UML

Recently, I had a need to quickly visualize some UML relationships and I discovered a visualization package called PlantUML.  PlantUML is open source and comes in a variety of generous licensing options.  However, using Dex with PlantUML will require an additional installation of GraphViz.  However, it’s painless and well worth it.

As always, images link to a live version.

Here are the battles in the Game of Thrones visualized in a sequence diagram:

Thrones_UML_Sequence

And here are the battles visualized as a class diagram:

Thrones_UML_ClassDiagram

Vis.js

I recently discovered an open source visualization library called Vis.js.  Frankly it’s awesome.  There is a lot of attention to details such as implicit pan, drag and zoom capabilities and it scales very well.  It’s network diagrams lay out and behave arguably nicer than D3 visuals though I am still working some kinks out of hieararchical layout behavior.

The physics model behind them is spectacular, not surprising since the author, Alex de Mulder, has a degree in Applied Physics.

Less words, more pictures and interaction!

Network Diagrams

Now that VisJS networks are integrated into Dex, we can easily things like this:

Here’s a simple one where we take a dataset which records the survivors and deceased passengers of the ill-fated Titanic.  We relate them by gender and passenger class.  Nodes and link sizes relative to the number of passengers within each category.

 

Vis_Network_Titanic

 

Next, is a visualization of President/Party affiliations.

Vis_Network_Presidents

Let’s not forget about the UK visualization of Prime Ministers/Party affiliations:

 

Vis_Network_PrimeMinisters

And next is a more complex and more hierarchical example relating the Game of Thrones Attacker King and Affiliation to the Battle and back again to the Defending King.

Vis_Network_Thrones

Timelines

Last but certainly not least is the Timeline.  The timelines scale up to thousands of events and the zoom is amazing.  It smoothly zooms down from centuries to decades to years to month/year to month/day down to discrete time ranges in the millisecond range.  This is fantastic if you wish to view time sequence data at both a high and low granularity.  Here are some examples:

Use the mouse-wheel to zoom in and out of the timeline and left-click/drag to move around.

OJ Timeline

In the spirit of all the OJ specials coming out right now, here is a timeline of the OJ Simpson Trial.

 

OJ_Timeline

World War I Timeline

Hard to believe such a global tragedy began with a simple assassination.

WWI_Timeline1

Another view zoomed out:

WWI_Timeline2

D3Plus Ring Networks

I find that I am really loving the D3 Ring Network’s ability to display only adjacent links.  Here’s a Dr Who Network.

DrWho_D3Plus_Network

Tons of links converge at a miniscule node to the upper right “Serve Big Bad”, clicking it yields a node centric view which looks like:

DrWho_D3Plus_BigBad

Wrapping Up…

Anyway, that’s not it by a long shot, but it’s all I have time to discuss tonight.

Posted in General | Leave a comment

Game of Thrones Visualized

Dex 0.8.0.5 is around the corner, and I wanted to introduce a number of new visualizations which will be available.  I’m also a big fan of the the Game of Thrones series so I figured I’d combine the two into a single blog post.

First of all, a big thanks to Chris Albon for creating the war of the five kings dataset in the first place.

Dex Workflow

So first I create a Dex workflow with a bunch of visuals as seen here:

Thrones_Overview

The workflow simply reads the dataset and selects the columns and the order I want to visualize things in.

Click any of the visual images to go to a live version.

Parallel Coordinates

Thrones_PC

While this visual isn’t new, it’s such an awesome way to view multi-dimensional data.

Ring Network

Here we view the data using the Ring Network from d3plus.  This visual is an awesome way to view relational complexity.

Thrones_RingNetwork

Orbital Layout

When I first encountered Elijah Meeks blocks site, I realized that I had hit the motherlode and that I’d be coding up visualizations all day.  The Orbital layout arranges hierarchies using a celestial mechanic metaphor.  Children of a parent node orbit the parent.  Grandchildren orbit the children and so forth.  This leads to some beautiful and sometimes unexpected structure.

Thrones_Orbital

Treemaps

I ported a basic Treemap into Dex.js.  This will deprecate the old Treemap.

Thrones_Treemap

This is a work in progress, but I am also working on porting Bill D. White’s titled Treemap into Dex as well.  If you are unfamiliar with Bill’s work, you should check out his blog.

Bill is also doing some amazing work with D3/WebGL integration.  About 1000 times more sophisticated than anything I have ever attempted.

I plan to steal it all 🙂

Here is a view of the Titled Treemap work in progress.

Thrones_TitledTreemap

Sunburst

The Sunburst is getting some love, and being ported into Dex/Dexjs.  Label fitting is still a problem.

Thrones_Starburst

Radial Dendrogram

I discovered a radial dendrogram from @FernOfTheAndes over here and converted it to Dex immediately.  This is a rough first look, but it’ll get better.

Thrones_RadialDendrogram

 

Sankey Particles

Last but definitely not least is the Sankey particles.  It took me about 5 minutes to pick my jaw off the floor after seeing Elijah’s demo.

After I recovered from the initial shock of seeing something so shiny, my blood started flowing and I immediately emailed Elijah for permission to incorporate his work.  Elijah, being a super-cool dude, had no problem what-so-ever.

I have a few minor bugs in my translation, but here is the current state of things:

Thrones_SankeyParticles

Wrapping Up

Anyway, that’s all for now.  The next version of Dex with all this stuff should be out by next week.  I’d release it tonight but there’s a nasty cut/paste reference bug which is still eluding me.  That and it’s Thanksgiving.

Speaking of which, HAPPY THANKSGIVING!!!!!

  • Pat
Posted in General | Leave a comment

Chart Explorer

I figured I would take a break from coding to introduce a new display component within Dex called the Chart Explorer.

Below is a picture of it in action.  Click the image to go to a live version:

ChartExplorer_Screen1

This component allows you to explore data interactively within the HTML page.

How to use it:

When the chart explorer comes up initially, it appears empty.

ChartExplorer_Initial

On the left is a series of controls.  The orange button selects the chart type.  The blue buttons are the columns you wish to visualize.

By dragging these columns to the green drop zone, we can visualize the data in a number of interesting ways.

Such as line charts.

ChartExplorer_Screen1

And parallel coordinates, etc…

ChartExplorer_Screen3

If you wish to remove a column from the visualization drop zone, simply drag and drop it over to an empty region on the selection pane.  It should disappear.

Anyway, this is a rough view of where some of the Dex visuals are going.  A new minor version of Dex (0.8.0.5) should be out within the next few weeks with this capability.

Until next time!

  • Pat
Posted in General | Leave a comment

Dex 0.8.0.4

As promised, I am focuses on tight iterations of Dex.  The latest release of Dex can be downloaded here.

This update has been focused on:

  • Usability
    • You can now cut/paste multiple items within a workflow
    • Better icons for most tasks.
    • Task are tied to extensive online documentation via getbook.
    • Drag-And-Drop should work more reliably now.
  • Inline/Online Documentation
    • Most task are now documented.
    • More documentation being added daily.
  • Bug fixes
    • Too numerous to name.
  • Visualizations
    • NVD3 chart refresh
      • Multiple Horizontal Bar Chart
    • C3 chart refresh
      • C3 Scatterplot
    • Google
      • Diff Bar Chart
      • Diff Pie Chart
      • Timeline
    • Dex Charts
      • Radar Chart (Rough draft)

Enjoy!

  • Pat
Posted in General | 7 Comments

Dex 0.8.0.3 Update

Announcing: Dex 0.8.0.3

This is a minor update to yesterdays release.  The changes include:

Bug Fixes:

  • Fixed path issues in c3js demos
  • Changed legend location of c3js graphs to lay out on the right.
  • Included missing demo dataset “diffdata.csv”
  • Added a new demo for reading and graphing jstat data.

New Task:

Select Columns – Allows the user to quickly select and order columns.

  • Pat
Posted in General | Leave a comment

Dex 0.8 Released!!

After a very long time since the previous release, I am happy to announce that the new release of Dex is available from the Dex download site.

Dex 0.8.0.3

Getting a release out the door is like playing whack-a-mole.  I finally had to settle on good enough being good enough.  From here out, my delivery model will be changing from one of sporadic releases to one of continuous delivery at a cadence of 2 weeks to 2 months.

Here is a video showing the installation:

And here is a demo of some of the basic capabilities:

Good luck!  Please let me know if there are any issues with the installation or the product itself and I will try to assist.  In return, I always love to hear with folks are doing with the tool.

Take care!

  • Pat
Posted in General | 2 Comments