Tableau Consultant @ The Information Lab.... this is my personal blog.

Tim Ngwena

My Music Taste V2

Full Blog post to follow.

Tableau Design Month

It's Tableau design month again and to kick it off, Tableau have asked the viz community to do their own take on the recent David McCandlees visualisation on "The World's Biggest Cash Crops"

With that in mind, and with Tableau and Alteryx to hand, I got working on my own entry. Here it is.

Let drugs not be the focus here.

I wanted to focus in more on the rest of the crops on the chart. Whilst all items in the original visualisation were crops, it sort of felt like the visualisation was actually telling more of a story about how lucrative drugs were rather than focusing on everyday crops we're used to hearing about. With that in mind, I've filtered out the the drugs by default but you can add them back in by using the drop down filter at the top. 

Lets make it easier to compare crops against each other

The second thing I wanted to do was to allow you to compare all the main metrics of a particular crop with another crop. I found the circles a little hard to compare and having a separate bar charts all sorted by one measure also made that harder, increasing the number of places you had to look to get an overview. What if you could do all that on one chart? It's a great thing there's a great series of posts on radial charts over at The Information Lab blog (part 1part 2 and part 3). I also built my own slightly different grid / axis using circles rather than a pentagon in Adobe Illustrator. You can grab it too just by saving the image below. It's a PNG image so if you'd like a white version, just invert it in any image editing software or even Powerpoint and off you go.

Lets see if we can see some trends.

The last thing I wanted to do was to make it easier to see trends and so instead of just doing a radar chart which typically don't have filled shapes, I used the dual axis function in Tableau to create a polygon. This then allowed me to see the crops as shapes and as such compare crops with the same general shapes. e.g. if you just keep wheat, rice, barley and maize you can see they have the same general shape but in different scales. The same is true with palm, vegetables, potatoes and cassava. The drugs are on a scale of their own.  

How I used Tableau and Alteryx

Tableau doesn't natively support radar charts but at the end of the day radar charts work on a two dimensional plain so they can be plotted on an x and y grid if you just figure out the math behind it and give each axis a consistent measure. In this case I scored the performance of each crop against the best crop in that measure (not including drugs) and gave it a score out of 10. To speed this up a little I put the base calculations in Alteryx and got it to do the math and the data shaping in about 5 minutes of work. 

Grab everything I used and did for this viz

In the spirit of sharing, here's a zip file of everything I used and did to make this visualisation. Tableau and Alteryx both have trials and free versions that you can use to play around with. Any questions just ask below.

Listening habits using LastFM & Moves data V1.

It turns that out I've listened to Justin Beiber for a total of just over 45 minutes, across 2 albums and 5 unique tracks. This somehow happened in December 2012 and thankfully has never been repeated since.

It's been a while since my last post, (apologies on that front), but today's post follows on from that and continues my journey into visualising my personal music habits using LastFM data. A couple of weeks ago on Thursday, I was fortunate enough to be speaking at the London Quantified Self Meetup about that, video to follow shortly.

My Music Listening Habits from Ken Snyder on Vimeo.

View The visulisation

NB: the Wikipedia web view seems to not load for some people. Download the workbook and view it in Tableau Reader or Tableau Desktop.

Exporting the data 

After quite a bit of FAQ hunting and scripting here and there, I finally managed to get an export of my LastFM data in a format that I could do something with. The export included: Time in the form of an epoc time stamp, Track name, Artist Name, Album Name, Track MBID, Artist MBID and album Album MBID. Wondering what MBID's are? they're mega useful. In short they allow you to uniquely identify an attribute in the event that say two artists have the same name or two albums have the same title, more here. Once exported, the rest of the process is described in rough detail in the QS presentation above. Feel free to drop questions below.

How I got into data visualisations: starts with LastFM.

Since I started at The Information Lab, a common question I've been asked is how I ended up working with Tableau. To me the answer to that question is a little broader, I joined the team from a design and digital marketing background with no native experience of data visualisation in a business context beyond branding, marketing and communication guidelines. So how did I end up getting into data visualisations?


It starts with LastFM

In 2006, I discovered LastFM. The idea of being able to track what music I listened to was an exciting one. I wanted to believe that there was a logic to my music taste and that by tracking it, I'd discover the secret to my listening habits. The truth is, a month later when I went back, I realised I'd only tracked 100 plays. I hadn't setup the scrobbling properly so I invested some time into that and forgot about it. A couple of years later, back I came and I produced a visualisation similar to the one below.

Music Taste on Lastfm -My music taste as a tube map

I was pretty excited by this and I started wondering what else I could track. Back in 2008, there wasn't much you could do. Bear in mind phones had only just started being smart and apps were a foreign concept. However in 2009 things had picked up and Runkeeper launched. Once again I found it awesome that you could track your run and see a much more granular level of detail and stats about your run.

My first run on Runkeeper - Click the image to view the full run and map.


Then came the Feltron reports

Fast forward a few years and I came across a designer by the name Nicholas Felton. Every year or sometimes every other year, he produces a report on his travels logging his trips, photos, people he spend time with, down to small details such as his most worn shoes.

Feltron report 2013Click the image to view his reports.

Ok his reports don't exactly conform to the principals that Stephen Few sets out but nevertheless, the breadth and consistency of his tracking techniques allows him to build visualisations that paint a much richer story about his life. I started challenging myself to collect some data about my day to day activities and started wondering what other insights you could glean if you collected data about yourself. 


... then came

Conspicio is a Latin verb, it means "to see, to notice, to understand". So like most hobbyists out there, I gave my new hobby a name bought a domain, and set up a project, a project to collect enough data about various touch points of my life to create visualisations that were easy to look at, helped you notice trends and also helped you drill down further and understand that data. I quickly discovered several barriers, the biggest being simply tracking things in a consistent and meaningful way, whether it meant writing it down, logging activities via your phone / computer or automatically using a device. I scaled the project down and looked at passive alternatives and found a happy balance with fitness trackers, my tracker of choice being the Fitbit Flex (more on that in an upcoming post) - If you're interested, signup and I'll let you know when there's something awesome to see.


I'd started collecting data but what about displaying it?

... Yet another barrier, I had stuff in CSV files in excel and whilst excel had charting tools, I'd always had the vision of this data being alive, refreshing via the web through APIs and generally being more than a series of cells. I played around with various tools but never found one that handled the data in a flexible way. If you're a Tableau user, you're probably shouting "what about Tableau?"... I'd tried it in its early days but it felt like a heavy handed solution for some very simple data. It had also proved difficult to connect up to some of the APIs and databases I'd setup so I parked it temporarily. 

Instead I took to my design background and designed some simple web interfaces and graphics that allowed me to track a few basic things like steps and distance travelled but in short I'm still on that journey. 

Fitbit Data - The kind of data I've got stored


The final step 

The truth is, that's a lie, it's journey that's still happening. This month, I was fortunate to join a team of dedicated Tableau Gurus, we even have a Tableau Zen Master on the team and since then, the journey just changed lanes and I'm being exposed to so many concepts and techniques in data visualisation that I hadn't even know was possible, and to the Tableau fans, yes I take back what I said about Tableau earlier. It turns out that with Tableau, that most of the time, the question to ask is how to do something rather than whether it can do something.

Trial separation with Facebook: Part1

I'm not the first and by no means the last to do this, but I thought I'd touch on some of the challenges I'm facing and how not being on Facebook is changing the way I look at online and social experiences. Part 1 will focus on the reasons for leaving and actually doing it without breaking some of the deep links (social sign on, contacts, messaging, events, birthdays etc) that have rooted themselves almost exclusively in the platform. Part 2 will be a few months from now essentially reflecting on how it's been.

Why Leave?

That's the easy part, Facebook simply doesn't add value to my social experience both on and offline. It burns time, it's become a sea of mostly poor punditry and content that gets re-hashed over and over, it's getting super busy with adds or more sneakily, 'promoted stories' that I often don't want, and whilst privacy settings have improved, the whole system is such a mission that frankly, I'd suggest starting again to anyone whose 'cleaning' their profile.

I accept it's a free service but part of the unspoken contract that I personally held with Facebook was that I'd continue to have my information on there on the basis that the net benefit would far out way the investment in time and effort to maintain it. It simply stopped doing this months ago and I guess I decided to do something about it.

Leaving Facebook is...

... really difficult, and although the company has improved the mechanism to do this properly (delete your stuff off their servers entirely), it's a bit of a mission and can take time. My advice is this, have a trial break up with Facebook by deactivating your account. Deactivation means your account (and all your stuff) stays on their servers waiting for you should you wish to return in the future. In the event you forget something or need access again you can dive back in, reactivate your account and deactivate it again. Also Facebook add new features all the time, it's in a constant flux and should they make more steps int he right direction, it might be handy to carry on from where you left. The ideal option would be to turn you timeline into a static one-page signpost informing your contacts about other ways to reach you.

A few things to consider before deactivation

Spend one week signing out of sites that require sign in and sign in with your email address instead. You'll realise how many you signed in to using Facebook. Whilst doing this, make sure you have something like Last Pass running or Apple's Key Chain service. You'll have to remember a lot of passwords and services like those mentioned above can take care of this for you and leave you only having to remember one password. I had 43 unique sites that used Facebook login and given recent security scares, it's good practice to use a different password on each and every one.

Disconnect key services from Facebook. I'm talking about apps that might use Facebook as an authentication system or even your phone. For example iOS6 and 7 both had integration with Facebook so disabling it makes sure your account doesn't cause any problems on your device or vice versa. 

Once you're ready to put the account to sleep, download your information from Facebook. This option creates a download of all the information you've ever uploaded to Facebook and emails you the link. Once downloaded you essentially have your archive of content on your desktop. This is handy if you use Facebook as a photo storage solution or have messages that you'd like to keep. 

If you're a developer or page manager, you've still got quite a few steps ahead of you. I manage several pages and groups as part of my freelance work so before deactivating the account, you have to make sure you transfer any admin roles to colleagues or profiles. If you have Facebook apps, you'll need to shift admin roles or delete the apps. The deactivation won't work until you have. 

App deletion - Removing apps from the Facebook developer console.

Finally, let people who need to know that your going to de-activate your profile. It might be odd if they come to search for you and find you've totally disappeared. It's also quite handy to do this while still on Facebook.

Deactivation Screen - Just a few things to look after before you deactivate your Facebook account.

Do it

That's it, press deactivate and enjoy having the weight off your shoulders. 

Looking after your email unsubscribes.

I came across a great email unsubscribe experience today so thought I'd share it. I recently declared war on my inbox. I used to use my inbox like an RSS feed reader and so signed up to numerous email newsletters but now they've started to frustrate my inbox habits (inbox zero) so as they come in, I've been unsubscribing. For those of you asking why you should even bother with providing a great experience for those who want to become more distant, I recommend a brief post by Jay Bear on embracing those who reject you.

ShowYou are setting a great example for email unsubscriptions.

ShowYou made this experience awesome. First off, when I clicked the unsubscribe option, it immediately unsubscribed me. The link in my email was tailored for that purpose so I didn't need to login and all in all it took a second to unsubscribe.

The experience didn't end there though. The next thing they deserve credit for is great copy and great design. It didn't just say thanks but no thanks, but they realise that most users usually press unsubscribe out of frustration (not always caused by the sender) and so they start with an apology and their word that they won't send more emails your way.

The mobile optimised design also helps sell this tone. It's clean simple and isn't flooded with a survey asking them why they want to leave. Ok I appreciate feedback is handy but you should be asking for this in all your emails rather than during the unsubscribe phase. Instead, ask for this right next to the unsubscribe option in the body of your email. When they click unsubscribe, it's often too late.

Next up, they also realise that just because I unsubscribe, it doesn't mean I want to end the relationship with them. They offer other ways to engage with them, in this case, their app is what they want you to use instead. Those calls to actions stand out more than the the "Learn more" and "Create your channel" links. They finish it off with links to their social feed and a contact method should you feel the urge to voice a concern or provide feedback.

Pizza Express: not the best way to do it.

How does your email rate next to this? hopefully not as poorly as Pizza Express ?

It seems that they anticipated me opening the email on a desktop computer because the page wasn't optimised so I zoomed in with a couple of taps, I tried to give them feedback but then they hadn't optimised the responses to fit in the iOS rendered selection field so, I had to rotate my phone to get the full text of options. Lastly the thank you page is separate page so that again loads in tiny text at which point, I'm reaching for the close tab button rather than zooming in again and reading the text, but it turns out I wasn't missing out on much as the message was purely a confirmation message.

Is it ready yet? ... skip perfection & launch early

You may have noticed the addition of a beta label on the top left of all pages on my site. If you're not familiar with the jargon, a beta version of a website, product or app is essentially a version that's mostly there but still has some bugs, modifications or adjustments to be fixed, based on feedback from real world testing.

Surely you can test it before making it public?

Yes you could do that, however, it's often difficult to recreate some testing conditions, for example for websites, you can't predict how platforms like Wordpress will behave when loading pages over mobile phone networks or you might want to optimise the site for speed on the live server rather than the testing environment on your computer.

The second disadvantage of doing this is that you miss valuable feedback from real life people / situations. I had a soft launch for my site two weeks ago and in that time I've received valuable feedback from friends and clients who've experienced it on a range of devices and network speeds.The feedback not only highlights areas for improvement but it can also highlight areas that have been executed well, think of it as a pat on the back.

This has helped optimise the site based on real needs rather than issues which in reality make a marginal difference to a user's experience and as for the pat on the back, that inspires the developer(s) to push on (don't under estimate the pat on the back, it can be the difference between meeting your deadline and pushing it back a week).

All too often, we strive to get it right the first time around

Don't get me wrong, this doesn't mean that what your launch shouldn't have high standards, that's still important, you don't want the kind of fallout that Blackberry and the American Government have been dealing with in the last few weeks. That said, you do have to make some room for real life analytics, a chance for your audience (especially the loyal early adopters) to give you feedback and be part of your product's success, a chance to test your theories and assumptions before you push them out to your entire audience.

So what's between v 0.2 and v1 for

Between version 0.2 to 0.5 I will mostly focus on how content is displayed and then subsequent versions will see the addition of some nice to haves that I've sketched out in my notebook (image above). The full road map looks like this:

Post your thoughts or questions about beta products below. 

Iconic: Awesome icons for the web.

I've been experimenting with icon sets for the web and one of the most compelling reasons that's caused them to appear in my workflow recently is the flexibility and control they offer you compared to images. The best example is Font Awesome, a vector based icon font with over 361 icons in its library delivered through a web font and styled (size, colour & animation) with css.

But what if you could do a little more than just size colour and animation? that's what Waybury a collective based in San Francisco is doing. What I like about this icon set is that not only are they designing some of the best icon fonts for the web but they're taking it one step further with concepts such as responsive icons, collection wide styling for fills and line width, and data visualisation techniques with icon fonts. That's why I've backed the project on kick starter and you should too. 15 days to go at the time of posting. 

Tracking user interaction with Google Analytics. updated

Have you ever wanted to know how many people hover over a certain button but don't actually click it? or maybe you want to know how many people start filling in your form and and give up half way? or maybe you run an e-commerce or fundraising site where conversion rate optimisation could be improved, with more granular data about what people are doing when they look at your page? if so keep reading.

The challenge

Whilst building this site, I came across a small challenge. How do I track what pages users look at when everything other than the blog and portfolio items are one page. To clarify, when you click 'Home', 'Work' etc, a piece of code (javascript) wakes up and nicely moves the page down to save you scrolling down the one page design. For the code junkies out there I'm using Scrollit.js to do this and my code for page navigation looks like the image below, there's no href element that defines a link.

Enter 'event tracking'

This is where the Google Analytics' Event Tracking feature comes in. With some small modifications to the html and some labels as defined in the support document you can track events such as mouse actions e.g. clicks and hovers, keyboard strokes, form field entries and many more. I'm not often a fan of Google documentation but on this topic it's really good, check it out here to see what you can do. Granted it does require someone who knows what they are doing with basic html, preferably with javascript experience and a good knowledge of the CMS or environment the site runs under, but there are also plenty of tutorials out there that guide you through it.

My solution

Put simply, I implemented some click event tracking and some labels so that when Google analytics gets the data through, it's clear which buttons are being clicked and what actions are being taken.

The results

This all means that in google analytics, I get the following statistics about clicks on each of the menu items. 

Don't forget you can setup multiple events and tracking options on one page to build a richer picture of what's going. Now over to you, I'd love to know what you might use this for or if you already do, how's it working for you. As with everything on Google analytics, you have to allow for a small margin of error and the occasional quirk so don't take the stats as 100% fact but it will help guide your decision making.  Let me know your thoughts. 


Since this post was published, Google has launched new features and update the platform to include easier ways of tracking interactions. Head to the Google Blog to find out more

BBC Radio 4 Digital Hackday

On the 12 July, I was fortunate to have had the opportunity to take part in the digital challenge day being hosted by BBC Radio 4, a hack day of sorts put on to help them approach a range of challenges in innovative ways. 28 teams took part in the day consisting mostly of agencies and a team of freelancers (my team, we formed the group on the day and labelled ourselves Team Green* (long story)) and a few individuals who took the challenge on solo.

The day involved tours, huddles, plenty of muffled conversations, flip-boards, quirky creative types and of course a range of inspiring and dynamic ideas that tackled a range of issues. See the thing is I can't say much about those issues, we had to promise to keep it a secret although I won't be held responsible if the NSA were listening in. Up for grabs was the opportunity for the winning team to work with the station on their concept conceived on the day.

Whilst Team Green didn't win, after 28 pitches, 28 proposals, and 14 days of deliberation, I'm pleased to announce that we came a close second and BBC Radio 4 are in conversation with us about using the concept we developed as part of an existing campaign they will launch later this year. Awesome!

The team at Poke came out with the top prize and will be working with BBC Radio 4 on their concept in the coming months. Look out for it, it's pretty cool!

* Team Green consisted of Sarah Cole, Patrick Hanon, Gavin Brelstaff , Ashley Tirri and myself.
** BBC Radio 4 Digital Challenge Day: Link

Animated gifs: conveying more through images.

Communication is becoming more interactive and images don't always do a moment the justice they deserve; that's where gifs come in. Gifs have been making the rounds on the internet recently, you'll find them all over sites such as Buzfeed or the BBC. A gif (graphics interchange format) is just another image format but it supports sequences which can be looped forever, looped a few times or played once. The result? you can put together animated sequences like the one below that bring an image to life.

Here's some I made while on Holiday in Brussels using Jittergram for iOS.

Making a Gif

The crew at the Verge have put together a neat post on this so have a look at the full post here or watch the brief video below on how to make your own gifs.

Some things to watch out for

  • Very distracting; gifs can take attention away from content you want readers to focus on, you may have noticed I spaced out this post so you didn't immediately see more than one gif on your screen. Use a gif to add to your narrative, not take away.
  • Size matters; my general advice is to make sure you keep your gif file size as small as feasible. This means balancing between the size, quality and length of the animation. For the images I created above, I kept them under 1Mb by down-sizing them and altering the quality.
  • Frame rates and loops; try and make sure that your loops are smooth and the frame rate is appropriate for the animation you have in mind.
  • Don't get too carried away; keep it simple and clear. If you want to make something a little special invest in an app or get a trial version of the right software which can make complex gifs far easier as shown by the Verge staff in the video above. Alternatively use websites such as Blingee or Deal With It GIF Maker or simpler yet just find an already made gif using Google.
Upadte: 11/05/2013 Few apps I should have included in the post:  Cinegram lets you create looped gifs but also allows you to isolate movement, LoopCam for iOS recently got an update which brings it on par with other apps, Giffer for iOS has a less polished UI but carries a ton of features and Gifboom which is more social and sports a gallery of gifs created by other. Have a go and post your creations in the comments below.

Animated speed drawing: university of york

I've always liked the speed drawing technique especially after I watched RSA's series of animated videos. The animations are great. They animate compelling speeches adding animation that delivers a message efficiently whilst keeping you engaged by also stimulating your visual senses. It's a brilliant bit of creativity.

Lets have a go

So how hard can it be ? ultimately there's never one way of doing something, in my mind I thought a massive whiteboard, skills with a camera and some planning would have done it. That's how the technique works in most cases, just Google 'speed drawing' and 'whiteboard' together and you'll find an array of methods. However, we took a different approach centered around the illustrator and the fact that we were all working on the project remotely. Even now, the entire team that worked on this hasn't met. Hugo had a great vision for how the animation was going to come together and given some information about my skill levels in post production, he got to work.

The topic ?

I was in a fortunate position to have worked on a project at the University of York, tasked to streamline the way content is delivered to students before they start their studies. the key challenge being the information need being exceptionally high for students before they arrive but not enough bandwidth in the channels we had to get the information through effectively. most topics translated nicely into a web page with some guidance around procedures and process but when it came to explaining how the University worked and where students fitted in, most solutions fell well short of being engaging, and we had taken the view that if it wasn;t going to be interesting and consumed in an effective way we wouldn't include. that's when we explored a range of solutions and settled on video. In essence I suggested we had a go at essentially replicating the style of the RSA animate videos.

How it worked.

Hugo sketched some styles and examples whilst the script and overall production was overseen by the remainder of the team. We went through five versions of the script. That involved trying out the first section of the animation in full ( a 30 seconds clip). Hugo was hand-drawing sketches importing them into Flash CS6 ( formerly Macromdeida Flash) and then animating them by using the timeline and animation effects there. This had two advantages. The sketches were converted to a vector format so we could scale and animate freely without loosing quality and it allowed Hugo to include a sound track to make sure his animation was in time. Once all 9 sections of animation was complete, it was exported to the Flash .swf format and then imported to Adobe Premier Pro CS6. Once here, my job was to essentially piece it together. I had done the sound recordings earlier on in the process, my technique rather amateur. Essentially an iPad with a sound recorder app set to the best quality, iPad placed on a pillow to reduce echo and several takes of each section to make sure tone and audio quality was OK. Once complete, they were edited in Adobe Audition to clean them up and amplify the sound a little. the work flow in adobe software meant that i could edit items straight off the timeline without having to take them out and place them back in. i know Premier Pro CS6 isn't the industry standard but for the way I work and my basic experience, I really welcome how easy it is to move files around from one program to the next.

Closed captioning.

This was a lot less painful than I thought. I thought I was going to have to find or buy software to do this but as it turns out, You Tube takes the pain out of this process. We had a script so we just took that and followed the guidelines on how to make it transcript friendly uploaded it with the video and YouTube did the rest. It uses a clever speech recognition system and some basic instructions you add to the clip to get the timing of the captions right. Once it's done that it then allows you to download a closed caption file with time codes and everything properly marked.

Speed Drawing: Conclusion

Flash and Premier Pro don't get on. I spent the best part of 2 days tweaking and optimising sections of the video to get to a point where I could get an export out of Premier Pro that didn't have a rendering issue or scrambled sound. In the end, that involved generating jpeg files for any still shot in the video, screen recording a section rendered in Flash CS6 at 1080p, saving it as an MP4 and then importing it back into Premier Pro to get round a section of the flash file that played fine straight off the desktop but caused the rendering process to crash.

All that said, that was the first time at this techniques and you have to allow for these things especially if you have no budget and have never used some of the software together to understand their limitations. Al in all pleased with the outcome, we'll be carrying out focus groups once term has started to measure how effective this and the other streamlined content has been.

Free Adobe Indesign Calendar Template

Making a calendar in Adobe Indesign is a fairly easy process if you employ a template, or use one of the many scripts out there that gives you a foundation on which to build on. That said, many of these scripts either leave you with tweaking to do or they take the fun away from designing the calendar. So I've put together a boilerplate for calendar design in Adobe Indesign which you can download, tweak or modify very easily.

You can download the template I created below, all I ask for is a tweet. I've stripped it of content but left the styling I used. This can all be editted very quickly using the object and paragraph stylesheets which I've labelled. At some point, I'll write a script to generate the dates but till then, Enjoy !