Building Data Visualizations with SVG+HTML+JavaScript

Posted on October 12, 2013 by bidesi

Hi,

If your day job is to build reports, I am pretty sure you must have at least once gotten comments similar to “Oh, can we move the tool tip inside the bar in the bar chart?” or “Can you also add a trend line to a Stacked Bar Chart?” and then you rush back to Excel, Tableau, BO, Cognos etc., and go check the chart’s options to see if that is available. Recently while learning JavaScript I came to learn an awesome concept called Scalable Vector Graphic. I found that JavaScript + HTML + CSS which is already an awesome combo works very well with SVG. So I set about my all time to-do task i.e., to build a chart from scratch which could be tweaked easily and plugged over data. So, without further ado…

Now, I can play around with anything here., let’s go crazy.

Now, for the Code….

<!DOCTYPE html>
<html>
<head>
<meta charset='utf-8'>
<title></title>
<style>

</style>
</head>
<body>
<script type='text/javascript'>
//console.log('Initializing....');
var barWidth = 30, canvasHeight = 300, canvasWidth = 800, barSpacing=20,toolTipPos = -25;
var data = [['India',200],['USA',60],['Africa',120],['Europe',95],['Australia',45],['Asia',20]];
var svgBase = document.createElementNS('http://www.w3.org/2000/svg','svg');
svgBase.setAttribute('height',canvasHeight);
svgBase.setAttribute('width',canvasWidth);
document.body.appendChild(svgBase);
//Function for Rendering Line
var drawLine = function(x1,y1,x2,y2){
var tempLine = document.createElementNS('http://www.w3.org/2000/svg','line');
tempLine.setAttribute('x1',x1);
tempLine.setAttribute('y1',y1);
tempLine.setAttribute('x2',x2);
tempLine.setAttribute('y2',y2);
tempLine.setAttribute('style','stroke:rgb(0,255,0);stroke-width:1');
svgBase.appendChild(tempLine);
};
//Rendering Y axis
drawLine(barWidth,10,barWidth,canvasHeight);
//Rendering X axis
drawLine(0,(canvasHeight-40),barWidth*data.length*2.2,(canvasHeight-40));
for(var i=0; i<data.length; i++){
//Rendering Bars
var tempRect = document.createElementNS('http://www.w3.org/2000/svg','rect');
tempRect.setAttribute('x',(barWidth+barSpacing)*(i+1));
tempRect.setAttribute('y',(canvasHeight-50)-data[i][1]);
tempRect.setAttribute('height',data[i][1]);
tempRect.setAttribute('width',barWidth);
tempRect.setAttribute('id','Rect'+i);
tempRect.setAttribute('fill','blue');
svgBase.appendChild(tempRect);
//Rendering Tooltips
var toolTip = document.createElementNS('http://www.w3.org/2000/svg','text');
toolTip.setAttribute('x',(barWidth+barSpacing)*(i+1)+4.5);
toolTip.setAttribute('y',(canvasHeight-45)-data[i][1]+toolTipPos);
toolTip.setAttribute('fill','black');
toolTip.textContent = data[i][1];
svgBase.appendChild(toolTip);
//Rendering X-Axis Labels
var seriesName = document.createElementNS('http://www.w3.org/2000/svg','text');
seriesName.setAttribute('x',(barWidth+barSpacing)*(i+1)-2);
seriesName.setAttribute('y',(canvasHeight-20));
seriesName.setAttribute('fill','red');
seriesName.setAttribute('id','xlabel'+i);
seriesName.textContent = data[i][0]
svgBase.appendChild(seriesName);
//console.log(i+' th element done!')
};

/*var toolTip = document.createElementNS('http://www.w3.org/2000/svg','text');
toolTip.setAttribute('x',30);
toolTip.setAttribute('y',50);
toolTip.setAttribute('fill','black');
toolTip.setAttribute('transform','rotate(30 50, 45)');
toolTip.textContent = 'Junk';
svgBase.appendChild(toolTip);*/
</script>
</body>
</html>

I haven’t spent much time on making this code leaner, but it is possible to define functions and objects and then
be able to switch between different charts on the fly.

HTML Manipulation using LeapMotion

Posted on September 28, 2013 by bidesi

Hi,

Recently, I have gotten hold of an excellent piece of equipment called the Leap Motion, And believe me it is awesome.

The below tutorial gives an example on how to manipulate HTML using a Leap Motion Javascript Library. The expectation of the tutorial is that the reader has understanding of basic programming concepts like Loops, Conditions, Functions, Objects, Methods, Properties and HTML.

The purpose of the below code is to move an image across the webpage using the inputs from a Leap Motion.


<!DOCTYPE html>
<html>
<head>
<title>Leap Graph Explorer</title>
<script src="leap.js"></script>
</head>
<body>
<script>
var controllerOptions = {enableGestures: true};
Leap.loop(controllerOptions, function(frame){
var img = document.getElementById("pic");
var wid = img.width;
var ht = img.height;
if(frame.gestures.length>=1){
document.getElementById("test").innerHTML = frame.gestures[0].type;
if(frame.gestures[0].type=="keyTap"){
img.style.width=(wid*1.1)+"px";
img.style.height=(ht*1.1)+"px"
}
if(frame.gestures[0].type=="swipe"){
img.src="./screenshot1.png"
}
if(frame.gestures[0].type=="circle"){
img.style.width=892+"px";
img.style.height=580+"px"
}
}
if(frame.hands.length > 0){
var hand = frame.hands[0];
//vectorToString(hand.palmPosition);
//document.getElementById("test").innerHTML = hand.sphereRadius;
document.getElementById("pic").style.left = hand.palmPosition[0]-100+'px';
document.getElementById("pic").style.top = hand.palmPosition[1]-50+'px';
}
})
</script>
<div id="test">start</div>
<img id="pic" src="./screenshot12.png" style="position: absolute; top: 20px; left: 15px"/></div>
</body>
</html>

As you can see that the most important part of our code is the JavaScript. Let us have a closer look shall we;


var controllerOptions = {enableGestures: true};
Leap.loop(controllerOptions, function(frame){
var img = document.getElementById("pic");
var wid = img.width;
var ht = img.height;
if(frame.gestures.length>=1){
document.getElementById("test").innerHTML = frame.gestures[0].type;
if(frame.gestures[0].type=="keyTap"){
img.style.width=(wid*1.1)+"px";
img.style.height=(ht*1.1)+"px"
}
if(frame.gestures[0].type=="swipe"){
img.src="./screenshot1.png"
}
if(frame.gestures[0].type=="circle"){
img.style.width=892+"px";
img.style.height=580+"px"
}
}
if(frame.hands.length > 0){
var hand = frame.hands[0];
//vectorToString(hand.palmPosition);
//document.getElementById("test").innerHTML = hand.sphereRadius;
document.getElementById("pic").style.left = hand.palmPosition[0]-100+'px';
document.getElementById("pic").style.top = hand.palmPosition[1]-50+'px';
}
})

To keep the post concise, I will quickly go through what is being done in the code.

Leap Motion as any other Controller Device runs an infinite loop polling for input updates from the user, Which can seen by Leap.Loop Method.
We are enabling default gestures like swipe, circle, tap by setting the controllerOptions variable.
We can get the image attributes using an object built on the document object.
To display the gesture being done, we display in a html division called “test”
We update the image’s height and width by increasing at a rate of 1.1px for each “tap” gesture. Similarly “circle” gesture just resets the size back to original.
Next in the line, is the code to change the image’s x and y based on the palm position returned by Leap Motion hand Object.

This is how we can achieve simple html manipulation using Leap Motion and JavaScript.

Time Savers

Posted on August 20, 2013 by bidesi

This article is a list of time saving code snippets;

1. List out all files in a folder using R;


file.list <- list.files('C:/Users/MyName/Desktop/')
write.table(file.list,'C:/Users/MyName/Desktop/files.txt')

2. Creating a bunch of variables without any Manual intervention;

mylist <- c(1:10)
for(i in mylist){
assign(paste('movie',i,sep=''),i+10)
}

I will keep adding more.

Unzipping a humungous number of files using R

Posted on July 22, 2013 by bidesi

Hi,

Assume you have this situation;

There are 400 such zip files and you want to unzip them. Imagine you are in an R session and do not want to exit R just for this. How do we achieve this? Can we do this quickly and efficiently? Yes, we can.

A combo of foreach, doSNOW/doMC and utils libraries will do the trick.

rm(list=ls())

# setting up working directory
setwd('/root/Desktop/multi-modal-train')

# reading list of all zip files
zip.files <- list.files('/root/Desktop/multi-modal-train', include.dirs = FALSE)
zip.files

# loading required libraries
library(foreach)
# Windows fork
# library(doSNOW)
# c1 <- makeCluster(1)
# registerDoSNOW(c1)

# linux fork
library(doMC)
registerDoMC()

# setting operation variables
base.dir <- '/root/Desktop/multi-modal-train'
i <- 1

# parallel loops for faster processing
# unzipping files can proceed in parallel
foreach(i=1:(length(zip.files)-1)) %dopar%{
unzip(paste(base.dir,zip.files[i],sep='/'),exdir = 'unzipped')
cat(paste('unzipping',zip.files[i],sep = '-'))
gc(reset=TRUE)
}

# Windows fork
# stopCluster(c1)

The output is below;

Happy R’ing

Parallel Looping, Multi-Core processing in R

Posted on July 14, 2013 by bidesi

Hi,

It has been as while since I last posted anything. If you know me, you know that I rant over loops in R. Vectorization was everything to me. I frequently avoided using Loops in R.

On a kaggle challenge, I had to process audio files. Feature extraction of these audio files gave me time series and trying to compare two large time series is a cpu intensive task. For my code which was doing distance optimization, Vectorization just didn’t cut it. Vectorization was taking a long time.

Finally, I read about parallel processing using foreach loops, and implementing them lead my optimization steps to run quickly. Parallel loops ran quicker than the vectorization code that I had written. The results were a relief. Then I read about doMC which helps utilize the multi-core processing ability of your CPU. Then I found that doMC is not available for windows. Fortunately there is doSNOW for windows that does the same thing. This improved my time even more.

For my code(which is distance optimization) the numbers are below;

Vectorization(sapply) -> 30 mins

only Foreach loops -> 22 mins

doSNOW + Foreach loops -> 18 mins

Below is a snippet of code that employs Foreach and doSNOW;


# loading libraries
library(doSNOW)
library(foreach)

# initiating cores, my
# machine has a dual core processor
my.clusters<-makeCluster(2)
registerDoSNOW(my.clusters)

i <- 0

# running parallel loops on both cores independantly
# here we are training 16 trees in parallel, on two seperate cores of my CPU
rpart.trees <- foreach(i=1:16, .packages = 'rpart')%dopar%{
data.train <- rbind(train_set1sep[which(train_set1sep$row.sep==i),-c(11)],train_set0)
rpart(ACTION~.,data.train)
}

PS: Remember that this methodology only works for code that can be executed in parallel. Think Map-Reduce with data replication factor of 1.

How to convert pixelmap to a Picture – JAVA

Posted on May 26, 2013 by bidesi

Hi,

As part of the Kaggle Challenge, I have had to visualize training data which is a bunch of numbers with pixel intensity. To convert this into a picture file like jpeg or bmp I needed convertors. I found some JAVA code online but it didn’t solve my problem. I needed to tweak it a bit to make it work for my example.

The input to the program is a pixel map like this;

To convert this into a picture, you can use the below code;

note: My pixel map is of 2304 pixel intensities which translates to 48*48 Pixel image. Hence, the code below works with this. For other dimensions update the pic accordingly;

import java.awt.image.BufferedImage;
import java.awt.image.WritableRaster;
import java.io.File;
import javax.imageio.ImageIO;

public class pixtoImage {

/**
* @param args
*/
public static void main(String[] args) {
// TODO Auto-generated method stub
File imageFile = new File("D:/Pics/0-6.png");
//3 bands in TYPE_INT_RGB
int NUM_BANDS = 3;
int[] pixelMap = new int[48 * 48 * NUM_BANDS];
int band;
for (int i = 0; i < 48; i++)
{
for (int j = 0; j < 48; j++) {
for (band = 0; band < NUM_BANDS; band++){
pixelMap[((i * 48) + j)*NUM_BANDS + band] = Integer.parseInt(args[((i * 48) + j)]);

}
}
}
BufferedImage picImage = getImageFromArray(pixelMap, 48, 48);
try{
ImageIO.write(picImage, "png", imageFile);
}
catch(Exception e){
e.printStackTrace();
};
System.out.println("Written");
}

public static BufferedImage getImageFromArray(int[] pixels, int width, int height)
{
BufferedImage image =
new BufferedImage(width, height, BufferedImage.TYPE_INT_RGB);
WritableRaster raster = (WritableRaster) image.getData();
raster.setPixels(0, 0, width, height, pixels);
image.setData(raster);
return image;
}
}

Output for the pixel map shown in the image above is

May this snippet save you some time, Have fun…..

Reading humungous csv files in R

Posted on May 17, 2013 by bidesi

Hi,

The other day, for kaggle I had to read a csv file into R. It was around 4GB in size. None of the text editors were able to open the file. Even Access text import failed. I had to work with a macro or had to look at open source RDBMS like MySql. Luckily, my teammate gave me the idea of using file stream reader and writers in Java, Also my manager showed me this link.

I didn’t need to read the entire file, for initial experiments I needed only a part of the file. So, the below is a code snippet to get a part of the csv;

# initializing file readers
x<-file('extra_unsupervised_data.csv','rt')
x
y<-file('unsupervised_trim.csv','wt')
y

# reading lines
line<-readLines(x,n=10001)

# writing lines
cat(line,file='unsupervised_trim.csv',fill=TRUE)

# closing files
close(x)
close(y)

May this snippet save you some time!!!!!

BIdesi

Work, Play and everything in between……..