Big Data — What is HBASE

hbaseThe great thing about Big Data technology is that there are so many tools in the Data Scientists Tool Belt. The bad thing about Big Data Technology is that there are so many tools in the Data Scientists Tool belt.

When we talk about the tools that we use when working with Big Data, an overwhelming majority will discuss Hadoop, the Apache foundations implementation of Map Reduce and Distributed File Systems (HDFS in this instance. Which was created by Doug Cutting after Reading papers on the subject produced by Google Engineers while he was at Yahoo. (He is now at Cloudera). But big data tools rarely if ever work alone. It is a collection of tools and databases that help Data Scientists be more effective in their analysis (or just help to speed things up).

One of these technologies is HBase. HBase is a non-relational (NoSQL) database that is a Java implementation of Google Big Table. It is what is referred to as a Columnar Database. As oppose to Relational Database which stores its Data in Rows, it stores its data in Columns.

So that’s easy to say, but what exactly does that mean. Lets start with the definition Google lays out in its document on Big Table.

” A Bigtable is a sparse, distributed, persistent multidimensional sorted map.”

Lets break that down to see what that means.


A database is said to be sparse because of lack of data but not in the traditional sense of the term which would usually mean that there are very few items in the database.  In relation to HBASE, it is called sparse because of its ability to have sparse data in its entities.  What this means is as opposed to a relational table that would  require you to fill out all (or most) of the fields (Think of a customer data table in a relational data base), a columnar database can be empty or NULL without it adversely affecting the database functionality.  In addition, this also gives you the added benefit of being able to add other pieces of data you would like to capture on the fly.  In a Relational Database, you create a schema (FirstName, LastName, SS#, TelephoneNumber) and hope that you have thought of all the data you need to capture at the time of creation.  The NoSql schema-less databases allow you to add fields when needed or discovered without interrupting the normal flow of the operations.

Distributed and Persistent

HBase utilizes HDFS (the Hadoop Distributed File System) to distribute data across several commodity servers.  This is how Hadoop and and HBase are able to work with vast amounts of data. It is based on another article from Google on the Google File System which Doug Cutting used as a basis for HDFS .  We will discuss HDFS and DFS in more detail in another post.

Multidimensional sorted Map

A Map (sometimes called and associative array) is a collection where the index of what is being stored does not have to be an integer but can also be arbitrary string.  It is a collection of Key/Value pairs where the key is unique.  The Keys are sorted in lexicographical order. (Not alphabetical, nor Alphanumeric, but sorting on the Unicode value of the string)

What you gain/give Up

Using HBase allows you to store your data both preprocessing and post processing in HBase and gives you some greater flexability and host billions of rows of data with rapid access. The downside is that when you use HBase instead of HDFS, tools like Hive (SQL like retrieval of data) are 4-5 times slower than plain HDFS. In addition, the maximum amount of data you can hold is about 1 petabyte as opposed to 30pb in HDFS.

 In the next post will dive deeper into the specifics of HBase, like set up, usage, and analysis.

New WP & W8 Apps from the West

AppsAs a Developer Evangelist from Microsoft I help developers develop apps for Windows Phone or Windows 8. I cover the Western United States. Because of that, I wanted to do a post to showcase some of the apps that are coming out of the west. Of course, because Silicon Valley is part of the West Coast, I am sure you will recognize a few of these apps, but they are not all from big brand names.

Help me support the developers working hard on the west coast to build apps. Download the apps (All are free or have a trial) and then review them too… As a developer myself, I love to get ratings and feedback for my games and apps (I included my game on here too :) )


Hotel Tonight – Windows 8 – Get a great deal on a hotel TODAY.  I love this app, I am always last minute so it not only helps me get a room last minute but find really cool hotels to stay at.

Scribed – Windows 8 – If you like to read, try Scribed to read thousands of books for $8.99 a month, I am checking it out using the free trial.

Interactive Touch Books for Kids – Windows 8 – My daughter loves this, she is in the drag drop stage and wants to play this again and again.

Madagascar Math Ops – Windows 8 – I might be starting my daughter a bit early on this but she loves the Madagascar characters.  Try it out and let me know what you think.

Dave In the Cave – Windows Phone – (Shameless Self Promotion) Help Dave Escape the Caves, 20 levels of fun.  The first 10 may be easy but then it starts to ramp up and really makes you think about your next move.  Check it out and let me know what you think.  It is the first game I created.

Baby Connect (Active Logger) –  If you have a baby you are always keeping track of things, pee, poo, food, sleep, repeat.. This is a great way to not only keep track, but to share with others that help to take care of your little one.

That’s it for now.  Download, rate, and let me know what you think too..  Im proud to show off apps coming out of the west.. keep them coming.



GameOn Event Resources

If you attended the GameOn event where we showed you how to build Windows 8 Games using GameMaker Studio, I mentioned a few resources that will help you on your journey to building your first game.  You will find these resources below.


GameOn Sample Timeline to get you from start to finish before the June 30th Deadline.

If  you are new to Game Maker and designing games, you may find it helpful to have a rough schedule of how to section out your time so that you can finish your game before the deadline of June 30th.   Since you will need the game to be in the Windows 8 Store by June 30th to be part of the contest you will need to plan accordingly. This gives us 15 days to work with.


Day 1: Ideas, Story / Character building,

On this first day, there are some decisions you need to make.

  • What type of game you are going to make (Platformer, Endless Runner, Maze, Physics Game)
  • What is the “Story” of your game.  Who is the good guy, who is the bad guy/s, possible obstacles, etc.
  • Maybe testing code fragments to see how long an idea might take to code

In this phase you want to make sure you are realistic in your idea.  Make sure it is something you can complete in the days required. (Don’t try to recreate DOOM in 5 days)


Days 2,3 & possibly 4: Sample artwork,  building the engine, testing everything as you go

In this phase you are working with “placeholder” artwork while you figure out your game.  Whether you download it or create stick figures, you will use this to do testing. In addition,  you will start with possibly one level, making sure characters work as expected (jumping, sliding, running etc) and that characters react the way they should in collisions, that bullets fire correctly (if you have any), that gravity is set correctly, etc.  This is where you will be testing out your gameplay.  This MAY turn into your first level OR just might be a testbed for things, it all depends.  Make sure you have all of the Windows 8 specific features implemented during this section.  Most, if not all, can be copied from many of the starter kits that can be found on the Resource Page.


Day 5&6: Assets GFX, sound, etc, which will have been planned in day 1

In this phase you will be creating the assets for your game.

  • Characters
  • Ground
  • Backgounds
  • Sound Effects

Make sure you are creating characters that can be worked on quickly.  Like a 4 image walk cycle instead of a 12, etc… this will keep you within scope



Days 7, 8 & possibly 9: Level design, building rooms

Once you have built your engine, you will want to start building multiple\levels or rooms.  This of course will depend on the type of game, for an endless runner you would plan on how to make it increasingly difficult and engaging.  You will use the pieces you built in the last section to pump out rooms quickly.   After doing one or two rooms, settle on how many rooms you think you can do in the time allotted (don’t over estimate) because you want to make sure the last room has some closure for the user.  Did they rescue someone, or win a prize, or escape to another world (Where you can add more levels later on with and update)


Day 9/10: Testing

This is both, testing for bugs and testing for good gameplay.  Have other people play the game and give you feedback (Incorporate what you have time for).  If you want to use Debug mode you will have to test using “Windows” mode instead of Windows 8 (Native or JS)  this will help you pinpoint and track little oversights by letting you follow variables more closely.


Day 10 – 15 Submission to the Windows 8 store.

In this section, you will package up your game (information on resource page) and submit to the Windows 8 store.  You will want to give it pleanty of time so that it is approved and in the store by June 30th.


Good Luck and happy Gaming.



Adding Windows 8 Features to a Game Maker Game

Download Template Here :

As is true for any platform, you want to make sure that you take advantage of the possibilities that the platform gives you.  Building Games for Windows 8 is no different.  In Windows you will sometimes NEED and sometimes WANT to add Windows 8 Functionality.

For example, when a game is put into snap mode (As shown below) you will need to handle this event, pause your game, and show an image in the snap mode.  This is unless you plan on allowing users to play while in snap mode.  I can see that for only a few possible games.  For most you will want to pause.









In addition to this, if you plan on having your application connect to the internet for things like online leader boards. You will need to implement a privacy policy ( which is located on the settings charm).  Your application will fail certification if this is not included.









This, of course goes hand in hand with declaring Internet Use in your Global Game Settings.











Finally, there are a couple of things you may WANT to implement. First is the Share Charm.  This can be used to share a screenshot, text, URL or other items that can help your app get more exposure by having users share your application with friends.









And last but not least.  Is implementing an app bar in your game.  You can place many of your Game Commands here if you wish.








I have included these items in a sample project and have commented it extensively.  If you need help implementing this in your own game or need further clarification on anything in the game, just ask.

Download Template Here :






Day 5 – Photo A Day May (Zoom)

OK… for those of you that are following this to see some creativity in photography, you may be disappointed in todays picture.

Ever since I looked at the list for May and saw that day 5 was Zoom… I  could not get out of my head the song from a T.V. Show that I used to watch when I was a kid call, what else, Zoom.  It was a poor mans Mickey Mouse Club.  It had a weird language called something like Ubby Wubby Nubby Tubby or something like that and had kids to a bunch of funny skits (well funny to me because I was a kid).

I really couldn’t find a good picture so to keep it in the theme, I took a pic of my screen :)













You can see the opening credits here :

It really brought back some memories. Does anyone else remember this?

Tomorrow is High Angle… .So I will get back to the “real” shots.


Day 4 – Photo a Day May (Black & White)

Since this day was “Black and White”  it could have really been anything so I took some liberty with this one.













Although now that I think of it, I should have done a Starwars themed pic…  Oh Well.

Tomorrow is Zoom…. Zoom, Zoom,  Zoom……


Day3 – Photo A Day May (Glass)

It’s getting pretty hot here in Southern California but the mornings have been awesome.  The early morning and late afternoon sun sets the stage perfectly for my Glass picture.  Bonus points if you can tell me what the bottles were in their previous life.





















Tomorrow is Black and White… that leaves it pretty wide open.  We will have to see :)



Bonus Pics:  I had a little friend visit while I was shooting.  I think he was thirsty.

















Day 2 – Photo A Day May (Bold Color)

So todays Challenge was Bold color.  One of the things I love about my house is the back yard.  It is fully stocked and spring just makes everything come alive.  This is one of the bogonvias we have in the backyard with the morning sun hitting it.  I hope it does it justice because it looks great in person.















Tomorrows picture is Glass… hmmmm not sure what I am going to do for that.