DataSense

I am often asked usually by programmers - What is Data Warehousing & how do I learn it? I explain to them we use all the same tools that you do but differently. That’s when I coined the term Data Sense. It describes the essence of Data Warehousing and separates Data Warehousing from rest of Programming. Every aspect of IT from Hardware / Software infrastructure to Design, Development and QA is done with massive data flows and need for data precession accuracy and meaning.

Sunday, February 18, 2007

What is a BIG data warehouse?

When is a data warehouse BIG? Number of tables? A couple of Hundred Dimension tables and few dozen fact tables. That defeats the very purpose of data warehousing? A data warehouse must have as many dimensions as you want to anyalyze the business by. And no one thinks in 150 dimension space. It must have as many fact tables as there are completely independent transactions - when they are of completely different grain & location in dimension space that they cannot be put together.

Is it then a large fact table? Do you really want to store and analyze on fly data at such lowest of low grain? The transaction should be at a grain you want to analyze the business by.

A data warehouse is BIG when it is at it is BIG at its right size. Otherwise its just a BIG Mess.
Posted by at