Before you implement a BI solution…

Below pointers would be help you if you are considering a BI solution.

1. DIKI- Data-->Information--> Knowledge --> Intelligence.
Where you are on this progression line, where do you want go from where you are? Determining where you are could be tough. Stable operational/transaction processing system(s) would meet your data and most information needs. A rightly implemented DWBI solution significantly lower your costs to address complex informational needs and also serves you as a platform to deliver reliable, faster and low cost Level 3- analyses, prototypes, and to let you build Level 4- forecasting and predictive models.

2.Design Organization: Consider following functions - Operations/Infrastructure: Hardware, System software, Database, Feed monitoring teams. Data Architecture: Data modeling & standards, Engineering: Development teams, Standards, Monitoring and support teams, Data governance, stewardship-Business & IT, Data Quality/Arbitrage.
Lack of a particular function/standard poses challenges at a later point in time as your DWBI evolves. At times you can combine some functions based on size of your organization.

3.How will you reach where you want to reach?

(a) Which service provider/Partner will you choose if you’d like to outsource? : Evaluate partners. Rely on your network. Evaluate cultural fit. Rest will fall in place.

(b) Which tools you need? Evaluation of any ETL or BI tool takes 3-4 weeks. Take your partner's advice. Don't rush and buy every fancy thing out there in the market. Buy only what you need; don't go for exotic tools and appliances if you don’t need to. Remember tools cannot bring about Business Improvements; those tools are as good as you want them to be.

4.Define success: Medium term and long term for DWBI solution. Define quantitative metrics and intangibles. Cost benefits analyses, ROI over a period of 2 quarters to 20 quarters. This is very important. Usually no 2 stakeholders give same responses when asked about the success of recently deployed (<8 quarters) DWBI.

Data Integration and Data Quality Management-My views

Data Integration and Data Quality Management is essential to make a DWBI solution successful. How does one measure if a DWBI solution is successful? Answer depends on who the subject is, how long the DWBI solution has been implemented. Usually within first few weeks after implementation, solution appears to be successful; but after Business Users starts using reports and analytics for decision making , then issues surface- " Metrics do not seem to be correct" Assuming solution adheres to initial business requirements and functional requirements, what explains occurrence of these issues? Source Data. Information in a Warehouse/mart, information &intelligence provided by reports and analytics will not be free from data quality and integration issues which exist at the source.

Two Options to solve these issues exist:
1. Develop business rules to massage & manipulate data to account for source data issues. This is an easier option as it involves dealing with limited stakeholders- DWBI sponsors, technology teams, Business users. But this option is a dangerous options: as it leads DWBI presenting different information from what operational systems' report. In the long run, this option limits usage of DWBI solution and makes DWBI solution a failure

2.Right option is to clean up data at Source/Opertional systems. This option requires commitment and buy-in from stakeholders across the enterprise. Cost and complexity involved with this options are much greater than those involved with option. I recommend a phased-approach to implement this option. Each phase should have definite goals to improve data in certain subject area(s). Analyze and identify various data ingtegration and data quality issues across subject areas. Prioritize issues based on cost,complexity and functional/business impact. In a larger context, this option is closely related to the topics I discussed in my earlier post.

DQ and Data Information Knowledge and intelligence( DIKI)

Jan 12 2008: There has been recent surge of interest in Data Integration- CRM, CDI, MDM. So far the Data qaulity management - identifying and resoliving DQ issues has been largely limited to set of applications. A successful data integration should always be supported by and built on a strong DQ management established at the enterprise level; in other words, there will be and should be an increasing focus on data goverancne and data stewardship.