Now that Master Data Management (MDM) technology has been around for nine or ten years, what is the state of play in the MDM tool market? Are they relatively mature, similar to BI or Portal technology, for example? Or are they still somewhat immature?

My contention would be the latter, unfortunately. In Gartner terms they are somewhere between the Peak of Inflated Expectations and the Trough of Disillusionment (my view not Gartner’s, read about the Gartner Hype Cycle.

Niche tools still proliferate, big vendor solutions immature

Although there has been a lot of consolidation in the MDM market place, niche solutions still proliferate and the mega-vendors (IBM, Microsoft, Oracle, SAP) have yet to attain anything like the level of dominance they exhibit in the BI market place. Although none of these vendors would admit as much, BI technology has now evolved to the point where there is a reasonable degree of product-set homogeneity, and buying decisions in this market are as likely to be motivated by non-functional factors as they are by who can produce the prettiest dashboards.

MDM business processes are immature

The supporting business processes around BI are extremely mature – businesses have been producing management reports for more than one hundred years, even if BI technology has only enabled that process for 25-odd years (itself a long period in IT terms).

Master Data business processes, on the other hand, are relatively recent – although Master Data has been maintained in systems since the dawn of the computing age, the business problem which Master Data Management looks to solve has only arisen recently, as corporate systems multiply, diversify and require “data integration” in order to stay consistent. In fact, it’s interesting to note that MDM as a genre first appeared in the early 2000s and was to some degree initially shunned by the big ERP vendors who told customers “you won’t have this problem if you just buy everything from us”. After a while, it became clear that most organisations would in fact continue to have some diversity in their application landscapes, and even those that could paint their landscapes with one vendor still suffered Master Data problems anyway. And so the Master Data Management system was born.

In today’s world, data is ever more important, ever more diverse, ever more dispersed and ever bigger. So the challenge of Master Data Management is now high on the agenda of most CIOs. The million-dollar question is – are there tools available to help solve it?

What MDM Tools are Meant to Deliver

The concept of MDM is quite simple – all-important Master and Reference Data (e.g. countries, states, cities) should be maintained in a single repository, with appropriate maintenance and control processes. It should be automatically integrated to down-stream applications in accordance with their needs. These applications should not then provide maintenance capabilities of their own – they are consumers not producers. Let’s take a minute and explore what that would look like in a typical large business:

• It would mean that all product, customer, geography, sales-force, supplier, employee, financial, IT and company location data would be managed in the MDM solution.

• It would mean that all corporate applications would no longer have their own stores of this data, their own maintenance screens, their own concepts of the form and structure of this data – this would all be managed by the MDM solution.

• It would require extensive and flexible integration architecture to ensure that all the data got to the right applications, as they needed it.

• It would encode a vast number and variety of business rules: to ensure consistency and validity of Master Data according to the different business rules, which exist across all types of industries and businesses. For example, a new Product can’t be created without a complete Bill of Materials; all new Customers must have a complete address with Country, State, City, Street and Responsible Sales-Person.

• It would have a large number of users, across every business process.

• It would be regularly fed with updated Reference Data from an authoritative external source.

MDM Tools Today

This describes the vision for MDM, which was commonly outlined during the early days of the genre. Harsh reality has interjected now and the marketing pitch tends to be more realistic. So what is feasible with the tools that are currently available, and making the optimistic assumption that you are ready to tackle the process and cultural change needed to achieve the business benefits?

Well, MDM implementations are now normally described as “single-domain” or “multi-domain”. This describes whether they focus on a single entity, such as Product or Customer, or multiple entities. Multi-domain implementations still seem to be relatively rare, and one of the reasons for this is that there are relatively few MDM solutions that support multiple entity domains. Single-domain solutions focused either on Product or Customer are the most common deployments, and in fact several of the leading MDM solutions major on one or the other.

One of the other headaches for companies considering buying a MDM product is that some of the mega-vendors actually offer more than one MDM product, and some offerings are recent acquisitions whose relationship to other vendor products is not yet clear.

Author: Dave Littlewood, Principal Consultant, Altis Consulting.  Dave is an Information Management specialist with 15 years’ experience in strategy, architecture, implementation and management in Information Management solutions across Europe, the US and Australia. His experience includes consumer products, telecommunications, agri-business, utilities and Government.

I’ve previously had a simplistic view of Master Data Management (MDM), that its aim was to build and maintain a ‘Master Entity’ that is clean, unique and free of data quality problems. Something that proves costly on most Data Warehouse Business Intelligence (DWBI) implementations. However after attending William McKnight’s Master Data Management course I now realise there’s much more to it.

MDM is a more integral data solution than current DW implementations. Traditional DW solutions are a one-way street of information traffic: extracting, transforming and loading data from one system to another.

In this approach, latency and distribution are issues. MDM, on the other hand, allows consumption and update of source records in real time, and at the same time publishing quality data to other applications downstream such as DWs, Analytics, etc.

Master Data Management surfaced as much more than reference data. MDM provides clean, quality information that can be ‘distributed’ throughout the enterprise applying total governance on such information. The distribution concept introduces actors in the information management chain such as ‘consumers’ and ‘subscribers’. The advocacy was towards an MDM engine closed to the source systems applying data quality upon data entry.

MDM is blurring the boundaries of Data Warehouse to some degree, questioning the relevance of it in the future. According to McKnight, MDM subject areas enable analytical applications straight off the MDM data hub.

Although possible, I question the effort needed to implement an analytical MDM model that will provide the same capabilities of a well-designed DWBI solution, considering that the industry does not provide for example, MDM data models advocating best practices.

From an implementation perspective, vendors push hard their own models but in most cases, the final cost to customize those models for particular business models is extremely high.

MDM advocates the role of a data steward. Someone who is the business owner of corporate data, responsible for sign-off, approval and updating their value. This is done through what McKnight calls Master Data Workflow.

The workflow essentially passes the ball to the business prompting them to make the decision if a particular data item is fit-for-purpose and can be given the green light for its use enterprise-wide.

Enterprise Master Hub. MDM enables data as a service (DaaS). This ultimately provides the holy grail of quality information services: “build once, use many times”. On this front, MDM clearly provides great benefits up and down streams of data flow.

Obviously there are many other important points on MDM. It is not difficult to see why the value proposition of an MDM solution is more than interesting, particularly in situations where businesses are embarking on the upgrade or decommission of their ERP and CRM applications.

MDM can be applied at any time, however these represent prime opportunities to start the MDM journey. The context is perhaps a little more prohibitive or challenging in private enterprises, as well as government departments with mature DWBI capabilities. In these situations, it is wise to tackle such a journey with baby steps.

Perhaps the simplistic view that I started with is not such a bad idea after all. We should start by taking care of core enterprise entity data quality problems, applying quality processes and governance around conformed dimensions in order to acquire their golden status where the entity is clean, unique and free of data quality problems. After a successful implementation, and having the backing of the business, then move onto the next stage towards a complete MDM solution.

About the author: Florencio Espejo is a Senior Consultant with Altis Consulting, and has more than 10 years experience delivering IT solutions specialising in Information Management. He has a varied background working with organisations across a range of fields including manufacturing and healthcare services, as well as Government.

A colleague and I recently attended Sharepoint Saturday in Canberra. A day-long conference of all things MS Sharepoint. The sessions were broad ranging, appealing to business focussed and highly technical alike.

Most business intelligence tools in the market today are accessible via a browser-based interface or portal. With the increasing use of collaboration tools, integration between these and business intelligence tools to give users a single platform to meet all their information needs is an interesting direction.

My key takeaways from the day were that many of the principles advocated for best practice implementations of Sharepoint 2010 have significant parallels to data warehouse and business intelligence best practices, and that the rise of unstructured data is challenging the traditional view of a DW/BI solution.

Information Management Best Practices advocate:

• Finding information easily, and ensuring it is the right version (Getting the right information to the right person at the right time)

• Maintaining a single version of the truth

• Reducing duplication of processes or recreating information repeatedly to be more efficient

• Using an iterative approach to developing the solution

Fundamentally these principles are common sense but need to be underpinned by good information management governance. The key theme that came across from many of the presentations was that technology alone, in this case Sharepoint 2010, will never be the entire solution. This technology provides a flexible and easy to use platform that encourages self-service and collaboration.

Unstructured Data

With Sharepoint 2010 Microsoft is offering a platform that allows business users to access both their structured and unstructured data from one place. Centralising emails, documents and spreadsheets (the bane of many a traditional data warehouse) into one place, and offering an integration of these different data sources with structured data in a way that appears seamless (to the user) is a very attractive promise. The proof will be in the success of implementations that can deliver upon this.

We look forward to the possibility of a user group session that explores Sharepoint 2010 for business intelligence in more detail in the future. Thanks to the Sharepoint User Group in Canberra for arranging this session.

About the Author: Nadine Hamilton is a Senior Consultant with Altis Consulting and has over 10 years experience in managing, designing and implementing Data Warehouse and Business Intelligence solutions. Nadine has been involved in multiple DW and BI projects in the UK and Australia, in Finance and Government.

Source: Sharepoint and Information Management Best Practices

The engine will identify struggling students or those not engaging with their program of study and enables UNE to provide early intervention.

by Chloe Herrick

The University of New England (UNE) has launched an “automated wellness engine” in an effort to improve engagement with students.

The engine, designed by UNE and built by Altis Consulting in conjunction with the university’s ‘Early Alert’ program, provides early identification of students who may be struggling or experiencing difficulty engaging with their program of study. Once identified, the university can refer students to appropriate support services.

The new application uses the university’s data warehouse to extract student-related information from eight separate systems, analysing the data against 34 triggers which are potential indicators of student problems or potential disengagement from their studies. Some data is self-reported – for example happiness ratings generated by emoticons embedded in UNE’s “MyUNE” portal – while other data relates to students interactions with the university and their teachers, use of facilities, and their responsiveness to deadlines.

UNE assistant director student services, Rhonda Leece, said the idea of engagement was something the university had always taken seriously and with more students going into the online environment, the institution saw the need to evolve that experience.

“With Early Alert we are fundamentally renewing our commitment to students,” she said in a statement. “It’s a systems-driven strategy delivering personal connections that make distance education students feel part of the University.”

The engine was developed by Altis throughout 2010 with extensive trials of the system carried throughout the year. The trial indicated unit attrition rates for students involved in the trial significantly decreased, falling 7.5 per cent during the semester.

The engine’s daily reports identified the highest risk students then used that information to generate automatic email offers of support via UNE’s RightNow student relationship management system. AWE also generates trend reports for the wider university body, identifying information such including reasons for discontinuation of studies, or overall student happiness within individual schools or courses.

The uni claims all information remains confidential and students can participate if they choose.

SAP is experiencing huge interest in the release from existing customers, said Dave Weisbeck senior vice president of BI and EIM solution management.

Part of that may be due to pent-up anticipation, as the launch was originally expected to occur last year.

No critical problems with the software led to the delay, according to Weisbeck. Instead, the timing shifted due to the release’s ultimate scope, which changed further as SAP sought to tie it to technologies for in-memory data processing and CEP (complex event processing), he said.

In a demonstration, SAP showcased how its recently launched HANA (High Performance Analytic Appliance) database technology could dramatically speed up performance of a BI tool for analysing automobile recalls, compared to a traditional data warehouse.

SAP is betting big on HANA, rolling out a series of specialized applications that will sit on top of the software, which puts data into RAM for faster performance compared to reading off disks. For now, SAP is not positioning HANA as something that could replace databases such as Oracle’s, which underpins many SAP installations, but that could change as the product matures.

Later Wednesday, the presentation turned to mobile device support in the Business Objects 4.0 release, including for tablets like Apple’s iPad, Samsung’s Galaxy Tab and Research in Motion’s PlayBook. Some observers believe tablets will spark a major uptick in mobile BI adoption, since the larger form factor makes for a better user experience.

“In-memory plus mobility is the killer app from [SAP],” said Steve Lucas , global senior executive, business analytics.

Other components and capabilities in 4.0 include Data Services, for data integration and consolidation; Information Steward, which enables users to assist with data-quality efforts; and the CEP application Event Insight.

SAP also discussed the release’s operational improvements, which include better scalability through native 64-bit support and support for virtualisation.

In addition, Business Objects 4.0 delivers a more common look and feel across the modules, as well as better integration between them, said Forrester Research analyst Boris Evelson , via e-mail.

Business Objects 4.0 follows major BI upgrades delivered by IBM and Oracle within the past year.

Vendors are keen to strengthen their BI portfolios, since the software has continued to sell despite the global economic downturn, as enterprises seek competitive advantage and deeper insights into their business.

Larger BI vendors like SAP, Oracle and IBM are competing on every aspect of the BI stack, which can be decomposed into scores of components, from ETL (extract, transform and load) tools for collecting data, all the way to reporting and analytics, Forrester’s Evelson said in an interview this week.

“The point is, not everyone can do everything,” he said.

However, companies that are heavily standardized on application and database products from vendors such as SAP or Oracle “have very little reason” to look beyond those companies for BI tools, Evelson said.

But for companies with highly heterogenous environments or the desire for advanced capabilities like predictive modeling, “the choices are very, very wide,” he said.

As for existing Business Objects customers, the move to 4.0 should not entail any changes to licensing, Weisbeck said during a question-and-answer session.

SAP customers will also get a taste of the release’s capabilities in the next “enhancement pack” for the Business Suite ERP (enterprise resource planning) platform, which will include embedded dashboards and reports built on 4.0, Weisbeck said.

Also Wednesday, SAP announced six new industry-specific BI applications, which are based on the current 3.1 release. The next wave of industry applications will be created on 4.0, an official said.

Key Data Warehousing Issues to be Discussed at Gartner Business Intelligence Summits, February 22-23 in Sydney, and May 2-4, Los Angeles

STAMFORD, Conn., February 9, 2011— CIOs need to familiarize themselves with nine key trends in data warehousing and how they will impact the cost-benefit balance of technology deployed to deliver business analytics value during 2011 and 2012, according to Gartner, Inc.

Gartner analysts said the data warehouse is set to remain a key component of the IT infrastructure and believe that, as the demand for business intelligence (BI) and the wider category of business analytics increases, optimization, flexible designs and alternative strategies will become more important.

“The data warehouse remains one of the largest — if not the largest — information repository in the enterprise,” said Mark Beyer, research vice president at Gartner. “Only by being aware of the key market trends and how emerging technology solutions will blend with proven practices can the CIO avoid budget waste through ‘misdirection’ by the data warehouse management and delivery team.”

Gartner has identified nine key trends in the data warehousing market for 2011 through 2012:

Optimization and Performance
Advanced functionality for hardware management of input/output (I/O), disk storage and CPU/memory balancing are now included almost as a matter of course in data-warehouse-capable platforms. Some new entrants are focusing on optimization as a differentiator and nearly every data warehouse vendor is now addressing the issue of optimizing storage for the warehouse via compression and usage-based data placement strategies. Vendors are also expending great effort differentiating their products on performance claims and technology, in ways that are not necessarily significant to the use case.

Data Warehouse Appliances
Although there are many reasons why organizations consider buying an appliance, the main reason is simplicity. The vendor builds and certifies the configuration, balancing hardware, software and services for a predictable performance. The appliance is delivered complete and installs rapidly. If there are any problems, a single call to the appliance vendor is the first course of action. There is a secondary effect as well, in that appliances can speed delivery by avoiding time-consuming hardware balancing.

The Intensive POC
During 2010, most organizations heeded Gartner’s advice to perform a proof of concept (POC) with a “shortlist” of vendors during the selection phase of the data warehouse database management system (DBMS). Gartner recommends that POCs use as much real source-system extracted data from the operational systems as possible, while performing the POC with as many users as possible, creating a data warehouse workload that approaches that of the environment to be used in production.

Data Warehouse Mixed Workloads
There are six workloads that are delivered by the data warehouse platform: bulk/batch load, basic reporting, basic online analytical processing (OLAP), real-time/continuous load, data mining and operational BI. Warehouses delivering all six workloads need to be assessed for predictability of mixed workload performance as failing to plan for mixed workloads will lead to increased administration costs over time, as volume and additional workloads are added, potentially leading to major sustainability issues.

The Resurgence of Data Marts
A data mart is defined as an application-specific analytic repository of any size, normally with a specific, smaller group of users than a data warehouse. Data marts can be used to optimize the data warehouse by offloading part of the workload to the data mart, returning greater performance to the warehousing environment.

Column-Store DBMSs
Column-store DBMSs generally exhibit faster query response than traditional, row-based systems and can serve as excellent data mart platforms, and even as a main data warehouse platform. Gartner foresees several vendors changing the pricing model for the software from a more traditional per-user or per-core model to a price based on the volume of data loaded into the database.

In-Memory DBMSs
In-memory DMBS (IMDBMS) technologies exhibit extremely fast query response and data commit times and introduce a higher probability that analytics and transactional systems can share the same database. Analytic data models, master data approaches and data services within a middle tier will begin to emerge as the dominant approach, forcing more traditional row-based vendors to adapt to column approaches and in-memory simultaneously. BI solutions will emerge sooner rather than later, and these will leverage IMDBMSs with superior-performing products and will quickly become acquisition targets for megavendors.

Data Warehouse as a Service and Cloud
In 2011, data warehouse as a service comes in two “flavors” — software as a service (SaaS) and outsourced data warehouses. Data warehouse in the cloud is primarily an infrastructure design option as a data model must still be developed, an integration strategy must be deployed and BI user access must be enabled and managed. Private clouds are an emerging infrastructure design choice for some organizations in supporting their data warehouse and analytics.

Using an Open-Source DBMS to Deploy the Data Warehouse
Open-source DBMSs are still being used in both experimental and more formalized approaches. At this point, open-source warehouses are rare and usually smaller than traditional ones and also generally require a more manual level of support. However, some solutions are optimized specifically for data warehousing.

Additional information is available in the Gartner report “Data Warehousing Trends for the CIO, 2011-2012.” The report is available on Gartner’s website at http://www.gartner.com/resId=1530314.

Based on the technology de the famous iPad, the IBPad system supports special content to facilitate supervision and control tasks by some 30 supervisors or agents per shift who are responsible for customer and passenger care at the airport. The new portable terminal will enable such staff to monitor developments in more detail and to respond to customer needs more quickly and efficiently.

The continuous supply of information from Iberia’s Barajas Hub Control Centre will enable these agents to access a wide range of up-to-date information on which to base their decisions with regard to customers. This information includes:

•  The real status of check-in counters (open, closed, number of employees on duty, scheduled time for closing and departure). The IBPad can also show images taken by the cameras in the area to show the length of queues, etc.
• The list of passengers with reduced mobility on each flight.
• The list of unaccompanied minors on each flight with full real-time information /type, minutes remaining, individualized confirmation of pickup by Red Jackets, etc.).
• Information on passengers at risk of missing connecting flights and their status as customers (Iberia Plus Platinum or Gold cardholders, etc.)
• Information on passengers on intercontinental flights that have not yet checked-in and the number of minutes remained before closing the flight.
• Flights at risk of being delayed or cancelled and any other incident that would impact on customers, by departure zone and the number of customers affected.
• List by departure zone of the status and progress of boarding procedures: starting times, possible delays, staff on (number, names and ranks), percentage of passengers boarded, etc.
• Baggage remaining to be loaded on each flight, with passenger name, as well incoming flight with baggage incidents, along with the number and names of affected passengers.
• View of main customer care documents such as passengers’ rights, detailed procedures, style guide, audio flight announcements, information in several languages for customers, maps of airport zones, service schedules, etc. 
• List of employees due to come on duty, with clock-in times and first scheduled tasks. 
• List of employees with the least experience and their location and duties at all times, enabling supervisors to monitor them more closely.
• Details of all the check-in and boarding tasks to be performed for the companies served by Iberia, with confirmations of times, employees assigned, counters, and gates.

The IBPad system is loaded with 30 different applications which together put the entire airport in the palm of the employee’s hand.

Iberia’s handling unit, Iberia Airport Services, continues to use technology to enhance quality, and it Hub Control Centre and associated services at Madrid-Barajas are regarded as model for the world’s airlines.

Having had the pleasure of using an iPad over the Xmas/New Year break, I thought it would be of interest to review what ‘apps’ are available to enable Business Intelligence on the iPad.

Some of the newer vendors like Roambi, may have the upper hand in creating a visually and engaging experience, as their software architecture is purpose-built from the ground up for the iPad rather than an addition to an existing BI product line. For the moment I’d pick Microstrategy and RoamBI to evaluate further, with RoamBI being used for a proof of concept/’selling the sizzle’.

From what I’ve seen so far the iPad looks very well suited for ‘dashboards’, but then that’s probably not a big surprise, and there are definitely certain industry sectors where it will work well; one being the Wealth Management industry which I’ve worked in, where presentation and reporting can make or break a client signing up and remaining with the wealth manager.  You could imagine how well a meeting is going to go between a client and a private banker, where the private banker is able to run through the clients portfolio using the BI app on the iPad.  That’s going to look pretty damn good, and even better if the client can access the same information via their iPad after the meeting.  In fact any meeting where you want to show information to another person, and in particular a client or prospect, the iPad is going to win hands down, lets hear it for the end of those dull power point client facing meetings!

Articles of interest:

Hilton CIO: Mobile device use in hotels is skyrocketing

Project X Ltd. Aims to Lead Business Intelligence Mobile App Development

Mobile BI: How to Run Your Business On the iPad

BI Apps for the Apple iPad

iPad: What It Means for Visually Interactive Business Intelligence

I’m planning to try to keep this post up to date, so do let me know if I have missed off vendors.  You may also want to comment on whether the software works or not, any success stories or other links of interest.

The Blog-Health-o-Meter™ reads This blog is doing awesome!.

Crunchy numbers

A Boeing 747-400 passenger jet can hold 416 passengers. This blog was viewed about 7,900 times in 2010. That’s about 19 full 747s.

In 2010, there were 53 new posts, growing the total archive of this blog to 129 posts. There were 2 pictures uploaded, taking up a total of 170kb.

The busiest day of the year was April 8th with 72 views. The most popular post that day was The Top 10 Trends for 2010 in Analytics, Business Intelligence, and Performance Management.

Where did they come from?

The top referring sites in 2010 were en.wordpress.com, linkedin.com, google.co.in, and blogs.gartner.com.

Some visitors came searching, mostly for enterprise information management, advanced analytics gartner, gartner advanced analytics, master data management benefits for the enterprise, and oracle elt.

Attractions in 2010

These are the posts and pages that got the most views in 2010.

The Top 10 Trends for 2010 in Analytics, Business Intelligence, and Performance Management December 2009

The Advantages of Data Warehouse Appliances Revisited May 2010
3 comments

Advanced Analytics included in Gartner 2010 Top 10 Technologies to watch October 2009

Is it Time to Switch to ELT? June 2010

Kimball Group Design Tip #122 Call to Action for ETL Tool Providers April 2010