The WordPress.com stats helper monkeys prepared a 2012 annual report for this blog.
Here’s an excerpt:
600 people reached the top of Mt. Everest in 2012. This blog got about 9,700 views in 2012. If every person who reached the top of Mt. Everest viewed this blog, it would have taken 16 years to get that many views.
I recently attended a SAS Business Analytics seminar in Melbourne. The session provided insight from SAS, answering the following questions:
The session was well attended with a number of different industries and organisations represented.
The key takeaways for me:
a. Data Governance and Data Quality – As with any data project, if data quality is an issue the business insights you’ll generate will be at best low value and at worst wrong. A Business Analytics project, not unsurprisingly, needs good quality data.
b. One version of the truth – Integrated data ensures consistency of the insights generated and provides an easy access path to data.
c. Operationalisation of intelligence – Reducing the risk of having business insight locked up in individual resources, operationalisation of intelligence ensures insight generated can be used across the organisation.
d. Big Data!
There was a great case study,which for me really highlights the power and competitive advantage of Business Analytics.
State Fleet of New South Wales, Australia, have been able to accurately set the lease price of the 12,000 cars per year they lease to NSW public sector workers, by using ‘Predictive Analytics’ to accurately forecast the residual/sale value of a car at the end of the lease period. With this capability they are saving millions of dollars in potential losses. With a fleet of over 25,000 cars, every 1% error in the calculated vs. actual end of lease sale value of their fleet of cars will cost State Fleet over $3 million.
No seminar/presentation would be complete without a section and discussion on Big Data, and SAS gave us their take on Big Data, including what SAS see as the fourth V of big data – Value. When you think about it, this really is far more important that Velocity, Variety and Volume. Big Data ‘Value’ for SAS means focusing effort on analysis of data where high value insights can be generated. SAS further define Big Data as being able to perform analytics in a much shorter timescale than previously possible – ‘High Performance Analytics’ – so it’s not all about how big the data set is you have for your Big Data initiatives. You can still be doing ‘Big Data’ with a small amount of data; where the data set contains a large amount of untapped high value business insight but requires a high level of processing to unlock that value.
We had some practical demonstrations of current and new SAS tools that support Business Analytics – visualisation, mobility and reporting.
Altis has been actively engaged in a number of successful Business Analytics projects over the last few years, and this seminar has strengthened my understanding and belief that successfully establishing and embedding Business Analytics within an organisation can generate massive competitive advantage.
I look forward to sharing our success stories and blogging further about our thoughts on how to deliver insights via Business Analytics. Altis expects 2013 to be another growth year for Business Analytics, with the smart companies using it to their advantage.
July 16, 2012 By Simon McAlister (Altis Consulting)
From time to time, ETL will highlight data quality issues. There is often a choice between fixing the issue at the source or in the ETL processes.
In this blog I argue that the issue should be fixed at the source, but if that isn’t practical I provide some guidelines to ensure the best outcome.
Correcting the issue at the source is the gold standard as this will not only correct the data itself, but may address the root cause of the underlying problem (be it process, standards or a technical error). It’ always worthwhile aiming for the problem to be fixed at the source, for the following reasons. In other words … Go for Gold!
If the issue is fixed at the source, it can prevent similar or related data quality issues from re-occurring. A one-off correction made within the receiving system is no guarantee that the data won’t be reloaded or re-submitted incorrectly in the future, and the issue will re-appear. Correct it at source and it stays fixed.
If the issue is fixed at the source, not only is it fixed for the warehouse, it is also fixed for the source system and any other upstream users of the data. Never underestimate the breadth of impact a data quality issue may have. Imagine a spider’s web with lots of tangled threads that radiate out from the centre and you’re looking at a how data quality issues can thread through an organisation . Enforcing a solution at the source means everyone benefits from a consistent and correct view of the data.
Even though it may appear easier to just patch in the warehouse, appearances are often deceptive. It’is worthwhile keeping the following saying in mind:
“If you want quick and dirty, we can guarantee the dirty but not the quick”. One example of hidden costs is reconciling the source and the target.
The reality is though, sometimes we are forced to apply a patch in the data warehouse . The root cause may involve external systems over which we have little control. The time needed to co-ordinate the fix may be long and complex, meanwhile we need to do something to keep the data flowing.
In this case we need to settle (at least temporarily) for silver, but here are some steps to ensure that the fix is as effective as possible:
Ensure the issue is communicated so that other users of the data are aware of it. Those responsible for the root cause should know so that it at least doesn’t increase in impact. Highlighting the impact of data quality issues may be all that’s needed to get the ball rolling towards addressing it.
Minimize the flow-on effect of the issue. In other words, avoid becoming part of the problem. Once the root cause is fixed, it should be easy to remove the temporary fix. Don’t comprise on your standards to attain the best data quality possible. Remember – go for gold!
Keep an eye on progress towards fixing the root cause. Don’t drop the issue because the symptoms have disappeared for the moment.
Keep aiming for gold in data quality!
Simon McAlister
Source: http://www.uk.capgemini.com/news-centre/news/crunch-time-for-big-data/ and
Companies are awash with data, some generated by their customers or systems, some by third parties. These data are growing so fast – by about 2.5 exabytes a day – that 90 per cent of the stored data in the world today has been created in just the past two years, earning it the geeky moniker “big data”.
Whether big data becomes an organisation’s greatest asset or one of its gravest liabilities depends on the strategies and solutions it puts in place to deal with the epic growth in data volumes, complexity, diversity and velocity.
This message seems to be getting through. Among the survey’s other findings, respondents said the use of big data has improved businesses’ performance, on average, by 26 per cent and that the impact will grow to 41 per cent over the next three years.
Almost 60 per cent of companies said they planned to make a bigger investment in big data over the next three years, suggesting that the era of big data and big data analytics has already arrived.
To read the full article on FT.com, please click here: Crunch time for big data
Related links:
There is a massive amount of hype and buzz in the Data Warehousing and Business Intelligence market place surrounding the term ‘Big Data’. Recently we have even seen talk of Big Data as a replacement for Data Warehousing. I believe this is a misunderstanding of what Big Data is. In fact Big Data strategies only work if they co-exist with a well thought-out and supported Enterprise Data Warehouse. So I don’t believe we are witnessing the end of Data Warehousing – and here’s why.
First, what is Big Data? In John Bantleman’s recent blog Raw is More, he defines Big Data using the criteria of volume, velocity, variety and value. This is a great definition and captures exactly why the hype, buzz and excitement around Big Data will be with us for some time – businesses now have the means to collect, store and analyse huge volumes of data, from varied sources, at high frequency, in a very cost efficient manner – and this hasn’t been possible before.
I recall the days during the first dot-com boom, where trying to capture and store all the detailed data generated by people browsing a website – capturing every click, interaction and page viewed, over a period of more than a month was near-on impossible. A client involved in providing share trading services couldn’t hold more than 14 days’ worth of detailed browsing data – so think how difficult it was to generate insights into user behaviour. With the arrival of Big Data, this problem is no longer present; it’s possible to save the detail data for much longer.
So where does an Enterprise Data Warehouse (EDW), fit into the picture? Are we now witnessing the demise of an EDW, to be replaced by ‘Big Data’ systems? In short … no. For an organisation to get value out of their data they must be able to generate insights, quickly, effectively and for as many user groups as possible. For this you need a well-structured Data Warehouse.
In a recent Australian CIO article, ‘Five things CIOs should know about Big Data’, the misinformed idea is presented that in some way ‘Big Data’ allows an organisation to forgot all the hard work and thinking that goes into creating a well-constructed Enterprise Data Warehouse (EDW), The article suggests that a Big Data implementation will enable;
The article goes even further suggesting that ‘You can use a [Big Data repository] as a dumping ground, and run the analysis on top of it, and discover the relationships later.’ I’ve seen ‘data dumps’ and they are not fun to use for anyone. They typically suffer from extremely poor data quality, poor performance and lack of control – all of which is the reason we’ve spent 20 years refining the approach to supporting business in generating insight from Data Warehouses!
We believe that both ‘Big Data’ and Enterprise Data Warehousing need to co-exist, supporting the need for organisations to generate insight from all data. Big Data provides the deep analytical capability to generate insight from huge volumes of data and transactions that you just wouldn’t need to make available to everybody on expensive hardware, whereas an Enterprise Data Warehouse is bringing insight from data to as many business users as possible, in a structured and planned way.
Is there a meeting point in the future? We believe there could be – a ‘Big Data 2.0’ where an Enterprise Data Warehouse can take advantage of the infrastructure approach that ‘Big Data’ uses. In the meantime if your ‘Big Data’ vendor tells you that you don’t need that Data Warehouse any more, come and talk to us at altis Consulting for a more rounded and balanced view.
Australian innovation to revolutionise world ports
May 9, 2012 By andrew painter
The need for Data Integration has never been greater. Organizations are faced with an overwhelming growth in data, an increasing range of diverse data sources and the continued proliferation of systems within the enterprise.
In this blog I will talk about the hot topics and key trends for Data Integration in 2012 and beyond.
First, a definition – ‘Data integration refers to an organisation’s ability to gather data and information residing in multiple sources, combine it and provide a unified view to support business goals and initiatives’.
So what are the Data Integration hot topics and trends?
• Data Integration Platforms – the right mix of capabilities
• Managing social data
• Profiting from ‘Big data’ and analytics
• Driving efficiency from Cloud Data integration
• Data quality – the essential ingredient when taking data from multiple sources.
This week we will focus on trends in Data Integration Platforms.
Data Integration requires a range of capabilities including Extract, Transform and Load (ETL) working with complementary capabilities for data sharing via XML messages using an Enterprise System Bus (ESB) and data federation (or virtual data integration) utilising Enterprise Information Integration (EII).
A true Data Integration solution needs to include all these different capabilities to provide a complete platform to support all of the organisation’s Data Integration business initiatives e.g. Data Warehousing, Master Data Management, Enterprise Application Integration (EAI), SOA and Data Migration.
Companies who are successfully managing their data and achieving efficiency and effective business results are recognising that to succeed in Data Integration no single capability will address all business Data Integration initiatives. For example, an Enterprise System Bus (ESB) is not the right capability for bulk movement of data and complex transformations; this must be performed using an Extract, Transform and Load (ETL) capability.
Many Data Integration projects have failed due to an incorrect selection of the right Data Integration capability. With a Data Integration Platform in place, with the right mix of capabilities, this should no longer be an excuse for a failed project.
Next time we will look at trends in Social Data.
