2015 in review

The WordPress.com stats helper monkeys prepared a 2015 annual report for this blog.

Here’s an excerpt:

A New York City subway train holds 1,200 people. This blog was viewed about 4,700 times in 2015. If it were a NYC subway train, it would take about 4 trips to carry that many people.

Click here to see the complete report.

Posted in Uncategorized | Leave a comment

How to Big Data Enable your Organisation [Getting Big Impact from Big Data]

New technology tools are making adoption by the front line much easier, and that’s accelerating the organizational adaptation needed to produce results.

January 2015 | byDavid Court

The world has become excited about big data and advanced analytics not just because the data are big but also because the potential for impact is big. Our colleagues at the McKinsey Global Institute (MGI) caught many people’s attention several years ago when they estimated that retailers exploiting data analytics at scale across their organizations could increase their operating margins by more than 60 percent and that the US healthcare sector could reduce costs by 8 percent through data-analytics efficiency and quality improvements.1

Unfortunately, achieving the level of impact MGI foresaw has proved difficult. True, there are successful examples of companies such as Amazon and Google, where data analytics is a foundation of the enterprise.2 But for most legacy companies, data-analytics success has been limited to a few tests or to narrow slices of the business. Very few have achieved what we would call “big impact through big data,” or impact at scale. For example, we recently assembled a group of analytics leaders from major companies that are quite committed to realizing the potential of big data and advanced analytics. When we asked them what degree of revenue or cost improvement they had achieved through the use of these techniques, three-quarters said it was less than 1 percent.

In previous articles, we’ve shown how capturing the potential of data analytics requires the building blocks of any good strategic transformation: it starts with a plan, demands the creation of new senior-management capacity to really focus on data, and, perhaps most important, addresses the cultural and skill-building challenges needed for the front line (not just the analytics team) to embrace the change.3

Here, we want to focus on what to do when you’re in the midst of that transformation and facing the inevitable challenges to realizing large-scale benefits (exhibit). For example, management teams frequently don’t see enough immediate financial impact to justify additional investments. Frontline managers lack understanding and confidence in the analytics and hesitate to employ it. Existing organizational processes are unable to accommodate advancements in analytics and automation, often because protocols for decision making require multiple levels of approval.

How to accelerate your data-analysis transformation

How to accelerate your big data analytics transformation

If you see your organization struggling with these impediments to scaling data-analytics efforts, the first step is to make sure you are doing enough to adopt some of the new tools that are emerging to help deal with such challenges. These tools deliver fast results, build the confidence of the front line, and automate the delivery of analytic insights to it in usable formats.

But the tools alone are insufficient. Organizational adaptation is also needed to overcome fear and catalyze change. Management teams need to shift priorities from small-scale exercises to focusing on critical business areas and driving the use of analytics across the organization. And at times, jobs need to be redesigned to embrace advancements in digitization and automation. An organization that quickly adopts new tools and adapts itself to capture their potential is more likely to achieve large-scale benefits from its data-analytics efforts.

Why data-analytics efforts bog down before they get big

As recently as two or three years ago, the key challenges for data-analytics leaders were getting their senior teams to understand its potential, finding enough talent to build models, and creating the right data fabric to tie together the often disparate databases inside and outside the enterprise. But as these professionals have pushed for scale, new challenges have emerged.

First, many senior managers are reluctant to double down on their investments in analytics—investments required for scale, because early efforts have not yielded a significant return. In many cases, they were focused on more open-ended efforts to gain novel insights from big data. These efforts were fueled by analytics vendors and data scientists who were eager to take data and run all types of analyses in the hope of finding diamonds. Many executives heard the claim “just give us your data, and we will find new patterns and insights to drive your business.”

These open-ended exercises often yielded novel insights, without achieving large-scale results. For example, an executive at one automaker recently invested in an initiative to understand how social media could be used to improve production planning and forecasting. While the analysis surfaced interesting details on customer preferences, it didn’t provide much guidance on how to improve the company’s forecasting approach. Executives can often point to examples such as this one where early efforts to understand interesting patterns were not actionable or able to influence business results in a meaningful way. The upshot: senior management often is hesitant about financing the investments required for scale, such as analytics centers of excellence, tools, and training.

Second, frontline managers and business users frequently lack confidence that analytics will improve their decision making. One of the common complaints from this audience is that the tools are too much like black boxes; managers simply don’t understand the analytics or the recommendations it suggests. Frontline mangers and business users understandably fall back on their historic rules of thumb when they don’t trust the analytics, particularly if their analytics-based tools are not easy to use or are not embedded into established workflows and processes. For example, at a sales call center, staff members failed to use a product-recommendation engine because they didn’t know how the tool formulated the recommendations and because it was not user friendly. Once the tool was updated to explain why the recommendations were being made and the interface was improved, adoption increased dramatically.

Finally, a company’s core processes can also be a barrier to capturing the potential of sophisticated analytics. For the “born through analytics” companies, like Amazon and Facebook, processes such as pricing, ad serving, and supply-chain management have been built around a foundation of automated analytics. These organizations also have built big data processing systems that support automation and developed recruiting approaches that attract analytics talent.

But in more established organizations, management-approval processes have not kept up with the advancements in data analytics. For example, it’s great to have real-time data and automated pricing engines, but if management processes are designed to set prices on a weekly basis, the organization won’t be able to realize the full impact of these new technologies. Moreover, organizations that fail to leverage such enhancements risk falling behind.

Adopting new technologies to scale impact

Few areas are experiencing more innovation and investment than big data and analytics. New tools and improved approaches across the data-analytics ecosystem are offering ways to deal with the challenge of achieving scale. From our vantage point, three hold particular promise.

First is the emergence of targeted solutions from analytics-based software and service providers that are helping their clients achieve a more direct, and at times faster, impact on the bottom line. An emerging class of analytics specialists builds models targeted to specific use cases. These models have a clear business focus and can be implemented swiftly. We are seeing them successfully applied in a wide range of areas: logistics, risk management, pricing, and personnel management, to name just a few. Because these more specific solutions have been applied across dozens of companies, they can be deployed more readily. Collectively, such targeted applications will help raise management’s confidence in investing to gain scale. There’s still a need for a shift in culture and for a heavy emphasis on adoption, but the more focused tools represent a big step forward.

Second, new self-service tools are building business users’ confidence in analytics. One hot term gaining traction in the analytics world is “democratization.” Getting analytics out of the exclusive hands of the statistics gurus, and into the hands of a broad base of frontline users, is seen as a key building block for scale. Without needing to know a single line of coding, frontline users of new technology tools can link data from multiple sources (including external ones) and apply predictive analytics. Visualization tools, meanwhile, are putting business users in control of the analytics tools by making it easy to slice and dice data, define the data exploration needed to address the business issues, and support decision making. Companies such as American Express, Procter & Gamble, and Walmart have made major investments in these types of tools to democratize the use of analytics.

Hands-on experience (guided by experts in early go-rounds) helps people grow accustomed to using data. That builds confidence and, over time, can increase the scale and scope of data-informed problem solving and decision support. A technology-hardware company, for example, deployed a set of self-service analytics and visualization tools to improve the decisions of its sales force. The new platform helped the company to conduct customer analytics and to better identify sales and renewal opportunities. Since implementing the tools, the tech company has generated more than $100 million in new revenue from support and service contracts.

Finally, it’s becoming much easier to automate processes and decision making. Technology improvements are allowing a much broader capture of real-time data (for example, through sensors) while facilitating real-time, large-scale data processing and analysis. These advances are opening new pathways to automation and machine learning that were previously available only to leading technology firms. For example, one insurer has made major strides using analytics to predict the severity of claims. Automated systems instantly compare a filing with millions of claims records, cutting down the need for human intervention. Another analytics program can vastly automate search-engine optimization by predicting the type of content that will optimize engagement for a given company and automatically serving up content to capture customers.

Beyond new tools: Adapting the organization

The challenges we outlined above demand some new actions beyond the tools: more focus, more job redefinition, and more cultural change.

Focus on change management

Democratization and the power of new tools can help overcome frontline doubts and unfamiliarity with analytics. However, in addition to gaining confidence, managers need to change their way of making decisions to take advantage of analytics. This is the heart of the change-management challenge—it is not easy, and it takes time. The implication is that to achieve scale, paradoxically, you need to focus. Trying to orchestrate change in all of a company’s daily decision-making and operating approaches is too overwhelming to be practical. In our experience, though, it’s possible to drive adoption and behavioral change across the full enterprise in focused areas such as pricing, inventory allocation, or credit management. Better to pursue scale that’s achievable than to overreach and be disappointed or to scatter pilots all over the organization. (One-off pilots often appeal to early adopters but fail to cross the chasm and reach wider adoption or to build momentum for company-wide change.)

Leaders should ask themselves which functions or departments would benefit most from analytics and deploy a combination of new targeted solutions, visualization tools, and change management and training in those few areas. One telecommunications company, for example, focused on applying analytics to improve customer-churn management, which held the potential for a big bottom-line impact. That required the company to partner with a leading data-storage and analytics player to identify (in near real time) customers who would churn. Once the models were developed, a frontline transformation effort was launched to drive adoption of the tools. Moreover, customer-service workflows were redesigned, user-friendly frontline apps were deployed, and customer-service agents received training for all of the new tools.

Redesign jobs

Automating part of the jobs of employees means making a permanent change in their roles and responsibilities. If you automate pricing, for instance, it is hard to hold the affected manager solely responsible for the profit and loss of the business going forward, since a key part of the profit formula is now made by a machine. As managerial responsibilities evolve or are eliminated altogether, organizations will have to adapt by redefining roles to best leverage and support the ongoing development of these technologies. At the insurance company above, claims managers no longer process all claims; instead, they focus on the exceptional ones, with the highest level of complexity or the most severe property damage. Again, focus is required, since job redesign is time consuming. And it can be taken on only if the automated tools and new roles have been developed and tested to meet whatever surprises our volatile world throws at them.

Build a foundation of analytics in your culture

People have been talking about data-driven cultures for a long time, but what it takes to create one is changing as a result of the new tools available. Companies have a wider set of options to spur analytics engagement among critical employees. A leading financial-services firm, for example, began by developing competitions that rewarded and recognized those teams that could generate powerful insights through analytics. Second, it established training boot camps where end users would learn how to use self-service tools. Third, it created a community of power users to support end-users in their analyses and to validate findings. Finally, the company established a communications program to share the excitement through analytics meet-ups, leadership communications, and newsletters (which were critical to maintaining long-term support for the program). Creative adaptations like these will help companies to move beyond the hope that “we are going to be a big data company” and to root cultural change in realistic action.

New technologies, with their ease of adoption, point toward the next horizon of data analytics. For a glimpse of what the future might hold, consider what’s happening now at a leading organization that has adopted an innovative approach to embedding analytics capabilities within its businesses.

The company started with early-stage centers of excellence and a small corps of analytics specialists tackling business cases in bespoke fashion. Today, it rotates business leaders into a new type of analytics center, where they learn the basics about new tools and how to apply them. Then they bring these insights back to their respective business. They don’t become analytics specialists or data scientists by any means, but they emerge capable of taking analytics beyond experiments and applying it to the real business problems and opportunities they encounter daily.

We foresee the day when many companies will be running tens or even hundreds of managers through centers like these. That will accelerate adoption—particularly as analytics tools become ever more frontline friendly—and create the big impact that big data has promised.

About the author

David Court is a director in McKinsey’s Dallas office.

The author would like to acknowledge the contributions of Mohammed Aaser, Matt Ariker, Brad Brown, and Stephanie Coyles.

Posted in big data | Tagged | Leave a comment

Why Build a Data Warehouse? [Revisited]

Speaking with our clients, I’m often reminded that C-Level and Senior Managers still need to understand why you’d need to build a Data Warehouse, and ultimately what the organization will gain from money invested into it.

I had a dig back over my collected articles/links and re-discovered a couple of good slideshare.net presentation which present a good case for building Data Warehouses.

I also found it worthwhile to revisit what Ralph Kimball had to say on the subject;

“The goal of a data warehouse and business intelligence (DW/BI) solution is to publish the “right” data and make it easily accessible to decision-makers. Successful DW/BI implementations rest on the foundation of a dimensional model to deliver both ease-of-use and query performance.”

http://www.kimballgroup.com/data-warehouse-business-intelligence-consulting/dimensional-modeling/

I’d be interested to hear about your ideas/success stories of what works with the C-Level when talking about the need for a Data Warehouse.

Posted in Business Intelligence, Data Warehousing, ROI | Tagged , | Leave a comment

The Data-Driven Imperative: A Shift To Digitized Decisioning

Myth: Traditional reporting and analytics provide a ‘good enough’ perspective for forecasting success of Consumer Goods companies.

Reality: Forward-looking analytics that incorporate big data are a must-have for decision makers angling for a strong competitive edge.

Leading Consumer Goods companies approach today’s competitive landscape differently. Most rely on traditional business strategies for staying in the game, while those that rise to the top are embracing a new pillar for optimizing business decisioning: data.

This fundamental component of their success yields powerful competitive insights that drive decision-making and operational processes—these companies have become data-driven.

crThe expanding data landscape

Leveraging data and insights for competitive advantage is not new. What is changing, however, is the vast number of new, growing and diverse data types emerging as demands on businesses exceed existing analytic capabilities.

The good news is that more data is available than ever before, and business leaders know that nontraditional and varied data sources can be used to derive critical business insights that can put them in the forefront of their marketplace. The challenge, however, is that—despite being data-rich—many decision makers are still insight-poor.

Too often, executives struggle to extract value from data already under their control. And businesses are overwhelmed by the continually expanding data volumes from diverse sources, such as social networks, connected devices, Web interactions, and sensors.

Staying ahead of the “new” consumer

Maturing technology, Internet connectivity, social channels and mobility are connecting people and businesses while creating a sea of digital data. As a result, marketplace power has shifted from sellers to empowered consumers who expect cheaper products and faster services, and more personalized customer experiences.

So, what does a business leader do with this new consumer-lead, data-driven business imperative? First, they adapt their culture and behavior to leverage different types of consumer, retailer, and operational data. Data-driven businesses are more experimental in their data analysis, and have a greater tolerance for iteration and “fail fast” experimentation.

Second, they regard data as a unique and valuable asset, one that cannot be replicated by competitors. They see data as lifeblood that flows into and throughout the whole organization. To data-driven companies, data is an essential ingredient—in the same way that internal processes, machinery, and employees are core business components.

Third, they use analytics and integrated data to drive strategy and decision-making toward a tactical business plan and, ultimately, strategic success. This integrated data view can be across internal and external data sources, as long as they are relevant to company goals. In the Consumer Goods space, this includes retailer point-of-sale and loyalty card data, operational shipment data, social and mobile data, and consumer data. Data-driven businesses infuse analytics into their decision-making—from top to bottom—leveraging its value via integrated data.

Finally, data-driven businesses make fact-based decisions, rather than relying on gut or instinct. Most have built analytics and insights into the leadership level of the organization, often establishing a Chief Analytics Officer or EVP/SVP of Analytics and Insights to demonstrate focus in this space.

Measuring ROI in a data-driven business

As Consumer Goods organizations become increasingly digitized, it can be difficult to measure the benefits of shifting to a data-driven business. In fact, determining how data contributes to performance and the bottom line is a relatively new field; however, research reveals the following:

Data-driven businesses:

  • Outperform their industry peers by up to 6 percent
  • Generate 9 percent more revenue through their employees and physical assets
  • Enjoy a market value 12 percent above average
  • Are as much as 26 percent more profitable than competitors

Becoming a data-driven business requires top-down leadership and decisioning on the idea that data is an asset and analytics are an important investment for the organization. It also requires a cultural change, and an evolution of technologies that enable the business to fuel an agile decision-making process throughout the organization.

The data-driven business journey is one that leading Consumer Goods companies are pursuing rapidly, while laggards are distracted by historical views of their performance to drive the business.

Author: Justin Honaman , Teradata

Source: http://www.forbes.com/sites/teradata/2015/01/22/the-data-driven-imperative-a-shift-to-digitized-decisioning/

Posted in analytics, Business Intelligence | Tagged , | Leave a comment

The Analytics of Things

While it’s undeniably useful to connect inanimate objects and sensors to the Internet, that’s only a first step in terms of doing something useful with all those connected devices.

The phrase “Internet of Things” (IoT) suggests that the most important attribute of distributed sensors is connectedness. While it’s undeniably useful to connect inanimate objects and sensors to the Internet, that’s only a first step in terms of doing something useful with all those connected devices. “The Analytics of Things” (AoT) are just as important, if not more so.

The AoT term points out that IoT devices generate a lot of data, and that data must be analyzed to be useful. It also suggests that analytics are necessary to make connected devices smart and for them to take intelligent action. Connection, on the other hand, isn’t required for intelligent action. There are many different types of IoT analytics, and connection isn’t required for all of them.

Think, for example, about a “smart” thermostat, now available from a variety of vendors. These thermostats sense not only room temperature, but also whether people are in a room, their patterns of activity during the day, and so forth. In order to make sense of such data and take action on it, smart thermostats have embedded analytics that help them decide when to turn themselves up or down. So they’re smart enough—even without being connected—to save energy with little or no user involvement.

Smart thermostats can also be connected to the Internet through wifi, and there are some potential benefits from doing so. Remote monitoring and control is one. I can turn up my thermostat during my trip home from work, or check remotely to make sure my pipes won’t freeze.

This is useful for controlling remote devices, but connection also yields more data and more potential for analytics. The primary virtue of connected analytics is that you can aggregate data from multiple devices and make comparisons across time and users that can lead to better decisions. Comparative usage of an important resource such as energy, then, is one key analytical approach to connected data.

What other types of analytics of things are there?

  • Understanding patterns and reasons for variation—developing statistical models that explain variation
  • Anomaly detection—identifying situations that are outside of identified boundary conditions, such as a temperature that is too high or an image depicting someone in an area that should be uninhabited
  • Predictive asset maintenance—using sensor data to detect potential problems in machinery before they actually occur
  • Optimization—using sensor data and analysis to optimize a process, as when a lumber mill optimizes the automated cutting of a log, or a poultry processor automates the preparation of a chicken
  • Prescription—employing sensor and other types of data to tell front-line workers what to do, as when weather and soil sensing is used for “prescriptive planting” by farmers
  • Situational awareness—piecing together seemingly disconnected events and putting together an explanation, as when a series of oil temperature readings in a car, combined with dropping fuel efficiency, may indicate that an oil change is necessary

This partial list of AoT possibilities begins to suggest their elements in common. One is that they are often a precursor to informed action. Comparative usage statistics, for example, might motivate an energy consumer to cut back on usage. Predictive asset maintenance suggests the best time to service machinery, which is usually much more efficient than servicing at predetermined intervals. A municipal government could analyze traffic data sensors in roads and other sources to determine where to add lanes and how to optimize stoplight timing and other drivers of traffic flow.

Another common element in the AoT is the integrated display of information—pulling together IoT information into one place so that it can be monitored and compared. In Singapore, for example, the Land Transport Authority has put many of the functions involved in IoT central information gathering and analytics into place. Its data sources include road sensors, traffic light monitoring, espressway traffic monitoring, intersection monitoring, GPS devices monitoring traffic and road conditions on 9500 taxis, parking place availability sensors, and crowdsourced public data. At one point all of these sources were independent, but Singapore has now created an integrated “spinal cord” for them called the “i-Transport Platform.”

Thus far, the primary use of the unified data is to push it out to drivers through electronic signboards (and to companies to create innovative applications), but one could easily imagine the use of automated analytics to change traffic light patterns, increase tolls for vehicles going into the city, or to recommend optimal routes. One could suggest that automated action is another possible element of the AoT, though it doesn’t always take place.

Who needs to bulk up on AoT? Organizations should start building their sensor data analytics capabilities if they have relatively little experience with sensor analytics or little exposure to fast-moving big data. If they see a lot of data coming and no clear way to make sense of it, the analytics of things is going to be important to their future. For example, if your organization (like the US military and intelligence sectors, for example) is using drones to capture a great deal of video, you may want to rapidly focus on capabilities to analyze video data and detect anomalies with little human intervention. The Secretary of the US Air Force lamented in 2012 that it would be years before humans could possibly analyze all the video footage captured by drones in war zones.

We’ll learn much more about the AoT as these connected devices proliferate. We’ll learn how to extract the data from them for analysis, and where best to locate the analytics. We’ll learn what kinds of analytical features and functions are most helpful. For now, it’s just useful to remember that the Internet of Things is only useful if those things are smart, and that will happen through the Analytics of Things.

Review Deloitte Analytics Trends report, to explore trends to watch in 2015, including innovations in security, cognitive analytics, data monetization, and more.http://deloi.tt/1BIo8YQ

About the author:
Tom Davenport, a world-renowned thought leader and author, is the President’s Distinguished Professor of Information Technology and Management at Babson College, a Fellow of the MIT Center for Digital Business, and an independent senior advisor to Deloitte Analytics.

Posted in big data, Internet of Things | Tagged , , , , | Leave a comment

Israel’s Water Ninja

January 08, 2015

Israel's Water Ninja
Photographer: Roei Greenberg for Bloomberg Businessweek

Amir Peleg hunches his broad, 6-foot-3-inch frame into a tunnel leading to one of several reservoirs that supply water to Jerusalem. Condensation collects on the ceiling, inches overhead, like thousands of tiny stalactites. Peleg, an entrepreneur whose self-given job title is “chief plumbing officer,” catches a droplet on his palm. “Literally every drop counts,” he says. “This is the modern-day Gihon.”

Gihon was the ancient, intermittent spring that made human settlement possible in Jerusalem circa 700 B.C. Today, fresh water sources in Israel and the surrounding region are more precious than they were in the Bronze Age. About 1 million residents continually draw water from this reservoir, which is filled by pipelines snaking from the Sea of Galilee 90 miles north. Located at the edge of Jerusalem, the reservoir is held in a massive underground vault patrolled by armed guards to keep insurgents from poisoning the supply. Thick cement walls surround a floodlit pool of water, ghostly and luminous, 40 feet deep and wider and longer than two football fields.

Like most of its neighbors, Israel is a desert nation, and during the past seven years it’s struggled through a drought with record-low rainfall. In response, Peleg and others have come up with an array of innovations, from microscopic sewage scrubbers to supersize desalination plants to smart water networks. Israel now has higher agricultural yields than it’s had in nondrought years. It even has a water surplus, a portion of which (about 150 million cubic meters per year) it pumps to Jordan and the Palestinian Authority.

“I don’t think it’s overkill to say that Israeli entrepreneurs are disrupting and reinventing how the world creates and conserves water,” says Peleg, 48. He’s become one of the leaders of a water-tech movement that began in the 1950s, when Israel’s first prime minister, David Ben-Gurion, implored scientists and engineers to “make the desert bloom.”

In 2008, Peleg’s startup, TaKaDu, began designing software that uses mathematical algorithms to detect and prevent leaks in water pipelines. Peleg has silver, buzz-cut hair, arching black eyebrows, and a jaw like an anvil—George Clooney’s indomitable Danny Ocean meets the affable Schneider from One Day at a Time. He’s part swaggering CEO and part scrappy superintendent.

Detecting leaks may seem like a small concern, but it matters, especially in environments where water is scarce and expensive. Of Israel’s total water demand (2.2 billion cubic meters a year), less than one-tenth is supplied by freshwater sources such as the Sea of Galilee. The rest comes from filtered gray water—Israel recycles more than 85 percent of its wastewater—and from desalination, an expensive process that transforms saltwater into drinking water. “Among all conservation technologies in development, the most valuable detect leakage in networks,” says Avshalom Felber, chief executive officer of IDE Technologies, Israel’s biggest desalination company. On average, utilities worldwide lose more than 30 percent of the water they distribute in their networks. By comparison, Jerusalem’s utility—Hagihon, Peleg’s first customer in 2009—wastes less than 10 percent of its supply, thanks in large part to TaKaDu.

Over the past five years, venture capital firms and companies including 3M (MMM) and ABB (ABB) have invested more than $20 million in TaKaDu, and its software has been adopted by 14 other utilities in cities from Campo Grande, Brazil, to Bilbao, Spain. Last month, Peleg signed with Australia’s biggest utility, Sydney Water. Collectively, these utilities manage about 40,000 miles of water pipelines. Peleg won’t disclose TaKaDu’s revenue but says it’s grown more than 50 percent annually over the past two years.

TaKaDu’s software is designed, as Peleg describes it, “to slice and dice and analyze raw data measured by smart sensors in the water network.” These sensors monitor the flow rates, pressure, and quality of the water and identify bugs in the meters, valves, and other system equipment. From this data, the software can analyze when and where water is escaping.

“Until TaKaDu came along, the water-utility world was almost deaf and blind,” says Zohar Yinon, CEO of Hagihon. “Our network is not transparent without this software. It’s like an EKG or an X-ray, exposing the inner workings of our system on a real-time basis. We are no longer plumbers and water engineers; we’ve entered the world of preventive medicine.”

But can Peleg shift the world’s lowest-tech industry to big data solutions? “The best and worst thing Peleg has going for him is that he’s ahead of the game,” says Aaron Mankovski, who runs Pitango Venture Capital, Israel’s largest VC firm, and has yet to invest in TaKaDu. “Water is a cautious, nearsighted industry,” he adds. “But I have no doubt that eventually all utilities will go this direction. They will have to, to survive.”

TaKaDu’s headquarters are above a Pizza Hut and a pastry shop, in a glass-and-granite office building in a quiet suburb of Tel Aviv. Inside, the offices have the obligatory signifiers of the tech startup: minimalist couches and bean bag chairs, walls painted primary colors, and an open kitchen with a large picnic table for meetings and meals. The walls bear poster-size photographs taken by TaKaDu employees depicting water in some form, from dewy fields to foaming falls.

“Until TaKaDu came along, the water-utility world was almost deaf and blind”
Peleg’s office is cozy and modest. There’s little here to indicate he’s a veteran of three startups. He sold his last one, YaData, to Microsoft (MSFT), less than two years after he founded it. (A leading Israeli newspaper reported a rumored sale price of about $30 million; Peleg says he’s obligated by contract not to disclose the sum.) YaData was another algorithmic venture; its software helped online advertisers to more accurately target customers.

Peleg was influenced by his grandfather, who built Tel Aviv’s first luxury hotel. At 13, Peleg hacked the first Apple (AAPL) computers that came to the market in Tel Aviv and created a version with Hebrew characters that he sold to local businesses. At 17 he was accepted to Talpiot, the Israel Defense Forces’ most elite technology unit. Over eight years he learned to develop military drone operating systems and software to automatically identify key visual information in satellite images, such as tanks, missiles, and other targets.

Peleg then joined Elbit Vision Systems, a company that develops software for large-scale textile production. The software analyzes visual data to identify flaws in fabrics. TaKaDu is doing something similar today with pipe flows and pressures. “It all boils down to finding new ways to understand aberrations in data,” he says.

Peleg got the idea for TaKaDu at a technology conference in Vienna in September 2008, when he met a water engineer specializing in the Scada (supervisory control and data acquisition) systems that collect data from pipe-embedded smart sensors. The sensors use mechanical methods, such as rotating wheels, as well as ultrasonics to measure network flow and pressure and can transmit hundreds of data points every 15 minutes. Peleg wasn’t interested in the hardware, just the information generated. “I asked the Scada guy what he does with the data. He says, ‘We store it.’ I thought, ‘This is it! I’m going to mine this dormant data for golden nuggets.’ ”

Within a few months of the Vienna conference, Peleg had hired five programmers and was running TaKaDu out of his living room. A number of Peleg’s early recruits were from the Talpiot program. “I said, ‘Now our enemies are not people, but the leaky pipes underground,’ ” Peleg recalls. Instead of using algorithms to scan images, he was now building software that had larger implications for Israeli security and that might even help wasteful and drought-afflicted nations worldwide. His wife, Naama, wrote payroll checks from their family checking account: “Finally Amir was doing something real,” she recalls.

Hagihon CEO Yinon is munching cookies in a bunker-like basement that once functioned as the utility’s control room. “We no longer have a physical control room—TaKaDu has put it right here,” says Yinon, wagging his iPhone 6. “I can find out anywhere if my meters are accurate, my water quality is clean, my pressure is good, my flow is normal, my pumps are working properly, my infrastructure is humming … all these layers are integrated online.”

Six years after founding TaKaDu, Peleg has 35 employees and offers utilities a cloud-enabled service that presents the full gamut of information about the network’s operations. Peleg is essentially trying to do for water networks what Thomas Siebel did for customer relations management (CRM) in the early 1990s: rethink the interaction between organizations and customers and integrate all the layers of information a company has about a customer into a single interface.

TaKaDu’s software establishes a baseline of “normal behavior” within each network. The better it understands normal patterns of water flow throughout the day, the more accurately it can detect aberrations that indicate a leak or burst. It knows that water flows are highest in mornings and evenings, before and after people are at work. It also considers local factors: At a Netherlands utility, for example, the system detected spikes of flow at regular intervals one Friday afternoon; it noticed that these patterns corresponded with breaks in play during a World Cup game between the Netherlands and Spain, when fans were flushing toilets.

The software can also detect water theft. At Unitywater, a utility in Melbourne, the system noticed abnormally large flows coming from a fire hydrant; officials were notified, and they found a strawberry farmer siphoning water from the hydrant.

Within a year of adopting the TaKaDu system, Unitywater saved more than 1 billion liters of water. That translated into savings of more than $2 million. The utility also reduced the time it takes to repair problems in its network by more than 60 percent.

TaKaDu is not a standalone solution for utilities. “We’ve found that TaKaDu works best when connected with other systems,” says Yinon. Aquarius Spectrum, a company run by Israeli Zeev Efrat, uses advanced sound equipment to detect the exact location of leaks. Yinon uses TaKaDu software to identify the location of a leak within a neighborhood and then Aquarius’s technology to find the pipe it’s coming from. Another Israeli startup, Curapipe System, offers an automated leak repair system that plugs ruptures without digging.

When Peleg notes that TaKaDu has no direct competitors, it’s more a lament than a boast. “There are companies doing many different aspects of what we do, but none yet that encompasses all.” The French water and waste management company, Suez Environnement (SEV:FP), recently introduced a similar smart-network service called Aquadvanced, but it’s too new to the market to have made an impact. Peleg insists that he wants competitors to come in and “wake up the market, so utilities get more familiar with the future we’re all headed toward.”

As it is, only about 20 percent of utilities worldwide—and fewer still in the U.S.—are using smart sensors in their infrastructure. “Not everybody can see it yet,” says Zvi Arom, a member of TaKaDu’s board. “There are those utilities you meet and you tell them what TaKaDu can do, and they say, ‘I may as well believe in Snow White and Santa Claus.’ ”

Peleg lives with Naama and their three young kids in an agricultural village about halfway between Tel Aviv and Jerusalem. Their house is a modern expanse of glass and steel surrounded by eight acres of heavily irrigated farmland that Peleg calls his Eden. He has olive, pomegranate, avocado, lemon, fig, mango, and pecan trees, a vegetable and herb garden, and a small vineyard with merlot and chardonnay grapes.

The family pays the equivalent of thousands of dollars a year for water. Peleg, an avid cook who pickles his own cucumbers and brines his own olives, takes a certain pride in his water bill: “Americans think water should be free and unlimited, like air. But the philosophy in Israel is, if you want to have a garden or a pool, fine—pay for it!” Israel has a three-tiered pricing system, he says: “We’re only allowed to consume a certain amount of low-cost water for a family of five, for example. Above that quota the water is 50 percent more expensive. On the next tier, the pricing goes wild.”

About 20 percent of utilities worldwide use smart sensors
The pricing structures of many U.S. water utilities, Peleg says, encourage rampant consumption: “A third of the counties in America still charge a flat rate for water, whether you are a business or resident, you pay a flat rate. Like for $9.99, it’s all-you-can-eat water.”

Last July, as California suffered through a crippling drought that would claim 200,000 acres of crops, a pipe broke under Sunset Boulevard in Los Angeles and released 20 million gallons of water. “Our software could have prevented such a burst,” Peleg told a panel at a recent conference in Tel Aviv. “It would have picked up the problem when it was just a small leak.” This year, Peleg will introduce a cloud-based service for U.S. utilities that monitors water quality, which is tightly regulated by the U.S. Environmental Protection Agency.

Peleg and his peers are betting that as hardware costs decline, data tools improve, droughts become more common, and water pipes get older, the U.S. will become a more lucrative market for their products. “It is becoming a much lower-risk investment,” says Pitango Venture Capital’s Mankovski. “The biggest challenge for a water startup is to get the initial 10 to 15 customers and proof of concept. Peleg has done this and has ushered those relationships into long-term contracts. He has a real chance of providing the de facto tool for municipalities worldwide.”

Posted in analytics, big data, Case Study:Analytics, Internet of Things | Tagged , , , , | Leave a comment

So you wanna be a data scientist? A guide to 2015’s hottest profession

Unnamed1
Are you good at math? Like, really good at math? Do you also know Python and, oh yeah, have deep knowledge of a particular industry?On the off chance that you possess this agglomeration of skills, you might have what it takes to be a data scientist. If so, these are good times. LinkedIn just voted “statistical analysis and data mining” the top skill that got people hired in 2014.Glassdoor reports that the average salary for a data scientist is $118,709 versus $64,537 for a programmer. A McKinsey study predicts that by 2018, the U.S. could face a shortage of 140,000 to 190,000 “people with deep analytic skills” as well as 1.5 million “managers and analysts with the know-how to use the analysis of big data to make effective decisions.”

The field is so hot right now that Roy Lowrance, the managing director of New York University’s new Center for Data Science program says he thinks it has peaked. “It’s probably in a bubble,” he says. “Anything that gets hot like this can only cool off.” Still, NYU is looking to expand its data science program from 40 students to 60 over the next few years. The current school year won’t be over for another five months and 50% to 75% of its students already have firm job offers.

Why the explosion? Linda Burtch, managing director of Burtch Works, a Chicago-based executive recruiting firm, notes that while tech firms like Google, Amazon, Netflix and Uber have data science groups, the use of such professions is now starting to filter down to non-tech companies like Neiman Marcus, Walmart, Clorox and Gap. “All these are companies looking to hire data scientists,” she says.

The hope is that such professional will unearth new information that will prompt new streams or revenue or let a company streamline its business. Pratt & Whitney, the aerospace manufacturer, now can predict with 97% accuracy when an aircraft engine will need to have maintenance, conceivably helping it run its operations much more efficiently, says Anjul Bhambhri, VP of Big Data at IBM.

Though IBM just released its freemium, cloud-based Watson Analytics program this month, most often data scientists have to create homegrown software programs to analyze unstructured data, which is one reason that programming skills are required.

Schooling

Lowrance says there are basically three skills that a data scientist needs to possess: math/statistics, computer literacy and knowledge of a particular business domain (like autos, for example.) NYU’s program teaches those so that each area of expertise builds on the other. When you graduate, you’re sort of a jack-of-all-trades for data crunching. “When working on data science projects in coursework they have to do all the jobs,” he says.

Not everyone has to go through a college course to become a data scientist, though. A company called Metis, for instance, started offering a 12-week data science boot camp in September. The program, in New York, costs $14,000 and admission is highly competitive. Metis Cofounder Jason Moss says that about half the students come in with a Master’s or PhD.

Just a couple of weeks after the first boot camp ended in early December, Moss said six of the class’s 15 students had job offers.

“I don’t think it’s a replacement for college,” Moss says of his program. “I think college is about more than the fastest path to getting a job. I also don’t believe that you have to have gone to college to be successful as a data scientist,” he says. “There’s a personality type – innately curious, has grit, wants to figure things out — that does well.”

Anmol Rajpurohit, an independent data scientist and consultant, says being a fast learner is most important attribute for this line of work. “Generic programming skills are a lot more important than being the expert of any particular programming language,” he says. “Living in an age of rapid technology advancement, we see languages quickly becoming obsolete and new languages quickly getting popular. Thus, a fast learner will go a lot farther than an expert.”

Lowrance says that he believes boot camps and online-based courses can be helpful for candidates strong in some skills, but weak on others. One virtue of NYU’s program is that it teaches the skills sequentially so that they build on each other. “We give you everything you need in an order that makes sense,” he says.

Jon

Jon Greenberg

 

IMAGE: JON GREENBERG

What data scientists do

“On an average day, I manage a series of dashboards that tell our company about our business — what the users are doing,” says Jon Greenberg, a data scientist at Playstudios, a gaming firm. Greenberg is a manager now, so he’s programming less than he used to, but he still does his fair share. Usually, he pulls data out of Apache Hadoop storage and runs it through Revolution R, an analytics platform and comes up with some kind of visualization. “It may be how one segment of the population is interacting with a new feature,” he explains.

Greenberg got a Master’s degree in statistics six years ago. He expected to go into government work, but was surprised to see that data scientists were so in demand in the private sector. “It was definitely not as hot a field then,” he says. Now, he says he gets about one call or email a day from a headhunter. “It’s not me,” he says. “They probably bother everyone else [with this expertise].”

For Greenberg, employability is a plus, but he loves the work itself. “I think it starts with, you have to have an analytical mind. You have to be curious,” he says. “You have to be flexible and creative and think of a different way to solve problems.” The only downside of the job, Greenberg says, is the time spent “cleaning” data — pruning it to remove irrelevant findings. “That part’s not that exciting and you spend a lot of time doing it,” he says.

Rajpurohit says he spends a lot of his energy cleaning data, but also researching. “A significant part of my time is spent on research, because I often come across absolutely new problems and thus, have to study the latest literature on research in that particular field or reach out to experts on those topics for advice,” he says.

“Despite its name, data science requires a good mix of both art and science. The science part is obvious –- mathematics, programming, etc. The art part is equally important –- creativity, deep contextual understanding, etc. Both the parts put together make one a great problem solver.”

That said, Rajpurohit acknowledges that ‘working in Data Science is not even remotely as sexy or glamorous as it is being perceived these days. This field is definitely gaining significance (and seeing high pay offers) across organization, but there is a lot of not-so-exciting tasks that a data scientist needs to work on almost daily basis.”

Is this the career for you?

If the idea of spending much of your day programming and analyzing dashboards for relevant information appeals to you, then you might have the makings of a computer scientist. If you’re merely motivated by the salaries, though, you may have a tougher time. Consider: People who fall into this line of work often spend their spare time writing programs and analyzing data just to amuse themselves.

Adam Flugel, data science recruiter for Burtch Works, recalls a recent candidate, a PhD holder, who he placed at Electronic Arts this fall. “What really stood out was the work that he was doing for fun in his free time,” Flugel says. “He was involved in the online multiplayer game World of Tanks and led a “clan”, basically a team of players. He created a utility to scrape data from the game server and then ran analytics on that data to evaluate his clan’s performance. He used this info to figure out how to adjust their strategy, what types of players he should recruit to improve the team, etc.”

If you don’t love data for its own sake, then you will find it hard to compete with such candidates. Burtch, however, says everyone should learn to love data, if only for the sake of their career. “Within 10 years, if you’re not a data geek, you can forget about being in the C-suite,” Burtch says.

But what about Steve Jobs, Bill Gates and other such visionaries who saw the big picture and didn’t get bogged down in the minutiae of data science? “That was 30 years ago,” says Burtch. “I’m talking about the next 10 years.”

Have something to add to this story? Share it in the comments.

Posted in big data, Education | Tagged , | Leave a comment