CRN roundtable: the lowdown on big data

Lurking within these bulging repositories of data lie previously undreamt clues and patterns shedding new light on everything from consumer behaviour in the retail sector to the complex forces underpinning mineral exploration, disease and drug research and even how to discover, pre-empt and prosecute criminals. 

Once the sort of issue that only enterprises would have troubled to think about, the expanding possibilities, falling costs and easier deployment of big data solutions suddenly opens a new door to small businesses. At the same time the jury is far from in at the big end of town with enterprises still grappling with an abundance of choice, not to mention complexity.

But with so many different “Big Data" solutions in the market it is hard for companies to know which are the best for their purposes. After all, big data is only as powerful as the tools we use to mine it.

In this, the latest roundtable to be hosted by CRN, in partnership with Nextgen Distribution and Oracle, our expert group of executives and IT entrepreneurs reveal the essential business and technical insights to help resellers align their customers with the big data solutions that best address their needs.

In other words, to help the channel and its customers ride the wave of this phenomenal and disruptive trend, rather than being swept away in the tsunami.

Featuring:

Stuart Long, system sales director, Oracle

Scott Newman, senior pre-sales director Oracle

Chris Mendes, chief technology officer Sirca

Rob Silver, senior software engineer, Remora

Laura Brundle, NSW district manager Dataflex

Tiberio Caetano, senior researcher NICTA

Jonathan Rubinsztein, CEO Redrock Consulting

Andrew McLean, enterprise solutions manager Intel

John Walters, managing director Nextgen Distribution

Anthony Viel, partner Deloitte forensics

Peter Kazacos, managing director and chairman Anittel

CRN: Anthony, you were at the ground level with Deloitte Australia some years ago leading a global initiative with big data. It took a few years to get going, but now it sounds like one of the fastest growing businesses in the organisation.

Anthony: My business is about six years old. It started from nothing. I was the first recruit in what we call the analytics business. When I came into the organisation at Deloitte, in Australia, I was part of redefining what we offer within Forensic. Anyone who’d not come across a Forensic issue, the most public one in most recent years is Siemens’ [bribery scandal of 2007].

We used data and analytics to transform what the Forensic offering was. It wasn’t a big jump for forensic. Because you’re generally in data, you need to form the position pretty quickly to represent in a Court of Law or in front of a regulator or whatever the case may be – so a very robust, 100 percent focus.

Some things were not obvious to clients so you had to leverage some of the best analytics techniques around.

In the course of doing that we transitioned the business from five or six in the Australian marketplace to number one, where we will double number two in the course of four years. With that, we then started to say, ‘hey there’s something in this data stuff – our clients want to talk to us about this data stuff, everyone says we’ve got mountains of data but no insight --or oceans of data and only islands of insight if you will'.

We started to focus analytics on audits, the traditional backbone of a business like Deloitte’s. We started focusing analytics on internal audit risk services. We started focusing analytics on tax, and so on and so forth – so across everything that we offer in the space. We not only took it out to market, but now we’re starting to really transform our business,

CRN: So what do you think organisations need to focus on in order to develop effective big data solutions?

Anthony: Action. Everyone can build stuff around it and point to how sexy the opportunities that are presented are big data having more data available, having it faster available to you in a shorter period of time, but until you actually get the organisation to change, the justification for an investment that you need, or transformation that you need to take full advantage of big data, is not going to stack up.

Big data is at risk of becoming a dirty word, or two words, as the case may be, like ‘cloud’.

I’m proud to say for Deloitte the big data journey began in Australia.

Our CEO, who is an ex auditor, doesn’t speak like an auditor any more. I’ll take a bit of credit for that

CRN: Peter, as one of the largest and most successful resellers and more recently carrier organisations in Australia, talk us through where you see the opportunities in big data.

Peter: If you look at the three words, three product sets, you’ve got cloud, you’ve got big data and you’ve got tablets. I think they’re three of the biggest trends at the moment, but if we focus on the big data side, it’s interesting from words that have been used to describe it, how quickly people can understand - whereas cloud, ok, more people are understanding it, but they still don’t really get it in a lot of cases.

But ‘big data’, people know that they’ve got that, because it’s been cheap, people have been storing it. In terms of storage, before it was people saying ‘what should we store, because it’s going to cost us a lot of money’ --- now they are storing it – even though they’re not doing much with it, and traditionally in the past, the words around data mining was something that was really out there, really hard to do and for larger organisations.

Now it’s about people saying ‘we need to be able to understand more about what’s there’, and one of the areas I think there’s a huge opportunity is within marketing organisations that really understand that to be able to deliver targeted marketing campaigns.

But if you look at the types of ‘big data’, you’ve got stuff logged in data bases, but also more importantly is the stuff that’s held in all the social media sites around and I think you can’t really digest that information, because it isn’t structured, so you can’t just use simple tools, and you do need to have a more rich set of tools.

One of the issues I’ve got at the moment is that I see it as an opportunity. We are not a creator of products, so we need to have product that’s available to us to be able to use in that space, as well as service. We focus on two ends. We have this huge base in our SMB space. 

Based on service they don’t really want something on board, they want something to be able to deliver, some intelligent information and so I think it’s a huge opportunity in this space.

Anthony: It’s the fact that you are keeping more and more information about what it is that you’ve done or have not done, to discharge your duties and responsibilities to regulators, consumers and the like, and we have started to see the tipping point of that in the US in particular with all these large discovery actions. 

As an example, there’s one going on now around BP - obviously they had a problem in the Gulf, but the point is that the corporate, the director of the corporate has some accountabilities and responsibilities, and I haven’t been in the Forensic situation where you come into an organisation and the information hasn’t been there.

Now that we’ve put more and more obligations on things, in some spaces like in safety and in some of the sanctions on money laundering and that sort of stuff, jail terms – the flipside of the opportunity is to say ‘wait a second, if you’re not collecting it, somebody else is, because they’re more corporate responsible’ and it’s becoming more pervasive you want access to that information whether it be through social media or what have you. 

So it’s just a bigger lever to say get interested in this quickly, as the exposure that it’s creating is a real opportunity.

CRN: We are hearing a lot of talk about the exciting possibilities of ‘big data’. We are still at an early stage in terms of realising them, but Chris you seem to be someone who is at the coalface in terms of what could be discovered, particularly in terms of financial data, about what’s going on in the exchanges around the world.

Chris: Yes, there is actually a lot of stuff happening in that space. Obviously there’s a lot of things that we can’t talk about, and part of the reason that we can’t talk about it is that the people who use that data don’t want to tell us, who have got their magic algorithms - but interestingly one of the things that Anthony hinted at there was not so much from a strictly compliance perspective, but a purely competitive perspective such as people doing comparative costings on the efficiency of exchanges, based on stock market data.

It was a bit of an unexpected in this case, where it’s no longer the guy who wants to make money purely on transactions, but it’s actually having the exchanges compete with each other.

One of the other things that relates to what Peter was saying, is just the sheer use, and we’ve got a very big data set, so we’ve got all the data going back to 1996 for every exchange in the world, and that in itself is a huge problem, and obviously it is our core business, and we’ve spent a lot of time solving that.

The problem we haven’t been solving is actually processing the logs of what our customers are actually doing on that data – and recently I thought that should be easy, I’ll just go and get it done off the log and I’ll load it into the spreadsheet, because that’s about where my data processing capability begins.

And of course Excel just died a horrible death – so the logs going back for five years was a quarter of a terabyte, and that doesn’t sound like a lot of data, you’re talking 250GB – but to process that data and gain intelligence from it is a challenge. 

I think the real issue, or real surprise for our industry is the guy who comes out and can say ‘I’ve got a tool that that lets my accounts department stab at things’.

Jonathan: I think we have got the hardware and software that allows us to get insight from data. We do have the risk also that just having the ability to get insight, the insight itself needs to be relevant.  So I think working out what insight you want, and investing your limited resources is critical.

It all steps back to strategy. If you understand what your business is and where that insight can add value, because the problem is that you are hoping that there is insight and some structured data.

You’re hoping that you’ve got a whole lot of this data and so you might be able to find something cool in it, because you’ve got a tool – but guess what, you might (a) not find anything cool and (b) you might have no ability to take that insight and convert it into a return to your business.

I agree with Anthony’s comment that this not only being about revenue return, but there’s a risk also that you can manage, I think is really an insight, but I still think we do have the risk that you get technologists. I always say a ‘fool with a tool is still a fool’ and you’ve got technologists who get excited about technology, because guess what you can find out that a whole lot of people have done X, Y, Z and you go ‘that’s good’ and they’ll still do XYZ and you’ve just spent five million dollars working that out.

It’s important that we step back and work out where the insight might be. It’s like mining for minerals – guess what, you might mine in the wrong place and find nothing. So I think that where the risk we have with this Big Data, if we continue down the wrong paths, is we’ll waste a lot of money.

Tiberio: This is a very important point, and my view is that there are ways to prevent that from happening, and that has pretty much to do with trying to start from your business problem, or from your interpretation of a business problem.

Let’s include our business problems, some abstract way that the computer can understand and then let’s do the type of analysis in our ‘big data’ project that actually meets the requirements.

I’m a research scientist, I’ve been working for more than ten years on technology that tries to do that sort of stuff. Of course ‘big data’ is booming only now, but it’s been around for a while. I can tell you that particle accelerators have been there for a long time. We have been struggling with this problem for a long time. It’s just now everyone has access to ‘big data’. It’s got a cool name and it’s cheap to store.

But now we are making that link between business problems and technology. There’s a name for it, we call it ‘machine learning’.

I’ve been working on this for ten years. At NICTA what they do is to try and leverage that technology, that science, which is an important science in itself for the ‘big data’. We have realised that it’s really a key component of making the connection.

Jonathan: The question is around limited resources - getting to have an opportunity to spend in the new sales force for ‘big data’ and getting your eye on getting the ability to understand and drive it from a business perspective. So it’s not just articulating the problem, but trying to establish the return on the investment.

As people who either consult or sell, most of us think about these problems. One has to be able to step back and understand actually what you’re trying to achieve, how significant is this risk, where’s the revenue opportunity, quantify that. 

How likely am I to find that return if I do some cool analysis? Will I find that if Telstra is looking for, understanding that there might be some trends to hot spots or black spots in a mobile network, what is my business return if I spend 10 million bucks?

Anthony: With machine learning we can’t think of some of the hypotheses for instance that we should be testing for anyway. But I do agree with you about the need to link to strategy. I’ve been working in this business now for five years, and I haven’t put a case study on a table for a client that doesn’t have a significant return on doing things differently – of the order sometimes 50:1 better at a minimum 3:1. That is I could spend a third of my marketing dollars and still get the same result.

Then it comes back to us and changing the way the way we run our business, and if you can’t get people to adapt and accept that machine learning is saying to turn left instead of right, and you can’t do that right to your coalface, then it doesn’t matter how good the technology is collecting this stuff, doesn’t matter how good the analytic technique is, doesn’t matter how good the analytics is linked to the strategy that you’re trying to implement.

Then that’s why I come back to my first point, Big Data will be a dirty word before you know it.

John: An important issue in this area is that of static analysis versus real time analysis. Looking at the spreadsheet and doing the analysis of what we’ve already stored is one thing, but looking at real time ‘big data’ analysis allows us to market better or do this better or that better.

Anthony: That’s an excellent point. I think real-time is coming, the only ones that I see that are close to real-time at present would be the law enforcement agencies, some of the more advanced ones.  They’ve got lots of money and generally come from North America. 

Then there’s the credit card monitoring by the banks, but that would be less than 2 percent of what’s going on. The rest of it is looking back on static data in my experience.  Very few organisations – and I smiled when you said back to 1996 -  very few organisations are looking at stuff back three years ago, four years ago.

John: Five years is usually the minimum and most people aren’t interested beyond that.

Anthony: They’re not interested, yes I agree with you, but with some organisations you’re going to fall down. Brambles the pallet guys, they’ve got data back to 1964 available on their system, and transaction data back to ’87, and you could say that’s not real relevant, but when the GFC hit us, you go back and look for the last catastrophic event and that was around 2001, and then back to ’91 was the last time that it happened, so you can really benefit from that sort of longevity of information. But real-time, it’s coming but it’s not here yet.

CRN: Presumably one of the important technologies in terms of real time and the general advancement of big data and analytics is closeted in-memory analytics,

Tiberio: Yes that’s a big trend. Of course you can have a big cluster of computers, and to have your data distributed only in-memory. There are ways to leverage substantial amounts of data. Of course it’s hard to actually do that with extremely large amounts of data, because otherwise everybody would be doing it.

So there is a sort of a threshold there, and for different questions, different business problems. If you have everything in memory it would be worthwhile to invest in infrastructure for that. For others, you just have to live with everything and explore at least a substantial part of what you have in this.  So I think it really depends on your question. It’s not clear, it’s not something that you can say that there is the right answer for.

Andrew: It’s definitely a growth area. We expect to have two terabyte databases in memory, and certainly some of the products that we’ve [Intel] been bringing to markets make that much more possible.

IT is now front of centre in any organisation. Even if you’re looking at just the marketing of the organisation, the amount of spend that’s going from traditional above-the-line marketing into looking at social networks, and how do we really talk to our customers, and what can we learn from our customers. IT has to enable that to happen for an organisation.

And the question about the desire to do it, but also the budget to go and do these things is very real.  There are so many things that an organisation can be doing in this space, but do they have any extra money to go and do it? A lot of the time they don’t.

IT departments need to start looking at how to drive down the costs of our infrastructure that we have at the moment. How old is that infrastructure? How much is it costing us? How much more efficiency can we get in a data centre if we can have huge levels of consolidation? 

How, if we have simpler infrastructure, how much more manageability – so all the time we spend on managing our infrastructure, if we can get rid of some of those costs, it starts freeing up money and freeing up time, for IT professionals to start working on these problems that organisations have.

Organisations will live and die by their ability to respond to what their customers are saying, and the speed of response is going to be critical.

It won’t just be in low latency financial environments where that will be important. It will be your average consumers out there shopping. They are standing in a shop and they’ll get online and say ‘where else can I get this product, and what sort of prices can I get?’ and you have to be able to respond extremely quickly to customers.

I think more and more you’ll see in-memory will be very important, and certainly outside of science and outside of financial services, it will really start to grow. 

CRN: Robert, I’m curious as to what your customers are saying about big data.

Robert: Utilities is probably a space where we are actually playing with ‘big data’ in quite a substantial way. A term we quite often use is ‘data exhaust’. A lot of our clients have got a lot of machine-generated data that is not actually being captured. It’s just being blown out the window, and nobody is doing anything with it to actually gain market intelligence to try to improve their operations.

A utility organisation who has a number of power stations around Australia, obviously has market data on the sell prices of electricity, and prices go up and prices go down based on demand. Their power stations are generating a vast amount of data from all the SCADA devices in their machinery that can tell them a lot of useful information from a historical perspective, as well as in real-time and they’re doing this today where they’re getting real visibility of what’s going on. 

But one profound thing that actually happened throughout this exercise is that this organisation realised that power goes into a negative cost, so if the grids are over-producing and the data is telling them that, somebody has got to consume that data.