A new study by Tata Consultancy Services done in conjunction with Forrester Consulting looked at the state of IT within the global retail industry and CIOs’ attitudes and plans toward key trends and ever-more disruptive, challenging technologies.

TCS Study Shows Retail CIOs are Primed to Lead the Innovation Agenda

In-depth interviews with senior business and IT executives at global retailers found that the potential for CIOs to embrace disruptive technologies are too often hampered by a lack of key resources and business alignment. This is illustrated through the fact that almost two thirds (64 percent) of global retailers consider cost reduction as a major focus over the next few years, versus only two fifths (38 percent) citing innovation.

While the focus is still revenue growth, Retail CIOs believe that the disruptive technologies of mobility, social media, cloud and Big Data will continue to radically transform the retail industry status quo, yet they are not staffed or structured adequately to take full advantage.

Read more.

Read this:

Wal-Mart Stores, struggling to translate its brick-and-mortar success to the Web, is using free software named after a stuffed elephant to help it gain an edge on Amazon.com in the $165.4 billion U.S. e-commerce market.

With its online sales less than a fifth of Amazon’s last year, Wal-Mart executives have turned to software called Hadoop that helps businesses quickly and cheaply sift through terabytes or even petabytes of Twitter posts, Facebook updates, and other so-called unstructured data. Hadoop, which is customizable and available free online, was created to analyze raw information better than traditional databases like those from Oracle.

“When the amount of data in the world increases at an exponential rate, analyzing that data and producing intelligence from it becomes very important,” says Anand Rajaraman, senior vice-president of global e-commerce at Wal-Mart and head of @WalmartLabs, the retailer’s division charged with improving its use of the Web.

And then read this story from yesterday:

Wal-Mart Expects Flat U.S. Sales This Quarter

Delayed tax refunds in large part contributed to the slow start of the fiscal year, Wal-Mart U.S. Chief Executive Bill Simon said. At this time last year, Wal-Mart had cashed $3 billion in tax-refund and refund-anticipation checks, he said. It has cashed just $1.7 billion this year.

Some of that tax money is typically spent around Super Bowl time for television sets. Now, the retailer doesn't know how the money will be spent, Mr. Simon said.

Isn’t that a classic problem to be solved using Big Data. I would have thought that with as easy problem statements as “What did our customers do when their tax refunds were delayed?” or “what did they do when they did not buy TVs around Super Bowl”, and with history as old as Walmart’s, it would be easy to figure out customer behavior.

Apparently it’s not.

Or is there more than what meets the eye?

Over the last few months, there has been a lot of coverage on how Retailers are using Big Data. Wal-mart with its recent acquisition of Kosmix is one Retailer which  is in the forefront of this wave. Here is a collection of articles which discusses how Wal-mart is using Big Data.

How Walmart plans to use Big Data

Kosmix stands out for its ability to search and analyze connections in real-time data streams to deliver highly personalized insights to users. The platform powers TweetBeat, a real-time social media filter for live events. By using this intelligence, Kosmix is building a giant knowledge base called the‘Social Genome.’ This giant knowledge base captures information and relationships about entities such as people, events, topics, products, locations and organizations.

…

By analyzing their social media activity, Social Genome can make recommendations about products, events or any other activity that the user is interested in. For example, by using publically available social media data, the Walmart product store can suggest product recommendations, based on recent tweets or Facebook wall posts.

…

While the idea sounds great, doing this in reality is a huge problem — especially since there are thousands of data pieces flowing in a torrent from live data sources such as tweets, Facebook posts and blogs. The data flow was so fast that Kosmix could not rely on the traditional Map-Reduce or Hadoop framework that is typically used to solve Big Data problems.

“Social Media data is the fastest growing source of Big Data today. In addition to being Big Data, social media data such as Twitter also has a real-time nature — it’s not just Big Data, but also Fast Data. With mobile devices, location data is now a new source of both Big and Fast data,” explains Rajaraman, on the technical challenges faced by his firm while building the platform.

To address this Big Data and Fast Data problem, Kosmix developed its own in-house solution called Muppet, which processes streaming fast data in a lightening fashion, over large clusters of machines. Today, Muppet can manage and track data streams with billions of updates a day.

Getting a Handle on Big Data with Hadoop

Wal-Mart Stores, struggling to translate its brick-and-mortar success to the Web, is using free software named after a stuffed elephant to help it gain an edge on Amazon.com in the $165.4 billion U.S. e-commerce market.

As customers flock to social media, Wal-Mart expects sites such as Facebook and Twitter to play a bigger role in online shopping. By analyzing what social network users say about products on those sites, the world’s largest retailer aims to glean insights into what consumers want.

With its online sales less than a fifth of Amazon’s last year, Wal-Mart executives have turned to software called Hadoop that helps businesses quickly and cheaply sift through terabytes or even petabytes of Twitter posts, Facebook updates, and other so-called unstructured data. Hadoop, which is customizable and available free online, was created to analyze raw information better than traditional databases like those from Oracle.

“When the amount of data in the world increases at an exponential rate, analyzing that data and producing intelligence from it becomes very important,” says Anand Rajaraman, senior vice-president of global e-commerce at Wal-Mart and head of @WalmartLabs, the retailer’s division charged with improving its use of the Web.

Big data and the disruption curve

Big data projects are aimed at revenue growth, many efforts are being funded by business units and not the IT department and money is increasingly being diverted from large enterprise vendors.