Toluu

O'Reilly Radar

http://radar.oreilly.com/

The Commoditization of Massive Data AnalysisYesterday

Big Data is a major theme on the O'Reilly Radar, so we're delighted to welcome guest blogger Joe Hellerstein, a Professor of Computer Science at UC Berkeley whose research focuses on databases and distributed systems. Joe has written a whitepaper with more detail on this topic.

There is a debate brewing among data systems cognoscenti as to the best way to do data analysis at this scale. The old guard in the Enterprise IT camp tends to favor relational databases and the SQL language, while the web upstarts have rallied around the MapReduce programming model popularized at Google, and cloned in open source as Apache Hadoop. Hadoop is in wide use at companies like Yahoo! and Facebook, and gets a lot of attention in tech blogs as the next big open source project. But if you mention Hadoop in a corporate IT shop you are often met with blank stares -- SQL is ubiquitous in those environments. There is still a surprising disconnect between these developer communities, but I expect that to change over the next year or two.

We are at the beginning of what I call The Industrial Revolution of Data. We're not quite there yet, since most of the digital information available today is still individually "handmade": prose on web pages, data entered into forms, vid

DIY Appliances on the Web?November 18

Or, My Enterprise is Appliancized, Why Isn't Your Web?

I wrote a couple of posts a while back that covered task-optimized hardware. This one was about a system that combined Field Programmable Gate Arrays (FPGA's) with a commodity CPU platform to provide the sheer number crunching performance needed to break GSM encryption. This one looked at using task-appropriate efficient processors to reduce power consumption in a weather predicting super computer. In these two posts I sort of accidentally highlighted two of the three key selling points of task-specific appliances, sheer performance and energy efficiency (the third is security). The posts also heightened my awareness of the possibilities for specialized hardware and some of my more recent explorations that focused on the appliance market in particular got me wondering if there might be a growing trend toward specialized appliances.

Of course, specialized devices have been working their way into the enterprise ever since the first router left its commodity Unix host for the task-specific richness of specialized hardware. Load balancers followed soon after and then devices from companies like Layer 7 and Dat

Voice in Google Mobile App: A Tipping Point for the Web?November 18

As I wrote in Daddy, Where's Your Phone?, it's time to start thinking of the phone as a first class device for accessing web services, not as a way of repurposing content or applications originally designed to be accessed on a keyboard and big screen. The release of speech recognition in Google Mobile App for iPhone continues the process begun with the iPhone itself, of building a new, phone-native way of delivering computing services. Here are two of the key elements:

1. Sensor-based interfaces. Apple wowed us with iPhone touch screen, but the inclusion of the accelerometer was almost as important, and now Google has shown us how it can be used as a key component of an application user interface. Put the phone to your ear, and the application starts listening, triggered by the natural gesture rather than by an artificial tap or click. Yes, the accelerometer has been used in games like tilt, parlor amusements like the iPint, but Google has pushed things further by integrating it into a kind of workflow with the phone's main sensor, the microphone.
   

   
   This is the future of mobile: to invent interfaces that throw away the assumptions of the previous generation. Point an
   

   
   

Twitter's Hockey Stick Moment?November 18

Over the weekend, TechCrunch postulated that with a frenzy of election-related activity, Twitter hit its hockey stick moment in late October. The theory goes that Twitter saw a 25 percent increase in U.S. visits from September to October and is thus about to experience the sort of explosive growth that will propel it into mainstream consciousness.

That could well be the case. In the course of researching our new report, "Twitter and the Micromessaging Revolution," we found that Twitter's user base grew more than 500 percent from October 2007 to October 2008. But we were even more interested to discover that the service has enjoyed an usual effect: as more and more people have joined, the percentage of active users has remained constant [updated:] at about 20%. Among active users (those who post at least once a month), approximately 20 percent post daily and about eight percent post more than 100 times a month (not including known bots and feeds). Though web services usually see a drop in the rate of use as lots of tire-kickers come and go, Twitter's steady usage suggests that a jump in visitors during October could correspond to a big increase in regular users.

Yesterday, a Twitterer asked Tim O'Reilly why "Twitter and the Micromessaging Revoluti

Daddy, Where's Your Phone?November 17

I met recently with Vic Gundotra, formerly Microsoft's head of platform evangelism, and now VP of Engineering at Google, responsible for all their mobile efforts outside of Android. We were talking about Google's mobile strategy and the insanely cool new voice-activated Google search in the Google Mobile Application for iPhone. But what I really want to share is Vic's story about why he left Microsoft. It was one of those "wake up, the future is staring you in the face!" moments that we all experience from time to time, but often ignore.

The story goes something like this: Vic was out for dinner with family and friends. The adults were on one side of the table, the kids on the other. The adults were debating some issue, and Vic said, in response to a question from one of his friends, "I don't know."

His four-year old daughter Samantha, whom everyone knows as "Tiger," piped up from the other side of the table: "Daddy, where's your phone?"

"What do you mean, where's my phone?" She explained that she'd overheard the question. Why wasn't he just looking up the answer on his phone?

Out of the mouths of babes. Vic said that he realized in t