Intel Tackles Water Supply Problems With Big Data

With the majority of Big Data conversation focused on solving business problems it is easy to lose sight of the fact that the ultimate opportunity of Big Data is solving big problems -- problems that affect our entire society. One of them is hunger and the mix of Big Data and agriculture has the potential to arm companies and governments to tackle it more effectively than has ever been possible.

One of the companies looking at ways food-supply problems can be addressed with Big Data is Intel. The company currently has two research projects aimed at using Big Data to solve the world's food and farming challenges. One looks at irrigation and the other at snow mapping in the Sierras.

The two initiatives are part of a wider program at Intel to apply Big Data to a range of specific problems. Its short-term goal is to drive research insights, while the long-term aim is creating reference architectures that can apply across a variety of industries and drive value.

"We see these in terms of grand challenges and tough problems," Vin Sharma, Intel's director of planning and marketing for Hadoop, said. "We want to push the boundary. We do see commercial value, but it doesn’t have to be immediate."

The commercial value for Intel can be enormous in the long term. Every industry is looking at ways to use Big Data, different verticals at different stages. "With Big Data, we’re starting to see a secular movement across all industries," Sharma said.

If Intel creates a platform and reference architecture that works across all verticals, it will guarantee itself a big piece of the action, whatever it may be in the future. "We provide the platform and components, but we also talk to the end users to understand their demand patterns. It’s a good mix of things. By being able to solve problems, document, create a reference architecture, we’re able to use that as reference to replicate."

Big Data in 'Precision Farming'

The irrigation project is called Precision Farming. Intel is working with University of California, Davis, to perfect irrigation techniques by placing sensors in crops and other strategic areas to monitor soil and air moisture levels. "The rate of water supply to the crop is [currently] determined on an ad-hoc basis," Sharma said. "The notion is that there is a lot of waste because of over-provisioning. You instrument better and you tie that back to the supply of water and reduce waste."

He said said this approach could reduce the amount of water used for irrigation by as much as 50 percent.

There are challenges to this beyond technology, however. "We sense there is concern within the farming community. Let’s say the farmer is renting the land. If they transfer the ownership of the land, who owns the data? We want to be part of the effort to democratize [it] as well."

Mapping snow to plan ahead

The second project uses data from a lab that monitors the snow pack in the Sierra Nevada mountain range, correlating it with the size of California's water supply. The goal is to create a database of images that will enable governments and farmers to predict drought conditions and plan accordingly.

This is not a new problem. What is new is the amount of data available now that the lab uses remote sensing equipment instead of a stick in the ground to measure snow levels. The lab's "snow-mapping" technique deduces snow coverage by rating reflectance of snow on a one-to-seven scale.

"It generates about two terabytes every 15 days, and the source files [are] 75 gigabytes a day," Sharma said. "The challenge they want to address is make that data 'queryable.' We correct the files and load them into a database called EarthDB, based on work that Michael Stonebreaker did." Stonebreaker is a famous computer scientist who specializes in databases. Intel's Science and Technology Center funds the EarthDB project at the University of California, Santa Barbara.

California key state to tackle water issues in

It is relevant that these projects are being conducted in California. The state produces nearly half of U.S.-grown fruits, nuts and vegetables and is currently facing one of the worst droughts in recent years.

Extreme weather is causing the price of some foods, such as lettuce and avocados, to skyrocket, and many predict that shortages of certain kinds of produce will soon become a fact of life.

Benefit potential beyond Intel's bottom line

By tackling the big problems, Intel is investing in the future of all humans as well as its own. One of the immediate benefits for the company has been a quickly growing varied pool of Big Data talent.

"In the beginning we had one or two data scientists," Sharma said. "Because of our engagements with a number of these, now we have a very interesting group with specialized expertise in healthcare, telco and now increasingly in food and agronomics."

Success of these efforts will extend far beyond Intel's profits.