Expert Interview with Daniel Lemire on Better Uses for Big Data

Daniel Lemire is a professor of computer science at the University of Quebec in Montreal, Canada, as well as a dedicated blogger and DIYer (he makes everything from bread to furniture to robots). Open source software that he has created has been sourced by the likes of Google and Facebook. And when it comes to research, he’s interested in databases, information retrieval and high-performance programming.

In short, he’s a pretty smart guy and someone you want to include in conversations about the future of data.

We recently checked in with Daniel to get his insight on Big Data and what we should be doing now to help future generations harness this technology. Here’s what he had to say.

Can you tell us a little about your professional background and areas of expertise?

I am interested in indexing problems, as well as related issues such as index compression.

What news or innovations in the tech world are you following closely today?

I am excited about new upcoming processor technology.

What interests and/or excites you about how Big Data is being used today?

It has the potential to revolutionize science and engineering.

How do you think businesses can use Big Data more effectively and responsibly?

Though it did not start out this way, a lot of the focus of Big Data has been on marketing and social media. Though this is certainly an interesting and important area, I think that there are far more important applications elsewhere, such as improving medicine.

Who do you think is using Big Data in the most innovative ways today? What can we learn from them?

Google comes to mind. Though they did not invent Big Data, they are the industrial poster boy for big data. I think that the industry has tried to learn from Google all it can.

For example, we have seen a surge of interest for computer science talent, and I think that this partly comes from the push of companies like Google who did not hide the fact that they were trying to put their hands on the best people.

I believe also that Google illustrates a software-oriented paradigm: though having access to the data and partnership is valuable and even essential, it is nothing at all if you cannot develop great software.

What types of skills do you think CTOs and CIOs should be looking for in the developers and IT professionals they hire in order to stay at the forefront of database management?

How often do you hear the phrase “company X has the best database managers?” What we do see is that the big bucks are in more advanced problems like data integration, data warehouse architectures, and low-latency information systems.

On your blog, you seem to have a special interest in education. How do you think children today should be taught to better prepare them for a data-driven future?

Give kids capable computers and a solid Internet connection and things start happening. My boys play Minecraft, have their own YouTube channels, and so on. They catch up very fast.

There is a lot of very depressive and (sadly) often well motivated talk about kids missing the boat because they are not smart enough. I think it is a real concern and we need to work harder to make sure we find out how to get each new generation to exceed the previous one. That has not seemed to be enough of a concern. Frankly, it is research in education that is missing the boat.

What do you think schools spend too much time focusing on? What is the danger in this from the perspective of a computer scientist?

We spend a lot of time obsessing about 19th century problems. We teach long division, and we obsess over spelling. All of these things are better left to computers.

Connect with Daniel on Twitter and Google+.