Expert Interview with Antony Falco on Databases in the Cloud

Speaking with Antony Falco, founder and former COO of Basho, turned out to be a good lesson on databases in the cloud.

If you’re a programmer or developer who wants to spend more time flexing your creative muscles and less time getting mired in database maintenance, then it’s time to look to cloud-based services, says Antony Falco, now CEO and co-founder of Orchestrate.

These services allow programmers to be more creative and solve more problems by eliminating unnecessary tasks from their workloads. In the building stage, programmers should be focused on what makes their application unique and getting it out in front of users early.

“Anything that isn’t about your core mission is a distraction at best and a future limitation to your business at worst,” he says.

Here, Antony discusses how Orchestrate helps developers focus on building a better user experience and offering better design and features, as well as discusses the importance of harnessing big data for business.

Tell us the story behind Orchestrate ...

Before founding Orchestrate, I co-founded a company called Basho, which is well known for its NoSQL database, Riak. Time and again we found ourselves in the same situation - helping businesses navigate the confusing, crowded NoSQL market, which had grown to over 20 different databases, and educating them on how to join these databases together in order to power their apps. Because each database has its own unique strengths for different querying capabilities, many companies would need to queue up not just one database for a single application, but frequently three or more separate databases.

The use of multiple databases in production, which is often called polyglot persistence, allows businesses to add more features to their applications. However, it also comes at a high cost. Each database requires a certain level of expertise to deploy and manage, which means hiring DBAs or developers with the right skill set. And while the costs associated with running each database are not negligible, the majority of the spend incurred would be from the support and staffing required to run them in the years after launch. And it can take anywhere from weeks to months to provision these databases before a project can be built. Using multiple databases to power a single app, while allowing for the richer features we have come to expect in many social and mobile applications, also has trade-offs in the complexity of the application itself, often requiring multiple calls to the data layer to satisfy a user single interaction.

We watched as other infrastructure services began to become available via simple APIs. SendGrid and Twilio are great examples. There was a movement growing for complex infrastructure delivered through simple services. We knew how to build databases, and how to effectively join them together, so we thought - why not join the best NoSQL databases together and offer them all in a single, simple service? And Orchestrate was born - a database service that delivers the benefits of polyglot persistence, without the complexity, at a fraction of the cost and that is simple enough for most developers to use.

Who should be using Orchestrate, and how will it help them do their jobs better?

Orchestrate is a great solution for Web application developers, enterprise developers, Internet of Things developers, and digital and creative agencies that need to scale up and scale down campaigns rapidly. By removing the barrier of setting up and managing databases when building applications, developers can focus all of their time on what really matters to their users - a better user experience, better design and better features.

Why is Big Data so important to the growth and survival of businesses today?

Big Data is to businesses today what GPS is to personal navigation by car or on foot. What at first seemed like a luxury or the purview of specialists (like logistics companies and airlines) quickly filtered down to the mass market, bringing all the benefits of up-to-the-second positional feedback to the saleswoman, the tourist, the hiker, or the daily commuter. Sure, you can get along without GPS, but you are sacrificing valuable information that can save time, save frustration, and even save lives (here in the Pacific Northwest, we encourage our hikers to head out armed with at least basic positional technology).

Big data is the raw material needed to provide a corporation with situational awareness - where am I in my market? Where are my competitors? Who are my most valuable and least valuable customers? Where do they come from?

How long before that sort of information isn’t a luxury but a necessity for your company’s survival?

What can businesses do on their sites to better harness the data available to them?

This is the area where I see the most obfuscation by vendors and subsequently the biggest resistance among corporations. Everyone thinks they can’t start gaining the advantage of a more analytical, interactive, data-intensive approach to their market without a fully baked strategy, a Big Data Five-Year Plan cast into bronze and full of the promise of 50 percent quarter-over-quarter growth. Nonsense.

Like any seemingly daunting task, pick a small area, test some assumptions, learn the tools, and iterate quickly. Pick a single metric that matters - not something huge like revenue, but something important but discrete like shopping cart abandonment on weekends - and begin to dig into the factors that drive it.

Two things will happen - suddenly Big Data doesn’t look so big and daunting. Next, you begin to learn that the tools to analyze mean nothing without the tools to test and iterate. And here is where companies like Orchestrate come in - and I mention this not as plug for Orchestrate but to explain how real-time infrastructure and the application stack cannot be separated from big data and analytics.

If you don’t have a stack that allows for rapid iteration and experimentation, you cannot really take advantage of your Big Data learnings as efficiently as competitors that do. If every time you need to run an experiment where you add a new feature to your shopping cart flow - friend recommendations, for example - you have to stop, find an expert in graph and search, spin up new servers, hire someone or find an internal resource that knows these new databases, and wait for all this to get set up before you even start experimenting, I can guarantee what will happen: you won’t. Experimentation will stop. All your “learning” will be untested and you will go backwards, away from analytical decision making, to the bad old days of anecdote and gut-feel interpretation of data.

Big Data cannot benefit your business without a framework to test out new ideas and iterate quickly. Slow infrastructure means slow iterations. Competitors that use vendors like big Step and Orchestrate, which allow for rapid testing of Big Data learnings, will have a distinct advantage in the near future.

Think about it this way - would you rather be the visitor setting out on a sprawling wilderness hike armed only with a guidebook and a compass or the one with a GPS as well as a guidebook and compass?

Why is it that since their inception, databases struggle with being human friendly?

It’s less about how human friendly these databases are, and more about what really matters at the end of the day - design, cost, and time to market. There are plenty of skilled, proficient DBAs and developers out there who love to run databases, or are simply used to running them themselves. There’s nothing wrong with that.

But think of it this way: Say you’re an artist or a fashion designer or an architect. Your creativity is what shines through your work and what your customers or fans love. Are you going to go out and make your own dyes, or make your own wool, or cut the trees down yourself before you create the final product? At the end of the day, why re-create the wheel when you can get straight to making?

How is this changing?

It’s changing before our eyes - cloud providers and databases left and right now offer services so people can get up and running quickly. This is the future of infrastructure, where complex tasks and requirements are rolled into simple services that are easily consumable through APIs.

What are the biggest complaints or most common frustrations programmers and developers come to you with?

We see a lot of developers come to us after realizing that they need to add yet another database to their stack - but find that adding a new database is a non-starter due to the sheer time and cost provisioning another database requires. The cost to add and operate new features in order to remain competitive quickly pile up, decreasing the impact of the feature on the bottom line.

How do you help them?

Simple - Orchestrate integrates seamlessly with existing applications and infrastructure. Developers sign up, start using our API in minutes, and get straight to important job coding new applications and features.

What are some best practices developers should be aware of when using a database to develop software?

Developers should think about the feature they want before choosing the database, instead of saying “I know MySQL or MongoDB so I’m going to use that.” It isn’t easy to migrate between databases on an existing application running in production, so developers better make sure before choosing. A database decision is one you live with for a long time. We have tried to build something that has built in flexibility. If the developer starts out needing only simple key/value or document operations and soon realizes their users want full-text search or a social graph-based feature, no new databases and no migration are needed.

However, the question I’d stress is: Do I need to be the one running the database? Do I need to spend valuable resources on running the database or can those resources better serve my business or project goals when applied to addition front-end coders, another UX designer, or even a bigger marketing campaign? How often do users say, “Wow, this mobile game delights me because they run their databases really well”?

What are potential risks or drawbacks to using them?

The drawbacks are really the inverse of best practices. First, do I want to get mired in the business of running databases when I am trying to build a brand or reach a huge audience? Second, do I want to run the risk that, as my data model changes and new features are needed to stay competitive, I have to do a live migration or add new databases?

What are some of the coolest or most innovative programs you’ve seen that have been built using the help of Orchestrate?

A great example is MarvelousDB. The app was created over a weekend by pulling Marvel’s API, which includes the entire trove of Marvel comics, characters, and associated data. The developer built a feature-rich application using Orchestrate to query the data and used search, events and graph to allow users to explore all of the data in a beautiful interface.

Some of our customers are building exciting Internet of Things applications on Orchestrate, including Cargo.ai, a connected car startup. We have a few more case studies you can explore on our website, as well.