Why the IT World Needs a Ubiquitous Data Access Layer for Database Scalability
On a bright sunny day in May 2011, my good friend BV Jagadeesh, the former CEO of NetScaler, called me and asked me to meet a former NetScaler customer who had “solved the database scalability issue.” With my curiosity piqued I hiked up 3rd street in San Francisco to meet with Varun Singh, the founder/CEO of a young company named ScaleArc, in the lobby of a downtown hotel.
What impressed me from the start was that Varun didn’t show me a single PowerPoint slide. Instead, he explained the real-world pain points he encountered as a CTO building some of the largest Web applications and websites in India. He then took me through a live product demo of his product, ScaleArc iDB, to show me how he had solved the pain points. The demo hit home for me. It was nearly identical to the value proposition that NetScaler brought to the data center 10 years earlier. In NetScaler’s case, the ability to act as a L7 HTTP proxy to abstract internet users away from the web/app tier to allow Operations teams to architect, scale, optimize and protect the web/app tier, while achieving real-time visibility into traffic flows. What NetScaler did for the Web/app tier, ScaleArc is doing for the database tier.
But I was almost certain that this type of technology must have existed already in the SQL layer. Not so, according to Varun. He described the situation he faced as the CTO of a large media company. As he built more large-scale applications and more websites, his 15 best application engineers spent all of their time coding repetitive database engineering in each app, rather than working on the next great app. They were forced to do this because, today, all database engineering/networking is coded into each application, and there is a forced 1:1 relationship between apps and databases in the relational database world. This makes it incredibly difficult to scale relational databases, maintain consistent uptime (especially with growth), optimize transparently without having to continuously touch the applications, and obtain real-time visibility to identify problems immediately, and then act to solve them.
I still found it hard to believe that a solution for this didn’t already exist. If F5 and Citrix NetScaler (Citrix acquired NetScaler in 2005) had solved this problem for the web tier, surely someone would have applied the same approach to the SQL tier. So with the ink still damp on the Advisory agreement I signed with Varun, I began taking Varun and the ScaleArc iDB solution on the road, meeting with some of my closest customer friends who ran DevOps teams for Web 2.0 shops, as well as some customer friends in the traditional enterprise space. After a few months of meetings, I came to realize that Varun was right. The problems with SQL database scalability, uptime, performance and visibility are pervasive, regardless of whether you are a MySQL, a SQL Server, or an Oracle shop.
comments powered by Disqus