A Look Under the Hood of Onspring
When we founded Onspring back in 2010, we needed a great set of tools—including high-performance, easy-to-use databases and excellent development frameworks. In those days, some of the tools we needed didn’t exist or weren’t mature enough to be of any real value. We made do with what we could find at the time, always with an eye toward the future.
To define our technological goals at Onspring in simple terms, we needed the following:
- Extremely fast and highly flexible database
- High performance text indexing and searching
- Rapid feature development
- Simple and easy to maintain
- Superb scalability in all areas
The Big Choice: NoSQL Database
We began development of the Onspring Process Automation Platform using standard tools that had been around for decades, in order to get the ball rolling. We designed and built our code in a way that would allow us to quickly and easily replace individual parts of our data stores without significant effort. Then, several months into development, through significant testing, experimentation and patience, we were rewarded with a long awaited release to a NoSQL database we had been considering. Those updates, still in alpha, meant that we finally had something that met our needs. One problem down.
The choice of this database ended up being greatly influenced by something we did not ultimately expect. It met our basic needs of fast, flexible, scalable. What truly surprised us was our ability to rapidly accelerate our development. We chose a NoSQL document-based database for performance and scalability reasons, but were being rewarded with vastly improved development times. Most of the databases we reviewed and tested were document-based, even the key-value stores would store values as documents if you wished, but this one had a very high quality set of drivers beyond any of its competition. The excellent developers behind those database drivers clearly understood the power of their database and how it could best be extended into our world.
The ultimate value came with the realization that we no longer had to normalize data in any meaningful way. Sure, we still do some normalization, but only when it’s of great benefit. By being able to take an entire set of data and place it into one “document,” we were able to save substantial time writing methods that would have otherwise been required to disassemble and reassemble this record from dozens of database tables. The kicker? Substantially improved response times and little to no additional code to perform complex queries.
The implications of this were a 50-75% reduction in our development time. Our need for a dedicated or knowledgeable DBA went kaput. We no longer had to worry about adding new objects or columns into a traditional table. If we wanted a set of data to store something new, we just added the property to the object in code and completely ignored the database. If we wanted to avoid storing defaults in the database to save space, but still have perfect query-ability, we simply added a property to the serialization functions and the driver did the rest.
Next Steps: Text Indexing, Caching and More
All this power in hand, our development progress moved forward quickly, and our need for other, more focused solutions—caching, text indexing and relationship graphing—became more prominent. We had high hopes that our database of choice would give us what we wanted in some of these areas. However, we decided to compare the use of specialized, highly scalable just-as-modern tools against our database to evaluate performance.
Ultimately, we discovered that each job we have to do may be contained in several parts. And writing a management layer of communication between those parts was substantially easier and more responsive than trying to find one tool to rule them all.
We found an extremely impressive and highly scalable text-indexing tool. It’s grown up in recent years and actually has the capability to replace much of our database, should we choose that option. However, without the right mix of powerful drivers for our language, this won’t likely happen any time soon.
We use a third tool, a key-value store, as our caching layer. These reside on each of our web servers and have greatly decreased the number of queries to our database, especially for infrequently modified data. Proper utilization of this distributed caching layer was purely for invalidation.
Our final addition to the mix was a self-built graphing database to solve the issue of scanning the web of complex relationships between data that’s sometimes spun. More on this in a future post.
Adding It Up: The Power of the Platform
We’ve ended up with a highly powerful and scalable database at Onspring. This was easily the best decision we made in our technology. We also have the most powerful text indexing tool we’ve ever seen and the best caching software around. When you add in well-written drivers that each serve their purpose and a dedicated, talented development staff, what you get is a product as immensely powerful, flexible and scalable as Onspring.
And the best news for you: Our developers work to ensure that you don’t need developers to create elegant business solutions. You can do it all yourself without ever touching a line of code. Onspring integrates the world’s best technologies on the back-end with a brilliant, consistent software layer that gives you amazing performance, usability and control.