In demand


Sunday, June 28, 2015

CCK vs. Drupal and Wordpress: A Tale of Scalability

This a tale about how CCK came into being  and why it's unique database design uses database partitioning or sharding support for scalability. Sharding has become the go to solution for scalability and is used at  InstagramFlickr and Freshdesk. But I started using it long ago when the web was not so big and scalability was not a problem.

Data partitioning  is something that I  have used  a half dozen times in the past.  I had small business clients that used Microsoft Access databases and  MS Excel  for  their CRMs.  Many of them  started suffering from performance problems because the software was  designed to handle ten thousand customers using  one  administrator. The MS Access databases were seriously fragmented and overloaded.  Breaking licensing  I solved the problems by changing the VBA software to support using multiple MS Access databases and  network addresses.  This involved making changes to the database architecture to support partitioning.

When I started building a content management system  for small business I thought back to the problems encountered  by them. One of which was scalability and what happens when  they are still small in terms of personel  but  become big time when it comes to data.  Out of habit I went to an old solution by designing a  database schema that supported  partitioning.

Then I faced some technology problems which is why the development of CCK took years before beta.  I was still in Microsoft mode in my thinking and  LAMP  did not have all the pieces.  MySQL was not   supportive to my ideas and using SQL Server was not an option. Things like unique ids and network connectivity were difficult  in the shared hosting environment that most small businesses would use.  PHP was a good replacement for Visual Basic but the relationship between MySQL and PHP was not as good as Visual Basic, ADO and MS Access . Things got complicated very quickly. I dropped the project and started using Drupal and  Wordpress for clients.

Now to the rescue. Over the years PHP became better and  it's feature set expanded.  SQLite came into being and later became part of PHP. This solved the relationship problem between PHP and MySQL. SQLite, PDO and PHP together put me at ease in place that I was familiar. I learned new patterns like MVC architecture which removed the complexities of using unique identifiers and a horizontally partitioned database.

Over the years I also learned  from Drupal and Wordpress internals. I found them be lacking in what I had liked about working with VBA and MS Access desktop applications. They are so chaotic at their core which to me was the database architecture and not the source code. Time after time I found myself muttering obscenities because of the complexities and limitations of not just Drupal and Wordpress but all open source CMS's. Much of which was exacerbated by terrible database architecture it's been ten years and they are using the same schema and ideas.

Last year after receiving news of a personal medical condition I decided to drop all Drupal and Wordpress development. I also deleted any earlier attempts of  improving them by forking because this would not satisfy my need to fix the database schema. Added to that was my growing hatred of the GPL licensing they used.  Making a simpler and more scalable site builder had to come from scratch.

That's the story of scalability in the real world of web applications. Drupal has none though there are those that attempt to add it in using modules and core patches. The Wordpress solution is to throw in more caching , better hosting, domain sharding or hardware. But I consider such solutions to scalability problems non-competitive or too expensive for most. When it comes to scalability CCK is a pick-up truck while Wordpress and Drupal are large cars with poorly attached trailers. The Content Connection Kit is the result of  all my experiences and using new technologies to build a better solution.

1 comment:

  1. Your post about web design and development is really interesting. When comes to content management system, wordpress is widely used tool. It loaded with various stunning features that make website creation lot simpler. Wordpress Training in Chennai


Print this!