2010-03-22

Infinitely configurable web application

Today, I wan't to entertain the idea of making an application, which behavior can be modified in any way by the user (by user, I mean the administrator of the server running your app).

Most applications have a notion of a config file, which usually is just a collection of setting-configuration pairs. When a project gets bigger, settings are separated into sections. If the problem domain requires it, sub-sections can represent similar entities, with the same available settings, but different configuration. Think about a web server, which can serve multiple domains, configured in various ways.
Sometimes the sub-sections can create a never ending tree. The configuration is then being derived from the root node and all branches on the path to the node, which configuration we want to retrieve. Apache .htaccess configuration files come to mind.

How does this map onto web applications?

A web application has a tree structure, from the top, there are:
  1. domains,
  2. URLs,
  3. pages,
  4. parts of a page,
  5. elements on a page.
Getting right to the point, an example configuration file for this blog could look something like this:

<Site domain="blog.xek.pl">
  <StaticPage url="/">
    <SpanVertical>
      <Header title="XEK"/>
      <SpanHorizontal>
        <LastPosts no="7">
          <PostTitle with_date_header="true"/>
          <PostBody/>
          <SpanHorizontal>
            <PostAuthor/>
            <CommentsLink/>
          </SpanHorizontal>
          <PostLinksTo/>
          <PostLabels/>
        </LastPosts>
        <SpanVertical width="10">
          <BlogArchiveLinks/>
          <AboutMeSection>
            <Avatar/>
            <Name description="true"/>
            <ProfileLink text="View my complete profile"/>
          </AboutMeSection>
          ...
        </SpanVertical>
      </SpanHorizontal>
      <Footer/>
    </SpanVertical>
  </StaticPage>
  <ObjectPreviewPage url="/[object_id].html" object="BlogPost">
    <SpanVertical>
      <Header title="XEK"/>
      <SpanHorizontal>
        <BlogPost>
          <PostTitle with_date_header="true"/>
          <PostBody/>
          <PostLabels/>
          <PostLinksTo/>
          <PostComments/>
          <PostCommentForm/>
        </BlogPost>
        <SpanVertical width="10">
          <BlogArchiveLinks/>
          <AboutMeSection>
            <Avatar/>
            <Name description="true"/>
            <ProfileLink text="View my complete profile"/>
          </AboutMeSection>
          ...
        </SpanVertical>
      </SpanHorizontal>
      <Footer/>
    </SpanVertical>
  </ObjectPreviewPage>
</Site>

You don't have to make everything configurable at once. In first iteration you can make your app accept different settings depending on which domain (or installation) it is run, but shortly you will notice, that many pages are similar and belong to one type or class, you also have many repeating and common elements on pages.

Implementation details

I'm sure you noticed the use of CamelCase in my tag names, this is because everything from a Site to a CommentsLink can be implemented as a class. The class is then initialized with the configuration file, making the above file an instance of the Site class. The only thing you need to remember is that a child element in this configuration tree, must implement all interfaces the parent element want's to call. In the above example the BlogPost and LastPosts call the same API, and as a result, all children of these elements can be freely swapped between them. The above example shows only the data presentation part, but you can do editing, forms and fields just as easily. If you are implementing tabs or steps with different URLs, you may want to make the Page objects able to call themselves. As a side-note, you probably want to make the number of these interfaces small, so more elements could be used in more places. Interfaces should be documented.

How does this compare to current web-frameworks?

Well first of all, when I think about web-development frameworks, I think about Django. That being said, the above configuration scheme, shouldn't be to hard to implement. Modifying the core framework won't be necessary - this is because all configuration files are Python executable code.
The main problem in Django, is that all Controllers (called Views) are just functions. Their configuration is partially stored in urls.py, which specifies the URL path and potentially other configuration (which is passed as function arguments), and partially in settings.py, which usually just stores setting-configuration pairs. Having a function, you don't know what interface it implements and what other APIs are called internally, making the Views themselves hard to unit-test. You can stack function calls, but then, configuration becomes embedded inside your code. I know other frameworks implement Controllers as classes, which is a big step forward, making Convention over Configuration do wonders. The problem with these classes is, that they operate as singletons, what in my opinion, is a big needless constraint. In contrast, the "Infinitely configurable" design pattern instantiates each class with a configuration, making it a reusable component in more than one place of your application. There are other constraints in web frameworks, which you have to take into consideration - I will only mention Django template-tags and forms, each of which enforces a set of unchangeable rules (you have to edit templates to move a template-tag, you can't just move an edit form implementation to another arbitrary page).

Next step

Up until now, I talked about advanced users, users which can read and edit an XML file, but lets take it to the next level. You probably noticed the above file is kind of repetitive. This stops to be a problem when you move it to a different data structure, inside a database! Now, if you have it inside a database, you can make a nice admin panel for it :) . Each class, can render it's configuration form, and all classes can export information about where they can be placed in the configuration structure, in respect to other elements. This makes the application very, and I mean Very configurable. I leave the rest to your imagination.

What are the benefits?

Your application is now a product. You can distribute it to end users and they can at least disable features they don't want. If you implement it right, the code for these features won't be even loaded when the application is started, so your app can stay competitive to any newcomers.

Less code. After a certain break point, you may find out that many of the new features requested by your users, can be implemented only by changing the configuration.

DRY, by example. Let's assume you have a standard API, for editing forms, it is implemented in a class, from which other functionality inherits. As your site becomes more AJAX-friendly, a client requests a registration box, a newsletter box and also to move the poll to an AJAX reloaded page element. In my configuration scheme, you would write an AjaxWrapper class, and then change configuration for this one client.

Let me know what you think.

2010-03-21

Hello World

Welcome to my new blog, on which I grok architectural and low level problems in web application development, my goals being: quick but extensible implementations, readable and easy to review code, efficient and predictably scalable solutions.

About me

I have a 5 year experience in web application development - programming in Python, JavaScript and Erlang, and some other previous experience (like my abandoned mobile jabber communicator and my SoC project). I worked for 2 years at grono.net, which, at the time, was the largest social networking portal in Poland, and the largest deployment of Django in the world. I later moved on to work at Sensi Soft, which became The Python Development Shop in Warsaw. I had the chance of working and providing solutions for clients all over the world: Harte-Hanks "North America's largest owner, operator, and distributor of weekly shopper publications", Friday-ad "the largest Independent publisher in the UK", Junk Mail "Your King of Classifieds" from South Africa, Onet.pl "the biggest and most popular multimedia portal in Poland" (I can confirm this one), itself a part of ITI Group which owns TVN, a leading Polish free-to-air television channel. At Sensi Soft, I worked in a position of a Team Leader (together with more then 20 developers in one project), and later became the Solutions Architect.