Sunday, January 21, 2007

create myspace.com in 4 hours? Part II

Okay, forget myspace.com. Lets see what it takes to create and run a very simple online application. Our requirements are simple.(?)

Requirements

  • Let the user push some data via a web form to a database table.
  • Later the user can query back the same data.
  • One DB Table
  • The application is accessible online on some domain
  • Application works 24x7 with small maintenance windows
  • Anyone can access it
Now lets try to see what all steps are involved in development starting with a virgin machine. I am going to use perl cgi.

First up, we need to set up a development stack . we need a database, a web server and some programming toolkit of your choice. So download mysql , apache and download Camel Pack perl. If you are on linux boxes, most of the stack is ready made ( I do most of my development on win XP box) But creating the right stack is definitely an activity that takes up some time . if you may have perl but not the modules you want. if you may have java but not all the libraries you want. You may have version x when you want version y and so on.

After downloads, come the configuration part. Who has run apache or MYSQL just out of box? You do have to do configuration changes. even creating a single database, 2 users, a single table and configuring apache to handle cgi from your directory takes some time.

Lets come to form development part now. Say you are a very smart designer who knows CSS , HTML and javascript like the back of your hand. You can create both the HTML pages very quickly in a three column layout design. Still you need to make sure that the elements on your form are aligned. You have put the right javascript for form validations. You need to make sure of navigation from search to input and input to search.

Lets say it again that you are very efficient server side coder too. You can create 2 perl CGI scripts in a jiffy. One using DBI to store data in system table and the other to read back the data using tokens. You are a smart developer, so you get everything right in first go. Things like, checking for invalid inputs, making sure of case conversion when doing search , taking care of xss issues so people can not paste javascript URL on your form etc. But still, typing everything in an editor still takes time, isn't it?

Finally , out two forms are ready to be deployed. we input some data , check up the database and see that our table is populated. Now remember that the application is out there in the wild and anyone can do anything to it. So we need to test the forms a bit with all kind of edge cases. numbers-only name, string only dates etc. All goes fine with an occasional bug here and there and we are now ready to upload.

To deploy, you need a domain name and some server space. Lets say I order both from Yahoo. You get your stuff up on hosting server. Getting the domain up and propagated would take also some time. But is this the end once I can access my forms from my very own URL? Is everything over? of course, not. Let 10000 people put their data in your tables and then your searches would literally crawl!!! what happened here? Forgot to analyze your tables, isn't it? How do you take off your site while the maintenance is going on, do you know that?

Anyone can access your application. what if some guy uses some web pager from netbsd machines? what happens to all the groovy ajax validation stuff? Now you see, we need to put in browser match rules in apache config as well.

Morale of the story: slapping together 2 web forms with some server side script is no big deal. But to design and deploy an application that can be accessed by anyone and used in any which way is definitely a big deal. Online application look simple, but they are not!




© Life of a third world developer
Maira Gall