It is time to take the WWW behind the shed and put it out of its misery.

Almost everything that makes up the WWW is a hack. HTML, HTTP, SMTP, are all antiquated technologies. These old technologies were engineered with the sole intention of delivering and sharing text documents. Early web browsers and e-mail clients shared this same basic vision in the beginning. Even today, most media requires plug-ins and other technologies in order to be shared over the web. Where is the problem? It lies in the expectation in the end-users of the Internet. Over the last several years, the major players on the Internet are not delivering documents, but services and applications, which infer an expectation of user experience.

The expectation of a controlled user experience is not a reasonable expectation of the Internet. The Internet by definition does not have any of the building blocks for a true user interface. Cookies are a terrible way to manage state, and HTTP+HTML has no traditional support for static or persistent elements. To go further, the standing trend over the last several years has been to deliver web-based applications. The complexity of these applications has been growing over the years to include two-way communication, streaming data, persistent state, and all of the other elements that traditional computer applications contain. All trying to be consistent across dozens of different browser/OS combinations with different connection rates and privacy and security restrictions.

HTML Limitations

• Inconsistent across different platforms
• Not intended to be used as a UI framework
• Poor state control
• No built-in multimedia control

HTTP Limitations

• 2 connection limit per DNS host name
• Does not support Keep-Alive

The worst part of this combination is that the services that rely upon these technologies to deliver their service have little or no control upon how the two interact, because their service is a slave to whichever web browser the client uses to access them.

Standing Hacks

• Frames
• DHTML and DOM
  
• Plug-ins and ActiveX objects
• Java Applets
• XMLHttpRequest object

Java Swing

Java's Swing framework was probably the first reasonable attempt of a resolution. Its failures and shortcomings were few. Java is too heavy of a language for a UI framework. It isn't powerful enough to be fast and efficient. Java Runtime Environment has too much access to the client's machine, and not enough security for most people. On top of this, Swing's anti-aliasing and 2D drawing is less than attractive.

XULRunner

From those wonderful guys at Mozilla, XULRunner utilizes XML and JavaScript and runs within the Gecko Runtime Environment. It is still young, but shows great promise.

Adobe AIR

Adobe's Integrated Runtime, which was previously named Apollo, shows great promise. Not only does it utilize Flash and Flex, but also JavaScript and HTML. There are quite a few Open Source Flash initiatives such as MSASC and SwfMill for compiling ActionScript and XML into SWF files. Also, ActionScript has matured nicely over the years. If Adobe open-sourced an ActionScript compiler, this may just be the winning platform for the best way to build and distribute your applications.

While HTTP and HTML will most likely remain the foundation blocks of the Internet, I have great faith that many of the services out there with web-delivered applications will be looking towards other technologies such as AIR or XULRunner in order to better control the user experience and deliver richer applications.

Update:
This blog post criticizes web browsers failure to utilize multi-threading in multi-core systems. It is a valuable point and felt it worth adding here. http://pinderkent.blogsavy.com/archives/147

But, What Are We Really Building?

On every major web project that I have ever worked on, there comes that point where I find myself asking 'But, what am I really building?'. Often, the project has some idealistic mission statement about 'Providing people with X and interested in Y a way to connect with Z.' Often, Y=Z or X=Y and the mission statement is all the simpler. However, once the product is actually unleashed on the real world, only 1/10th of the people have X or interested in Y or even care about connecting with Z. The other 90% are interested in people with X or who are interested in Y, and want to connect them with P.

Reverse Your Mission Statement

What triggered me to write this blog entry was the comical, yet sadly true Cracked.com article
Internet Safety Tips. So, my point is this: While mission statements define the market and give support to the projects scope, your product will be ten times as successful if you also approach the mission statement in reverse. How do I allow only Z to connect with people with X who are most likely to be interested in Y. This is particularly in products that contain any part 'social networking' in their mixture of features.

Case in Point

Myspace.com has started to loose substantial ground for the first time in many months according to Google Trends for Myspace. Shortly after 29,000 registered sex offenders were found on Myspace, there has been a noticeable decline in activity and new membership. While some good PR and new features should be able to turn this trend around, you have to ask in hindsight, what could have been done differently to avoid this?

So, to all of you who are out there building your next great web product, please, please, please take the time to say, how do I keep people with X interested in Y from being targets of people attempting to connect them with P. Otherwise, do not be surprised when your product is shanghaied by the porn industry, illegal gambling, or other predators who can probably turn out a higher profit from your users than you can with their lack of moral fiber.

10 Years...

About a week ago, I got another year older. As is typical, you find yourself reflecting from time to time about the past year. Though, while driving home one evening, I instead had a flashback to 10 years ago....

10 Years Ago

I drove an '89 Plymouth Acclaim. It had a cassette player, cloth seats, and windows that you had to roll down. I had a cd player with a tape adapter, but it never worked right with the car radio. My cellular phone had a 10-digit lcd screen and could easily fit in my briefcase and I can even connect to my ISP at about 14.4k using it. My laptop was a Compaq Pentium 100MHz with 16MB RAM and a 4X CD-ROM drive that ran Windows 95. In those days, I was building websites using HTML 4.0, VBScript, JavaScript, Perl/CGI, Server-Side Includes, and was just learning about Allaire's ColdFusion and HomeSite.

Today

I drive a '99 Jeep Grand Cherokee with powered leather seats and windows. It has an SCD player that can also play MP3's, WAV and WMA files off a CD-RW, and has an animated interactive LCD screen that shows me song titles and also heads my CD Changer, and my XM Satellite receiver.

... That's right, I said "satellite"! OK, that really is not impressive, but if I said that 10 years ago, you would not have believed me...

My cellphone today has TWO COLOR lcd screens, and I can use it to connect to the internet as fast as 1.4Mbps. My laptop now is an IBM Thinkpad 2GHz with 2GB RAM and 16x DVD-ROM/CD-RW running Windows XP SP2 and Ubuntu. And, I could not even begin to list all the tools I use these days without forgetting something.


Ten Years From Now

I guess I will be driving an '09 American Something that has power doors, remote climate control, GPS, complete media center that plays DVD's and can stream other media from my home server.

My cell phone will be more of a portable access device that also streams multimedia, surfs the web, has a 20Mbps connection in some places and can interact seamlessly with my computers and car.

My laptop will most likely have some cool bi-fold or tri-fold screen, a 1THz processor, 40GB Ram, HD-DVD-RW that multi-boots in several OS's.

And, who knows why I will need it?

AIR: Adobe Integrated Runtime

AIR: Adobe Integrated Runtime

Some products have already pownced on this still-beta framework. And, I think with good reason. AIR can utilize any mix of HTML / JavaScript / Ajax / Flash / Flex development resources in order to deliver cross-OS desktop applications.

AIR has similarities to XULRunner. AIR is a runtime, like Java J2RE that needs to be installed onto the client's machine. The difference here is that while Mozilla and Sun may be household names to any geek with a computer, they still don't have quite the same name recognition that Adobe has with "normal" users due to their products (Flash and Acrobat). So the potential adoption curve of AIR is far greater.

AIR is beautiful. Flash content means superior anti-aliasing, and makes it far easier to animate your application. Even the installer is pretty.

If AIR gets even a quarter of the traction that Flash or Adobe gets, AIR is going to be very attractive, very fast. At that point, I see a lot of incentive for fat-client web applications to port their application if for no other reason than to eliminate the many, many pains of cross-browser support.

Good UI's are engineered.

I have crossed the designer-engineer border many times, and I have come to the conclusion that many people have the wrong mindset about "UI Design". Many relate the "design" closer to designer jeans, than they do to say designing a bridge. While it is the decoration that makes the finished product appealing, it is the structure that is most important.

This is how most UI Design processes go:

1. List out desired features
2. Sketch a few ideas
3. Mock until everyone agrees on something

The bitter truth is:

1. Features change
2. Usually your first instincts are wrong
3. Design by committee will give you a camel

UI Design is really no different than Network Design, Architectural Design, or any other form of structural design. You need to approach it with a set of given base parameters and develop a metric to measure all proposals against. A more proper procedure would be as follows:

1. Define the primary needs that the product provides
2. Script out the most common use cases
3. Draw flow charts to illustrate user paths
4. Define set of controls and display elements
5. Group related/parallel actions accordingly
6. Sketch out various options of all necessary screens
7. Measure if controls are present when wanted
   
8. Measure distance to action and number of clicks to complete action

Once screens, grouping and positioning is defined, it is now the time for decorations that will make actions intuitive and draw the right attention to the right items for each screen and to invoke the appropriate emotional responses from users. But, doing this earlier in the process often blinds you to other layout possibilities.

Now when you look at this second process, it may seem longer and more daunting. However, the truth is that when you skip over the initial scripting and mapping process, you loose the metrics by which you may measure the successfulness of all future design proposals. When you follow the complete design process, you actually can spend less time in mocking process, and have a more measurable certainty of success in the end.

Client MVC

A solid framework can make or break an evolving application. All to often, a simple application will grow too big for its britches and you are left with an organic mess. Lately, I am witnessing this the most with the re-emergence of fat-clients utilizing Ajax or similar technologies.

There is little in the way for JavaScript framework out there, with Google's Web Toolkit being one of the few that is widely used. JQuery/Interface + DOJO or SAJAX could also bring back some sanity to your project, but the lines between Model, View and Controller are still blurry and unmanaged.

I have found a couple good blogs that start to touch upon this problem, but no great projects that deal with the problem on a whole.

triad: a javascript mvc framework

Do RIAs Make You Rethink MVC?

And even beyond the all so common Ajax-backed fat clients, there are still other technologies in the client scripting world that need an MVC framework to work with. JSON, Flash, XAML, and XULE are just a few that come to mind.

Perl has Catalyst. Java has Struts. PHP has Mojavi. Python has Rails. But, JavaScript still lacks a flexible, multipurpose MVC framework that fits nearly any project. It seems the ideal candidate would be able to utilize existing toolkits interchangeably.

Avoiding 'Slow Script' Errors

So, you need to render 2,968 widgets to the DOM? Or, you really, really rather compute your data on the client, because the architectural geniuses you work with never thought you would want to cross-relate these two data sets, and it will be a load of work to re-architecture the back-end to handle this task? Well generally, my advice would be to avoid doing such tasks on the client, but that is not because it is not possible.

Sometimes when prototyping algorithms or cross-linked data presentations, I find myself pushing the computational barrier of JavaScript. Anyone who has done this, has managed to repeatedly trigger the 'Slow Script' warning asking the user to either stop or continue. These messages are easy to avoid with a little code modification.

function RenderDataSet(start) {
    var SEGMENT_SIZE = 100;
    var index = 0 + start;
    do {
        RenderData(index);
        index++;
    }
    while (index < dataset.length
        && index < start+SEGMENT_SIZE);

    if (index<DataSet.length)
        setTimeout("RenderDataSet("+index+");",1);
}

Since you are only allowed a few seconds of the thread's time to carry out your task, and there is no forking or spawning available for you to make new threads, you simply have to rely on setTimeout(expr,time). This releases the thread for a very small fraction of a second to allow other browser actions to occur. This is also a good practice to get into for real-time applications as well, since expensive tasks can slow down other features of your web application.

Crash IE6: With HTML!

Any serious client-side web developer has managed to crash a browser with some string of complex JavaScript, XSLT transform, or ActiveX Object. But what about with just some good old HTML?! Certainly a sixth-generation browser with several years on the market, such as Internet Explorer 6 could not be brought to its knees with some simple HTML. Well, guess again.

Ponder the HTML below and see if you can guess what evil it possesses:

<div>
<div><a name="input1"/><input name="input1"/></div>
<div><a name="input2"/><input name="input2"/></div>
</div>


I mean that looks perfectly safe and sane to me. Well, not to IE6. See, IE6 does not like self-closing anchor tags. But, instead of dropping the tag from the DOM, or adding an immediate </a> to correct the problem, it must do something completely different, because IE6 will crash when it stumbles upon this code.

I spent an hour of trial and error, searching for the culprit, before I isolated this code on one the web pages I was building. Not even for a split second did I suspect that it would be a bit of HTML causing IE6 to stop dead in the water.

AJAX: Asynchronous?

Asynchronous?

So AJAX stands for Asynchronous JavaScript and XML. As a technology, it sounds perfect for supporting client-server communications for fat-client web applications. But is it really? I mean, it is 'asynchronous' and it's HTTP, right? Well, not totally...

While the XMLHTTPRequest object does make an asynchronous call to the back-end server, these calls in themselves are not asynchronous on the TCP layer, only at thread-level are they asynchronous. Therefor, the calls themselves are synchronized and prone to being queued.

According to HTTP 1.1 specifications, only two connections are supported per process. Internet Explorer sticks to this limitation, while other browser such as Firefox support a higher connection limit by default. The means that requests get queued if there is more than the browser limit already active.

When requests queue up, long requests will block other requests, including images, javascript and css file calls. So, if you really want an asynchronous call system, AJAX is not the solution. But most likely a piece of the total solution.