Moving this blog

This is my last post using Blogger. I have moved my blog to TypePad to give it new life and a new focus.

GO TO MY NEW BLOG

New Focus

When I started this blog, I gave it the subtitle of "Exploring the technical, sociological and economical foundry blocks of the World Wide Web.". I found that most of my post hovered around the technological concepts of RIA and the web as a platform. Probably because I was working mostly as a software engineer at the time. As I have been shifting more to the user experience, interaction design and product management, the more I became dissatisfied in the details of the technology. I have not only the sociological issues more compelling, but the psychological and particularly cognitive issues of web as a platform more and more important to where the web is going, and what is in it's future.

So, I am shifting this blog to "Exploring the psychological, economical and technical structure of the Internet." I am preparing a flurry of blogs, and plan to start posting much more readily in the very near future. I may end up moving this blog, and if so, my next post may be my last on blogger.

Why CSS's CLIP is your friend

When you get to the point on a project where page load-time becomes a concern, there is a usual play book that most of us pull out.

1. Glob your CSS and JavaScript into as few files as possible.


2. Pack or minify your JavaScript.


3. Trim whitespace in your CSS.


4. Start pre-loading images for roll-over effects.

Maybe you go so far as to lazy-load content via JavaScript. However, I will still do you one better...

Glob your static images onto one image map. Place all of your icons, or all of your tabs and their rollover counterparts into one larger image. Then, in the css of each image element, add a Clip definition to the CSS or style attribute.

What are the benefits?

In this case, the whole is less than the parts. This means smaller file size, but more importantly, it means far less packets. Each file is at least three packets on the wire. So, with lots of small files, such as a dozen or so icons, you can reduce the number of packets on the line by as much as two-thirds.

The second nicety of using clip is that mouse-overs are really smooth when you just shift the clip's rect rather than swapping out images.

There is another cool side-effect. If your site is slow loading for any reason, (i.e. client's machine is bogged) images will load in neat batches, rather than trickling in. This tends to actually make your site seem like it is less broken. Many users are more often annoyed by that one image that has not loaded, than when the whole lot is still loading. It is just distracting. Pay attention next time, I bet you will agree.

What are the pitfalls?

There are only two. First, clip: rect(); can be a pain to specify. You almost never get the math right the first time. (x,y,w,h) would have been a far better specification. Second, you need to pay close attention to what images you group together. Do not group together an image that should be jpeg with an image that should be a gif, and when grouping gifs, pay very close attention to the color pallet and make sure you are grouping like-images and not sacrificing too much picture quality.

That said, this trick is one that really should make into a lot of developers performance play books.

Opening up to OpenSocial, OpenID, hCard and oAuth

So, there is a flood of propaganda coming out of the web 2.0 conferences either about OpenSocial, OpenID, oAuth or hCard. And many people seem to be smoking the peace pipe and passing it along. Sorry, but I don't smoke, and I don't read the tabloids. Usually, I like to wait until a technology is bound and in print, before I am ready to commit. Sorry, but it is an old guy's habit after seeing too many idealogical fads hit the Internet.

However, a co-worker said something today that actually made sense to me. And it was not about only having one login to remember, or how cool it was to link all your accounts together, or any other feature set that has been tried before. It was the way he related the registration process to a check out process, and indirectly related the registration information to credit card information.

...Okay, so you probably do not get why this excited me enough to write a blog about it, but read on...

For the last few weeks I have been doing a lot of reading on usability, design, and marketing. And the only thing that I think every time someone mentions these technologies is "So great. You want me to both convince users to register with my site, and do so using a process they are unfamiliar with?" What if my user does not have a GMail or Blogger account? How are you not setting me up to just confuse a lot of people? I mean, how do you expect to sell this to non-geeks? The n00bs? My mom? (Yes, my mom is my favorite test-case for usability. If she gets it, everyone does.)

So, the concept of these services as personal information brokers in the same way that Visa and Mastercard are brokers of my credit card information is powerful. If registration is a checkout, your login is your credit card information, and sign-in is like Amazon's 1-click purchase. This ability to relate these systems to a very well established user interaction might just give this whole thing some credit. I can now see a small sliver of light that suggests this stuff might make it to hard copy, might be around in three years, and might be worth my time considering.

To the many behind OpenSocial, OpenID, hCard, oAuth, and whatever else is out there, my suggestion is this: Stop selling this stuff to me, the web 2.0 geek, and start figuring out how you are going to sell it to my mom. You do that, and you will have a recipe for success. So, please go fire up your branding iron and get to work, because you have long chasm to cross if you really want this to catch on.

GUIDs, UUIDs, and Canonical Names in JavaScript

Creating unique identifiers outside of a database tends to be a bit of a pain. So, why would you do it? What if you were not using a database? What if you have distributed servers? What if you want clients to be able create distinguishable content without constantly contacting the server? For example, if clients are aggressively creating parallel content, then it would be advantageous for you to be able to lazily sync the content without causing significant lag and interruption to the user. I realize that most of you will not run into this, but for those of you who do, here are some helpful notes:

Firstly, I can't tell you how many postings I found that led to tutorials using an ActiveX object to create a UUID. If you have only IE-using clients that do not have stringent security settings, then I guess you can use this. Otherwise, it is no good.

I soon found This UUID library from AF-Design, which is alright, but there are obvious inefficiencies, such as the way it computes time in milliseconds out the long way instead of just using Date.getTime(). Also, a serious design flaw is that it doesn't use any unique data other than time in milliseconds to generate the UUID. To be a true UUID, it would need to utilize unique data to the machine such as IP, or even better the MAC address. Otherwise two machines could potentially generate the same UUID.

Understandably, there is no method for fetching the mac or ip address using only JavaScript within a browser. But, you can make the client's IP available to the JavaScript running on the client browser by echo-ing back the REMOTE_ADDR from a request header back to the client. For an example of how to do this in PHP, see http://unix.cms.gre.ac.uk/code/php/examples/remote_addr.php.

Since I usually do not care to have my unique id's in the traditional UUID format, I tend to go with a canonical name using an MD5 hash of the time and IP. The result is more reliable than the UUID example above, and is actually a bit faster and easy to implement:

function cn() {
    var t = (new Date()).getTime();
    var ip = '192.168.1.100';
    return MD5(t+ip);
}

I hope this helps any of you dealing with a similar design problem.

Zero-width whitespace and what it can do for you...

One of the most common layout problems I see in the web 2.0 space is accounting for the odd shaped content some people post. Long URLs are among the most common nuisance. Take for example:

http://www.google.com/search?hl=en&client=firefox-a&rls=org.mozilla%3Aen-US%3Aofficial&hs=G12&q=zwsp+zero-width+space&btnG=Search

A long URL like this often overflows its container and is a layout nightmare. Most people's response is to simply set an overflow value, and either hide the remainder, or have a horizontal scrollbar. Neither of these solutions are ideal. Either way you are obscuring content, and I think everyone agrees that, in a web layout, horizontal scrollbars are the devil.

The solution to this problem is Unicode character U+200C. (a.k.a. ​ or &zwsp;) This character achieves the exact opposite of the common non-breaking space ( ). Zero-width space does not render a visible space to the screen, but acts like a space for breaking up text when wrapping. Here is a javascript example of how to use it:

function linkify(text) {
  var pre = text.substr(0,text.indexOf("http://"));
  var rest = text.substr(text.indexOf("http://"));
  var url = rest.split(/\s/,2)[0];
  rest = rest.split(/\s/,2)[1];
  var link = '<a href="'+url+'">';
  link += url.replace(/(\/|\+|=|\-)/g,"$1&#8203;");
  return pre+link+'</a>'+((rest)?rest:'');
}

For the URL above, this would render the following html:

http:/​/​www.google.com/​search?hl=​en&client=​firefox-​a&rls=​org.mozilla%3Aen-​US%3Aofficial&hs=​G12&q=​zwsp+​zero-​width+​space&btnG=​Search

Now this link nicely wraps to fit inside its container, and one of your worst layout nightmares is easily solved. I don't think it takes much creativity to figure out the other potential applications for this character.

I cannot tell you what a pain it was to make sure every &, <, and > symbol in this post was properly escaped, so I hope you found this helpful.

It is time to take the WWW behind the shed and put it out of its misery.

Almost everything that makes up the WWW is a hack. HTML, HTTP, SMTP, are all antiquated technologies. These old technologies were engineered with the sole intention of delivering and sharing text documents. Early web browsers and e-mail clients shared this same basic vision in the beginning. Even today, most media requires plug-ins and other technologies in order to be shared over the web. Where is the problem? It lies in the expectation in the end-users of the Internet. Over the last several years, the major players on the Internet are not delivering documents, but services and applications, which infer an expectation of user experience.

The expectation of a controlled user experience is not a reasonable expectation of the Internet. The Internet by definition does not have any of the building blocks for a true user interface. Cookies are a terrible way to manage state, and HTTP+HTML has no traditional support for static or persistent elements. To go further, the standing trend over the last several years has been to deliver web-based applications. The complexity of these applications has been growing over the years to include two-way communication, streaming data, persistent state, and all of the other elements that traditional computer applications contain. All trying to be consistent across dozens of different browser/OS combinations with different connection rates and privacy and security restrictions.

HTML Limitations

• Inconsistent across different platforms
• Not intended to be used as a UI framework
• Poor state control
• No built-in multimedia control

HTTP Limitations

• 2 connection limit per DNS host name
• Does not support Keep-Alive

The worst part of this combination is that the services that rely upon these technologies to deliver their service have little or no control upon how the two interact, because their service is a slave to whichever web browser the client uses to access them.

Standing Hacks

• Frames
• DHTML and DOM
  
• Plug-ins and ActiveX objects
• Java Applets
• XMLHttpRequest object

Java Swing

Java's Swing framework was probably the first reasonable attempt of a resolution. Its failures and shortcomings were few. Java is too heavy of a language for a UI framework. It isn't powerful enough to be fast and efficient. Java Runtime Environment has too much access to the client's machine, and not enough security for most people. On top of this, Swing's anti-aliasing and 2D drawing is less than attractive.

XULRunner

From those wonderful guys at Mozilla, XULRunner utilizes XML and JavaScript and runs within the Gecko Runtime Environment. It is still young, but shows great promise.

Adobe AIR

Adobe's Integrated Runtime, which was previously named Apollo, shows great promise. Not only does it utilize Flash and Flex, but also JavaScript and HTML. There are quite a few Open Source Flash initiatives such as MSASC and SwfMill for compiling ActionScript and XML into SWF files. Also, ActionScript has matured nicely over the years. If Adobe open-sourced an ActionScript compiler, this may just be the winning platform for the best way to build and distribute your applications.

While HTTP and HTML will most likely remain the foundation blocks of the Internet, I have great faith that many of the services out there with web-delivered applications will be looking towards other technologies such as AIR or XULRunner in order to better control the user experience and deliver richer applications.

Update:
This blog post criticizes web browsers failure to utilize multi-threading in multi-core systems. It is a valuable point and felt it worth adding here. http://pinderkent.blogsavy.com/archives/147

But, What Are We Really Building?

On every major web project that I have ever worked on, there comes that point where I find myself asking 'But, what am I really building?'. Often, the project has some idealistic mission statement about 'Providing people with X and interested in Y a way to connect with Z.' Often, Y=Z or X=Y and the mission statement is all the simpler. However, once the product is actually unleashed on the real world, only 1/10th of the people have X or interested in Y or even care about connecting with Z. The other 90% are interested in people with X or who are interested in Y, and want to connect them with P.

Reverse Your Mission Statement

What triggered me to write this blog entry was the comical, yet sadly true Cracked.com article
Internet Safety Tips. So, my point is this: While mission statements define the market and give support to the projects scope, your product will be ten times as successful if you also approach the mission statement in reverse. How do I allow only Z to connect with people with X who are most likely to be interested in Y. This is particularly in products that contain any part 'social networking' in their mixture of features.

Case in Point

Myspace.com has started to loose substantial ground for the first time in many months according to Google Trends for Myspace. Shortly after 29,000 registered sex offenders were found on Myspace, there has been a noticeable decline in activity and new membership. While some good PR and new features should be able to turn this trend around, you have to ask in hindsight, what could have been done differently to avoid this?

So, to all of you who are out there building your next great web product, please, please, please take the time to say, how do I keep people with X interested in Y from being targets of people attempting to connect them with P. Otherwise, do not be surprised when your product is shanghaied by the porn industry, illegal gambling, or other predators who can probably turn out a higher profit from your users than you can with their lack of moral fiber.

10 Years...

About a week ago, I got another year older. As is typical, you find yourself reflecting from time to time about the past year. Though, while driving home one evening, I instead had a flashback to 10 years ago....

10 Years Ago

I drove an '89 Plymouth Acclaim. It had a cassette player, cloth seats, and windows that you had to roll down. I had a cd player with a tape adapter, but it never worked right with the car radio. My cellular phone had a 10-digit lcd screen and could easily fit in my briefcase and I can even connect to my ISP at about 14.4k using it. My laptop was a Compaq Pentium 100MHz with 16MB RAM and a 4X CD-ROM drive that ran Windows 95. In those days, I was building websites using HTML 4.0, VBScript, JavaScript, Perl/CGI, Server-Side Includes, and was just learning about Allaire's ColdFusion and HomeSite.

Today

I drive a '99 Jeep Grand Cherokee with powered leather seats and windows. It has an SCD player that can also play MP3's, WAV and WMA files off a CD-RW, and has an animated interactive LCD screen that shows me song titles and also heads my CD Changer, and my XM Satellite receiver.

... That's right, I said "satellite"! OK, that really is not impressive, but if I said that 10 years ago, you would not have believed me...

My cellphone today has TWO COLOR lcd screens, and I can use it to connect to the internet as fast as 1.4Mbps. My laptop now is an IBM Thinkpad 2GHz with 2GB RAM and 16x DVD-ROM/CD-RW running Windows XP SP2 and Ubuntu. And, I could not even begin to list all the tools I use these days without forgetting something.


Ten Years From Now

I guess I will be driving an '09 American Something that has power doors, remote climate control, GPS, complete media center that plays DVD's and can stream other media from my home server.

My cell phone will be more of a portable access device that also streams multimedia, surfs the web, has a 20Mbps connection in some places and can interact seamlessly with my computers and car.

My laptop will most likely have some cool bi-fold or tri-fold screen, a 1THz processor, 40GB Ram, HD-DVD-RW that multi-boots in several OS's.

And, who knows why I will need it?

AIR: Adobe Integrated Runtime

AIR: Adobe Integrated Runtime

Some products have already pownced on this still-beta framework. And, I think with good reason. AIR can utilize any mix of HTML / JavaScript / Ajax / Flash / Flex development resources in order to deliver cross-OS desktop applications.

AIR has similarities to XULRunner. AIR is a runtime, like Java J2RE that needs to be installed onto the client's machine. The difference here is that while Mozilla and Sun may be household names to any geek with a computer, they still don't have quite the same name recognition that Adobe has with "normal" users due to their products (Flash and Acrobat). So the potential adoption curve of AIR is far greater.

AIR is beautiful. Flash content means superior anti-aliasing, and makes it far easier to animate your application. Even the installer is pretty.

If AIR gets even a quarter of the traction that Flash or Adobe gets, AIR is going to be very attractive, very fast. At that point, I see a lot of incentive for fat-client web applications to port their application if for no other reason than to eliminate the many, many pains of cross-browser support.