Cross-browser testing is one of those aspects of web development that no one really likes, but everyone has to do. It’s gotten better, don’t get me wrong. As browsers have slowly crawled (some more slowly than others *cough*IE*cough*) towards achieving standards compliance, the differences between browsers has become far less drastic than it used to be.

Back in the day, it wasn’t uncommon for a website to look and work fine in one browser, and be utterly, horrifically broken in another. These days, if you’re coding to standards, your woes are more likely to be weird (but subtle) padding issues, some javascript issues, etc – not the complete and utter train wreck you’d have seen 10 years ago.

Planning and frequent testing will save your hide

Just because things are better now than they’ve ever been doesn’t mean we can get lazy. Cross-browser testing should be done often, and with gusto. One common pitfall with cross-browser testing is the temptation to leave testing until the very end, once the project has been completed. Problem is, if something is broken, you won’t necessarily have time to fix it, and depending on what’s wrong, the fix might take longer than you’ve allowed for.

You can avoid finding yourself in that kind of situation by testing more frequently throughout your development process. Whenever a large chunk of code or markup gets written, take the 5 minutes to check it on multiple browsers. By addressing these issues during these mini-milestones, you can take care of problems as they arise and avoid finding yourself in a bad situation the day before launch.

If you’re a small design shop, or an independent developer, you may not have access to the complete array of potential browser and OS combinations. I personally have a Mac laptop running Leopard, a Mac desktop running Tiger, four Windows XP machines, and a RedHat box set up, but even that doesn’t cover every possible combination – and you may not have as many machines at your disposal.

Fortunately, you’ve got some options, some better than others. The three primary ways you can test for cross-browser compatibility, short of actually running all of the browser and OS combinations on physical machines, are screenshot generators, browser emulators and virtual terminals. Each serves a slightly different purpose, and you may end up deciding that a combination of the three is most appropriate for your needs.

Screenshot Generators

Services like BrowserShots.Org and Litmus allow you to enter a url, click submit, and use their resources to generate multiple screenshots based on your selection of browser, setting and OS choices. Both have free and paid versions. (For the record, Snipe.Net looks like dog balls in IE4 for Windows… Hah!)

The free version of BrowserShots allows you to run a test on the full spectrum of browser/resolution/OS choices available, but your request will be lower priority than paying customers, so it can take a little while to get your results. Upgrading to a premium processing membership of $30 for one month bumps your job up higher in the queue for faster screenshots. Personally, I don’t think the improvement in speed is worth $30 a month, but you may disagree.

Litmus allows you to sign up for a free account which allows you 50 tests per month, but on only two browsers, both only on Windows, rendering the free version basically useless. If you don’t have access to IE7 and Firefox 2 somewhere, you probably shouldn’t be in this business. So the only useful way to use Litmus is through their paid version. They offer a few different packages, each with increasing features, starting from a day pass at $24, moving up to a monthly subscription of $49 a month for individuals and $199 a month for “team” membership. Once you get into the paid accounts, tests are done on 23 browsers, including Mac and Linux browsers. (A side note, I love how they say you get access to “all 23 browsers”, as if 23 is the final number.)

While their price structure is a bit steep and their browser/OS selection isn’t complete (no Firefox for Mac? Really?), it’s worth noting that Litmus offers HTML email testing across muiltiple email clients in their paid packages – something Browsershots doesn’t do. If you do a lot of email marketing for clients, this service may be worth it for that aspect alone.

Litmus also has a few value added features that may make it worth shelling out the money, depending on your needs.

  • Nicer design – more suitable to showing to clients directly if you don’t feel like downloading dozens of screenshots for each iteration of a site.
  • Ability to test password-protected areas of a site.
  • Automatically checks for validation errors in both HTML and CSS, with a link to the W3C validator pages – so if something doesn’t look right in the screenshots you get, you’ve got a shortcut to start figuring out why.
  • Version support, so each time you do a test, it creates a separate instance of that test as a new version for the site. If you test a page 6 times, you’ll see your current test and 5 historical versions in your control panel.

Me personally? I use the free BrowserShots service. They have more browser/OS combinations available and the wait time doesn’t bother me. I might use a paid Litmus account to test emails at some point though, as testing that manually can be a real pain in the ass.

Browser Emulators

The good news is, browser emulators are usually free. The bad news is that they are not available for all scenarios, and Macs seem to be at a real disadvantage here. Of course, Mac users can always use Parallels, WINE, or some other virtual OS that will allow you run Windows on your Mac, but that does require extra work.

If you usually develop on Firefox, the Firefox addon IE View can be very helpful. It simple provides a shortcut to an IE-engine rendered version of the page right in Firefox, that opens in a FF tab. Back when I worked on Windows, I loved this extension, but alas, it is for Windows only.

Also for Windows users, downloadable application DebugBar puts multiple versions of IE at your fingertips, including IE8 beta 2, IE7 IE 6 and IE5.5 on Vista and XP, as well as the installed IE. This one is handy because you can compare displays side by side, within one program.Β  I have gotten this to successfully install on my Parallels version of Windows on my Mac.

And the third one that comes to mind is Multiple IEs, a small downloadable application (again, Windows only, sorry) that lets you create separate instances of IE in many versions, going back as far as IE3. I managed to get this installed on Parallels, but it crashed every time I tried to use it. I had been using it for years on my native Windows computers though. I cannot speak to whether it works on Vista or Windows 7.

Virtual Terminals

Virtual terminals for cross-browser testing is a relatively new option for web developers, and brings something to the table that the other options cannot. Rather than simply seeing a picture of your site’s layout with screenshots, and rather than just being limited to IE as with the emulators, virtual terminals actually let you physically login to a machine running the OS and browser you’re looking to test. They are created using disk images, so when you logout, the image is destroyed and recreated for the next user.

What makes virtual terminals so unique and advantageous is that you are using the actual OS and browser you’re testing on. You can click around, test out your javascripts, login to protected areas and test functionality such as cookie behavior, and so on. What you end up with is a much more sophisticated test environment that results in a more genuine reflection of what the user will see and experience.

The funny thing is, with the pricey packages of the screenshot services, you’d probably expect to pay dearly for this much more advanced option, but in reality, they’re about the same as the screenshot services, with far more bang for your buck.

CrossBrowserTesting.Com even offers free access, and although the sessions are limited to 5 minutes and based on priority (paying customers come first if there’s a line), the free sessions are not limited in any other way. Their top-level paid package is still quite reasonable, at $29.95 for the first month and $19.95 each month after. For that top-level package, you get unlimited sessions (capped at a total of 40 hours of connected time), with a maximum of 30 minutes per session. They have a huge list of available OS and browser combinations, and for my money, this is the way to go. Check out their pricing structure here.

RealityBoxLabs offers a very similar service with a slightly different pricing structure. Unlike CrossBrowserTesting.Com, they offer a day pass for $5, and then and unlimited membership for $49 a month. They don’t specify which OS/browsers they offer, but they do mention that Linux will be available soon.

Which browsers to test?

The question of which browsers to test for is hotly debated in the web development community, but the realty is, there is no one single right answer. I can’t give you an answer, but here are some tips on coming up with the answer for your own situations.

  • Consider existing browser/OS stats for the site, but don’t limit yourself to that if you’re pushing for a large increase in traffic or reaching new demographics, since your browser/OS stats may change once you succeed.
  • Consider the browser/OS stats for the media outlets you may be pushing media from (such as Facebook and Facebook ads). If 90% of your traffic is going to come from a media outlet, make sure your site works on their top browsers.

My general rule of thumb, unless the stats indicate otherwise, is to develop and test on all of the major browsers, while still maintaining some level of normalcy for IE6 users. Depending on the project (and the client), I may have to work particularly hard to make sure the display on IE6 looks exactly the same as it does on current browsers, but if that’s not a requirement, I make sure it looks good on IE6. There may be some padding that isn’t quite right, but only the designer and the client would even notice.

For personal projects, it really depends on the audience. Snipe.Net has a user base of over 50% FireFox users, with the remaining large chunk being IE7 users. I made the decision to sacrifice some minor display points on IE in order to optimize for newer browsers. I am never so cavalier with paying clients.

Developing cross-browser compatible sites is easier than ever, both because of improved adherance to sandards, and because of advancing testing technology. Whether you choose to do it the old fasioned way, hopping from one computer to another hitting “refresh”, or whether you use some of these new-fangled, high-tech methods, it is your responsibility to your clients – and to their users – to make sure it works.

Advertisement

468x60_makemoney
ssd-virtual-servers-banner-468x60
heatmap
Previous post

Heatmapping and Better Conversion Rates

pc-gun
Next post

Handy Mac OSX Command Line Tricks

snipe

snipe

I’m a tech geek/dev/infosec-nerd/scuba diver/blacksmith/sword-fighter/crime fighter/ENTP/warcrafter/activist. I'm the CTO at Mass Mosaic and the CEO of Grokability, Inc. in San Diego, CA. Tweet at me @snipeyhead or read more...

  • I prefer the virtual machines solution (Parallels, as you’ve mentioned, as well as VirtualBox and VMware Fusion, are all options for Intel-based Macs; or Virtual PC for PowerPC-based Macs) to any of the others. It does, as you say, require extra work, but I find that if you set up a virtual machine for each browser setup you want to test, that work is one-off, and is in my opinion worth it if you have a lot of sites to test, and/or need to do testing on a regular basis (as any serious web developer should).

    The main benefit of this approach is that you can run all the stuff over your local machine, and thus get a much more realistic idea of the performance aspects of your page (especially if it’s Ajax-heavy). Also, with the setup set up, you can then set up checkpoints (aka snapshots) of “pristine” setups, and you can set up your virtual machine to auto-revert upon shutdown, making regular testing of websites really easy.

    But, like you say, setting up the machines in the first place does take up time, and you also require a lot of disk space. But if you have tons of disk space to burn (as I do), it’s probably the most “authentic” way to test different browser setups, short of actually buying separate machines for the job.

    YMMV. πŸ™‚

  • I prefer the virtual machines solution (Parallels, as you’ve mentioned, as well as VirtualBox and VMware Fusion, are all options for Intel-based Macs; or Virtual PC for PowerPC-based Macs) to any of the others. It does, as you say, require extra work, but I find that if you set up a virtual machine for each browser setup you want to test, that work is one-off, and is in my opinion worth it if you have a lot of sites to test, and/or need to do testing on a regular basis (as any serious web developer should).

    The main benefit of this approach is that you can run all the stuff over your local machine, and thus get a much more realistic idea of the performance aspects of your page (especially if it’s Ajax-heavy). Also, with the setup set up, you can then set up checkpoints (aka snapshots) of “pristine” setups, and you can set up your virtual machine to auto-revert upon shutdown, making regular testing of websites really easy.

    But, like you say, setting up the machines in the first place does take up time, and you also require a lot of disk space. But if you have tons of disk space to burn (as I do), it’s probably the most “authentic” way to test different browser setups, short of actually buying separate machines for the job.

    YMMV. πŸ™‚

  • Hi Chris – your setup sounds great for an individual developer on a Mac – but if you’re on a PC, or if you work in an office where several people are going to be testing, setting up that many instances can be more trouble than it’s worth. Other than whatever lag-time the VNC/Remote Desktop client may cause, you get an entirely accurate picture using the virtual terminals – they’re basically the same thing you’re describing that you have, without the hassle of setting up 20 different virtual machines. For me, the virtual terminal services are a way better way to go. I have space on my laptop, but I do some video editing on it as well, so I don’t always have that much space to spare – plus I’d rather login to a service, do what I need to do and logout and be done with it, rather than having all that extra crap sitting on my hard drive πŸ™‚

    That said, its awesome that you’ve got a system that works for you. And you know I always appreciate your feedback πŸ™‚

    • That’s true, I understand that setting up lots of virtual machines is not a solution for everyone. If you have a fast enough Internet connection to the virtual terminal service providers, then that is a very workable solution indeed, I agree. πŸ™‚ (Do they have any service guarantees about how many other users will also be using their server concurrently? One user’s load may affect another user’s testing performance.)

      Here in New Zealand, Internet speeds are really slow and laggy, so it’s unfortunately not a viable option. But once we move to the US, I look forward to having more options in this regard! πŸ™‚

  • Hi Chris – your setup sounds great for an individual developer on a Mac – but if you’re on a PC, or if you work in an office where several people are going to be testing, setting up that many instances can be more trouble than it’s worth. Other than whatever lag-time the VNC/Remote Desktop client may cause, you get an entirely accurate picture using the virtual terminals – they’re basically the same thing you’re describing that you have, without the hassle of setting up 20 different virtual machines. For me, the virtual terminal services are a way better way to go. I have space on my laptop, but I do some video editing on it as well, so I don’t always have that much space to spare – plus I’d rather login to a service, do what I need to do and logout and be done with it, rather than having all that extra crap sitting on my hard drive πŸ™‚

    That said, its awesome that you’ve got a system that works for you. And you know I always appreciate your feedback πŸ™‚

    • That’s true, I understand that setting up lots of virtual machines is not a solution for everyone. If you have a fast enough Internet connection to the virtual terminal service providers, then that is a very workable solution indeed, I agree. πŸ™‚ (Do they have any service guarantees about how many other users will also be using their server concurrently? One user’s load may affect another user’s testing performance.)

      Here in New Zealand, Internet speeds are really slow and laggy, so it’s unfortunately not a viable option. But once we move to the US, I look forward to having more options in this regard! πŸ™‚

  • I forgot to mention one other thing, which is that if you’re willing to fork out some money on a virtualisation solution, VMware Workstation has a “teams” feature where you can set up all your testing virtual machines within the same LAN segment. There are two purposes to this:

    1. You can set up the LAN segment to have latency built in (i.e., the LAN segment can be set up to slow down connections to, say, dialup speed, and/or drop x% of packets, to simulate real-life Internet connections). You can then really tell if there is some image or script that needs to be shrunk/minified.

    This assumes (as 2b below) that pages are served from a web server on the LAN segment, rather than being local files on the testing machines.

    2a. You can set up the LAN segment not to have any direct Internet access, which will give you an idea of whether you’re unintentionally linking some image/stylesheet/whatever from an external site. So it’s a really quick way to find broken internal links, etc.

    2b. On a related note, you will very quickly catch all the broken src=”C:Documents and Settings…” references (which seems to be a problem with some sites, unfortunately), especially if your virtual machine does not replicate the same folder structure as the web design machines.

    I hope this helps readers here. πŸ™‚

    I will finish by saying that I’ve been a paying customer of VMware Workstation for nearly 5 years, and I’m very satisfied with the product. I have no affiliation with VMware, and am not employed by them, nor do I have any personal benefit from promoting the product, so this is just my plain honest opinion. πŸ™‚

  • I forgot to mention one other thing, which is that if you’re willing to fork out some money on a virtualisation solution, VMware Workstation has a “teams” feature where you can set up all your testing virtual machines within the same LAN segment. There are two purposes to this:

    1. You can set up the LAN segment to have latency built in (i.e., the LAN segment can be set up to slow down connections to, say, dialup speed, and/or drop x% of packets, to simulate real-life Internet connections). You can then really tell if there is some image or script that needs to be shrunk/minified.

    This assumes (as 2b below) that pages are served from a web server on the LAN segment, rather than being local files on the testing machines.

    2a. You can set up the LAN segment not to have any direct Internet access, which will give you an idea of whether you’re unintentionally linking some image/stylesheet/whatever from an external site. So it’s a really quick way to find broken internal links, etc.

    2b. On a related note, you will very quickly catch all the broken src=”C:\Documents and Settings\…” references (which seems to be a problem with some sites, unfortunately), especially if your virtual machine does not replicate the same folder structure as the web design machines.

    I hope this helps readers here. πŸ™‚

    I will finish by saying that I’ve been a paying customer of VMware Workstation for nearly 5 years, and I’m very satisfied with the product. I have no affiliation with VMware, and am not employed by them, nor do I have any personal benefit from promoting the product, so this is just my plain honest opinion. πŸ™‚

  • Excellent info – thanks Chris! I didn’t know about the teams options – very cool!

  • Excellent info – thanks Chris! I didn’t know about the teams options – very cool!

  • great post, just tested my site. Works well in all apart from ie.6. No surprises there !

  • great post, just tested my site. Works well in all apart from ie.6. No surprises there !

  • Thanks for the mention and positive comment for CrossBrowserTesting!
    Much appreciated!
    Ken (one of the 3 founders)

    p.s. The link to the ‘huge list of available OS and browser combinations’ is pointing to the wrong place – I believe it should be http://www.crossbrowsertesting.com/configurations.php

  • Thanks for the mention and positive comment for CrossBrowserTesting!
    Much appreciated!
    Ken (one of the 3 founders)

    p.s. The link to the ‘huge list of available OS and browser combinations’ is pointing to the wrong place – I believe it should be http://www.crossbrowsertesting.com/configurations.php

  • Thanks for sharing this excellent resource (BrowserShots.Org). It sure can be useful for my programmers.

  • Thanks for sharing this excellent resource (BrowserShots.Org). It sure can be useful for my programmers.

  • You may want to take a look at this new fast cross browser screenshot service

    Currently it only supports Firefox, Internet Explorer and Safari on Windows, but as it is being actively developed more features and additional browsers will be added in the future.

  • You may want to take a look at this new fast cross browser screenshot service

    Currently it only supports Firefox, Internet Explorer and Safari on Windows, but as it is being actively developed more features and additional browsers will be added in the future.

  • Bird

    I looked at crossbrowsertesting.com, but in price comparison, I can't see the benefit. I can either install VMWare/Parallels for $80 and use that for the next two years, or I can pay $19/month over two years ($456) for crossbrowsertesting.com….

  • The benefit is that you have 25 or so different combinations of browsers and OS's. I run VMware to check modern Windows browsers, but if I want to check older versions of IE, Firefox, etc, it's not practical to use only VMware. Lots of our clents still run IE6.