Review Of Cross-Browser Testing Tools

Tuesday, August 9, 2011

At some point in the future, the way that all major browsers render Web code will likely be standardized, which will make testing across multiple browsers no longer necessary as long as the website is coded according to Web standards. But because that day is still a way off (if it will really come at all), testing your design the advanced browsers as well as legacy browsers is a necessary part of any project.

The old-school way to test code was to load your website on as many computers as you could find, using as many different combinations of browsers and operating systems as possible. That was fine if you had access to a bunch of different computers (and had some time to kill). But there are much more efficient ways to test across browsers, using either free or commercial Web services and software. In this article we review some of the most useful ones.

Free Cross-Browser Testing

Good news: very powerful free testing tools are available for Web designers today. Some are more user-friendly than others, and some have significantly better user interfaces. Don’t expect much (if any) support with these tools. But if you’d rather not spend extra money on testing, some great options are here as well.

Adobe BrowserLab

Adobe BrowserLab is a free cross-browser compatibility tool that lets you test a number of modern and legacy browsers, including various versions of Chrome, Safari, IE and Firefox. It gives you a number of ways to view pages, including a full-page view in a single browser, as well as side-by-side comparisons of browsers and an onion skin view. The service can access dynamic pages across the web, or viewed locally via Firebug or Adobe Dreamweaver CS5. The ability to create pre-defined browser sets is also useful, in case you don’t need to test on older browsers.

Browsershots

Browsershots is probably the most comprehensive free testing tool available. It includes Linux, Windows and BSD browsers. It also includes a number of browsers you’ve probably never heard of (like Galeon, Iceape, Kazehakase and Epiphany). For the most part, Browsershots tests on the most recent version of each browser, as well as on legacy versions.

While Browsershots does support a huge variety of browsers, the more you test, the more slowly it prepares the results. So, you may want to stick to the major browsers.

SuperPreview (Free and Commercial)

SuperPreview is Microsoft’s offering in this space (and it’s compatible only with Windows). It lets you define your own “baseline” (or default) browser, and it works with any browser installed on your system (and comes with the IE6 rendering engine built in). The fact that it only works with your built-in browsers does make it faster (because you’re not uploading anything or waiting for a remote server), but it also limits the number of browsers you can compare.

SuperPreview trial comes with 60 days of cloud services before you have to either buy it or go into reduced, (local browsers and IE 6-9 mode). In an online version, you have support for Chrome, Safari (Mac) 4+5, Firefox 3+4. You can also use an interactive mode to log into sites that require a login before displaying the page you want to test. There are also debugging tools for the DOM and onion skinning available in Adobe Browserlabs. Unfortunately, there is no support for Opera whether installed locally or in the cloud and you do have to have the version included with Expression Web to get the cloud services option but the base version with support for IE 6, IE 7, IE 8 (and IE 8 rendering as IE 7) are included with the free version as well as IE 9 if it is installed locally. (Thanks, Cheryl D Wise)

Lunascape 6

Lunascape is a triple-engine browser for Windows. It runs Trident (IE), Gecko (Firefox) and Webkit (Chrome and Safari), so that you can see how your website looks in all three, side by side. While it’s not a traditional browser compatibility tester, it is nonetheless a useful tool for designers and developers. One major benefit is that you get to view your website instantly in all three major rendering engines. There’s also support for Firefox extensions and plug-ins, so you can use developer tools like Firebug to diagnose compatibility problems.

IETester

IETester is a free (both for personal and professional usage) browser for Windows that allows you to have the rendering and JavaScript engines of IE10 preview, IE9, IE8, IE7, IE6 and IE5.5 on Windows 7, Vista and XP, as well as the installed IE. Only an alpha version of the tool is available. Windows 7, Windows Vista or Windows XP with IE7 minimum are required for the tool to run.

IE NetRenderer

IE NetRenderer lets you check compatibility in Internet Explorer versions 5.5 through 9. You’ll have to check each version individually, but the service is free.

Spoon

Spoon is an application emulation service. It provides free versions of Firefox, Chrome, Opera and Safari for Windows users. A number of versions of each browser are included: Firefox 2–5, Chrome 4–8, Safari 3–5 and Opera 9–10. Bad news: Internet Explorer is supported by Spoon virtualization but is not available by request of Microsoft.

Sauce Labs (free and commercial)

Sauce Labs provides a lot of browser and OS options and sets you up with a browser dedicated VM instance that you operate inside the browser of your choice. It also records a video of your entire testing session. The service offers 200 free minutes of testing per month and allows you to quickly build automated tests from your browser with Selenium.

Browsera (free and commercial)

Browsera provides automated compatibility testing. It automatically highlights differences in the way browsers render your design, thus simplifying the testing process. It also detects JavaScript errors, and the commercial version can test pages behind subscription or log-in walls. It can also test dynamic pages.

The free plan includes a limited number of browsers and low-resolution screenshots. Premium plans start at $39 for a single project and $49 to $99 for monthly subscriptions, and they support more browsers, provide high-resolution screenshots and let you test private pages.

Browserling (free and commercial)

Browserling is a relatively new cross-browser testing app. It supports a limited number of browsers (and not necessarily the newest versions), which makes it of limited use to some developers. It’s still in beta, though, so hopefully more browsers will be supported in the near future.

The free version comes with a five-minute session limit, and the developer version is $20 per month with no time limit.

Commercial Cross-Browser Testing

Commercial tools often have features not found in the free ones, including live interactive browser virtualization and mobile device testing.

Mogotest

Mogotest does complete browser-compatibility testing, including for private pages. There’s an API, so it can be integrated in your current tools and workflow. Mogotest also offers a website health report that tells you about broken links and pages, redirect loops and other issues common to new websites. The service also offer screenshot comparison tools for testing screenshots against each other as well as site-level testing including page consistency testing and individual page tests. HTTP basic and cookie-based login systems are supported as well.

There are two plans for individuals: a personal plan starting at $15 per month that lets you test up to 50 pages on three websites, and a freelancer plan for $45 per month that includes up to 10 websites and 350 pages. The team plans start at $125 per month and go up to $4499+ for unlimited access. The two highest-cost plans include custom reports.

Cloud Testing

Cloud Testing offers functional cross-browser testing. You record the user journey with your browser and Selenium IDE, upload it, and then Cloud Testing will run that script in multiple operating systems and browsers. It then provides screenshots and HTML and component diagnostics. No prices are listed on its website.

BrowserCam

BrowserCam includes testing tools for both desktop and mobile browser compatibility (the latter is still absent in many other tools). It also offers remote access for live testing on Windows, Linux and OS X configurations, and email capture for checking your HTML, RTF and TXT emails.

Pricing for BrowserCam starts at only $19.95 per day for a single service (and $24.95 for the browser, remote access and email capture package), up to an annual subscription price of $399.95 for a single service (and $499.95 for browser capture, remote access, email capture and multi-user access, or $999.95 for all of those features plus device capture).

Multi-Browser Viewer

Multi-Browser Viewer covers both desktop and mobile browsers. It includes 26 virtualized Web browsers, 5 mobile browsers (including the iPhone and iPad) and 61 screenshot browsers (meaning you can see how the website renders but not interact with it). It’s also available in five languages: English, Spanish, German, Russian and French.

Multi-Browser Viewer is $139.95 for a single-user license and includes a year of product usage and updates. Updates after the first year are currently $99.95. A free trial is available through the website.

CrossBrowserTesting

CrossBrowserTesting provides live interactive browser testing with remote VNC sessions. It also generates automated screenshots across multiple browsers for more basic testing. There are more than 100 browser and operating system combinations, including many mobile platforms.

Monthly subscriptions range from $29.95 to $199.95, depending on the number of users and the minutes of testing (minutes can roll over to the next month, but they’re not unlimited). A one-week free trial is available for all plans.

Testing Services Compared

The chart below shows the basic features offered by these cross-browser testing services and applications, making it quick and easy to compare.

Tool	Number of browser versions supported	IE?	Interactive testing?	Side-by-side testing?	Pricing
Adobe BrowserLab	13	IE6+	No	Yes	Free
Browsershots	60+	IE6+	No	No	Free
SuperPreview	Varies	IE6+	Yes	Yes	Free
Lunascape	3	IE6+	Yes	Yes	Free
IETester	6 versions of IE	IE5.5+	Yes	Yes	Free
IE NetRenderer	5 versions of IE	IE5.5+	No	No	Free
Spoon	16+	no IE	Yes	No	Free
Sauce Labs	40+	IE6+	Yes	No	Free — $499 per month
Browsera	9	IE6+	No	Yes	Free – $99/month
Browserling	9	IE5.5+	No	No	Free – $20/month
Mogotest	7+	IE6+	No	Yes	$15 – $4,499/month
Cloud Testing	4+	IE6+	Yes	Yes	Not specified
BrowserCam	90+	IE5.2+	Yes	Yes	$19.95 – $89.95/month
Multi-Browser Viewer	80+	IE6+	For some browsers	Yes	$139.95
CrossBrowserTesting	100+	IE6+	Yes	Yes	$29.95 – $199.95/month

Conclusion

Regardless of the tool you choose, testing early and often during the Web development process can save you from a lot of headaches later. Find a tool that fits your workflow (so that you’ll actually want to use it and it won’t be a hassle), and test whenever you make major changes to a design.

What tools do you use for cross-browser testing?

How has your experience been with cross-browser testing tools and services? Which ones do you use? How do you integrate cross-browser testing in your professional workflow? Let us know in the comments!