eBay Tech Blog

Many web performance testing service vendors, such as Keynote, offer last-mile testing data. In addition, many popular tools, such as WebPagetest, provide a convenient way to submit test cases so that you can collect large volumes of performance metrics. However, most of these services and tools are synthetic, meaning that they are not from real users. The testing agents in a computer performs the tests in a controlled environment.

The real world can be very different. A great number of parameters can affect the data dramatically. We can easily name a few such parameters: network connection speed, last mile speed in particular, browser type, computer CPU power. More importantly, the distribution of parameter values affects the results substantially. For example, we can get DSL and high-speed backbone test results from Keynote for a particular type of browser, but we don’t know exactly how many users have a network speed that’s comparable to the testing network’s speed. Other types of connection speeds that exist in the real world can also be missed in the tests. It’s difficult to put a weight on each of the tests and get an average that reflects the real world.

Several years back, at eBay we began building a framework called Site Speed Gauge to capture real-world, end-user site speed data. Site Speed Gauge has proven to be an effective way to understand real user experience with web performance, while also correlating site speed and the conversion of traffic to purchases. This framework is now the gold standard at eBay for monitoring site speed. Compared to synthetic tests with at most several thousand data points per day for a site page, we are getting millions of data samples for the same page. Large, statistically significant data sampling points help us to get stable trending and to identify real issues. This post provides an overview of how we use Site Speed Gauge to monitor and improve site performance as well as to collect meaningful business metrics.

The art of site speed measurement

We often hear questions such as, Do you know how fast or slow my web page is? Does it matter to end users? Depending on the source of the question, you’ll likely have a different answer. If a page takes 10 seconds to load and the person asking the question can wait that long, then the speed is fine. However, if I am an engineer, I probably would consider 10 seconds as too slow, and I’d want to know why the page is slow. Is the server response slow, or is the network connection slow? How long do I need to wait to see the first impression on the page? How long does it take to render the full page after getting the first byte of data? What is the speed on different types of browsers so I can tune the slow ones? If I am a business person, I would want to know how many users are affected by slow web performance; a distribution of performance data, such as the median or the 90th percentile, would be useful. Most importantly, everyone wants to know the correlation between site speed and the business metrics. Does site speed affect our company’s top line?

To answer these different questions, we will need various measurements and data analysis views. Throughout the design and enhancement of Site Speed Gauge, we have continually ensured that it is extensible to meet different needs. Site speed measurement is an art; you will experience its beauty when it exactly matches what any particular circumstance calls for.

How Site Speed Gauge works

The following diagram describes the 10 steps of Site Speed Gauge, from receiving a user request to reporting data:

10steps

For older browser versions, where the Navigation Timing object is not available, we mainly use JavaScript to capture the timers. Client-side timer measurements using JavaScript are relatively easy and accurate, as we can start the timer at the beginning of the page, and use the client machine’s clock to measure any points during the page rendering. The difficult measurement is the total end-to-end time, from when the user’s click initiates the page request through when the page completes with the browser onload event. We can use a cookie to store the timestamp when a user leaves the previous page, and measure end-to-end time when loading of the current page is completed. However, if the previous page is not from eBay—for example, if it is a third-party search or referral page—we will miss the end-to-end time metric.

Therefore, we instead use server-side timers for end-to-end time measurement. The beginning timestamp st1 is the time when an app server gets the request, and the end timestamp st2 is the time when another metrics collection server gets the site speed beacon request. We miss the URL request time for the real user’s page, but we compensate for this fact with the beacon request time. To handle the case of the app server’s clock and the metrics collection server’s clock not being in synch, we can use a time-synch service on the two machines. To provide sufficient accuracy, the synch service should return timestamps in milliseconds. Alternatively, we can use a single database timestamp to eliminate the time synch issue.

For the latest versions of browsers, we also send back measurements from the Navigation Timing object those browsers create. These measurements give us very useful information about client-side performance, such as DNS lookup time and network connection time. Through our analysis of the data, we have identified DNS and connection times as major sources of overhead if our datacenters and our clients are not located on the same continent.

Site Speed Gauge features

Currently, Site Speed Gauge supports these major features:

  • Key performance metrics are provided, such as total page end-to-end time, client-side rendering time, first byte time, DOM ready time, certain JavaScript execution times, server processing time, above-fold time, and graphical ads time. About 30 timers are available from the various stages of page processing.
  • Metrics are broken down by page, browser, device, international site, and client IP location. These breakdowns are in different selection dimensions when you query and view the data.
  • Data sampling is adjustable, from 100% sampling for low-traffic pages, to smaller percentages for high-traffic pages. For a heavily trafficked site like eBay, with billions of page views per day, big data processing and scaling requires controlling the sampling data size.
  • Through the gauge’s support for A/B testing, we can tie site speed to site feature changes. This ability is very useful for collecting business metrics correlation data; more on this in the next section.
  • In addition to collecting web performance data, we can plug in collection of other user behavior data:  user clicking, scrolling, and browser size data. We have built heap maps on top of the data to analyze user interactions with the pages.
  • We can plug in other browser performance objects, such as the Navigation Timing object available in new browser versions. As described previously, this capability enables us to get more visibility into the network layer, such as DNS lookup and network connection times.
  • Site Speed Gauge also supports other client-side capturing, such as JavaScript error capturing. Without this capability, we would be flying blind, unaware of problems until getting complaints from end users.

Integration with A/B testing

Tracking the trending of site speed helps us to identify site feature rollout issues, and to monitor site speed improvements we can achieve from various optimizations. In addition, we can run different versions of a web page at the same time and compare the site speed of each version. The resulting data enables us to correlate business metrics with site speed in a precise way. One of the characteristics of eBay’s business is seasonality; if we can simultaneously monitor site speed and business metrics for a page with seasonal or other variations, we can build meaningful correlations.

To enable such analysis, we have integrated Site Speed Gauge with eBay’s A/B testing platform. In this platform, business metrics are collected and analyzed based on testing and control groups; a user session id identifies which group a user belongs to. We use the same session id to collect site speed as well as business metrics. Once a user is sampled for site speed, all pages viewed by the same user are sampled so that we have all site speed and business metrics data for this user.

Several years ago, we ran two versions of the eBay search page. The versions had the same features but different implementations of the page, one our classic search and the other a new implementation. We collected data on site speed as well as buyer purchases per week (PPW, a business metric related to conversion of traffic to purchases), and over time we found a strong correlation between site speed and PPW. As the chart below shows, the correlation is not linear, starting from 10% site speed and 1 % PPW, increasing to 35% site speed and 5% PPW. Our interpretation of this result is that a small change in page speed might not have much impact, but a large page speed change can have a noticeable effect on end users; for example, a large reduction in site speed can cause user activity to drop, or even abandonment of the site, thus affecting conversions.

fig2_revised

As we established the correlation between site speed and PPW, business people and engineers alike began vigorously emphasizing site speed when they designed and implemented features. Now, the engineering culture at eBay is that site speed and optimization are part of the design, implementation, and rollout of features. Many important changes to the site first go through A/B testing to ensure that we’re maximizing site speed as well as business impacts. In addition, the Site Operations team uses dashboards and alerts to monitor site speed 24×7.

Data processing and visualization dashboards and tools

Although we typically collect 10% of site speed sampling data for high-traffic pages, these pages still generate large amounts of beacon data. A batch job running at certain time intervals performs site speed data processing and aggregation in ways that meet the different needs of various consumers of site speed data.

Site Speed Gauge provides dashboards and analytical tools that support data visualization in highly configurable ways. For example, we can select daily results, hourly trending on the 50th percentile, or hourly trending on the 90th percentile. We can view data on a specific eBay page, for a particular international or US site. We can see the site speed for various A/B testing groups.

Here is an example of the site speed breakdown by browser:

browser_site_speed

And here is an example of the browser usage breakdown:

browser_usage

For those who are interested in network latency and the impacts of a CDN, they can view results by geographic location. We process the data based on client IP address to support this feature. The image below shows a heat map of site speed for the United States. A picture is worth a thousands words; the heat map helps people identify page speed visually.

geographic

Conclusion

At eBay, Site Speed Gauge helps us to identify user web performance issues as well as to monitor site speed improvement efforts. Its abilities to collect real-world data and correlate business metrics provide powerful tools in eBay’s highly trafficked consumer-oriented web site. We built extensibility into Site Speed Gauge. In the past several years, we have enhanced Site Speed Gauge to support different needs, and we expect to continue enhancing it in the future.

{ 3 comments }

RaptorJS is an open-source toolkit for building JavaScript modules and UI components that function on the server and in the browser. RaptorJS promotes modularity, which is crucial to building reasonably complex HTML web applications that are maintainable, easily testable, and optimized. The central goal of RaptorJS is to enable applications to provide the best user experience.

As an eBay project, RaptorJS was designed to be extremely efficient and lightweight. Rather than being a monolithic framework, RaptorJS embraces a modular design; it is intended to work alongside existing JavaScript libraries—not replace them.

RaptorJS provides support for the following features:

  • Object-oriented JavaScript: Defining JavaScript modules, classes, mixins, and enums based on the Asynchronous Module Definition (AMD) syntax
  • Resource optimization: Resource minification, bundling, compilation, checksums, and CDN integration for optimal delivery of JavaScript modules and UI components to the browser
  • Packaging: Defining dependencies for JavaScript modules and UI components based on simple package.json files, with support for optional and environment-specific dependencies
  • Client-side asynchronous package loading: Downloading packages of JavaScript and CSS resources asynchronously after the initial page load
  • Server-side JavaScript module loading: Loading JavaScript modules in multiple server-side JavaScript environments, including Node and Rhino
  • HTML templating: Using the same extensible templating language on both the server and in the browser to build UI components and web pages
  • Widgets: Automatic binding of JavaScript behavior to DOM nodes associated with rendered UI components—whether the UI component was rendered on the server or in the browser

The following sections describe each of these features in more detail.

Object-oriented JavaScript

Using the RaptorJS library, you can define namespaced modules, classes, mixins, and enums. This library employs a clean and easy-to-understand syntax based on the AMD syntax.

To make building modular JavaScript applications even easier, RaptorJS provides an AMD implementation that extends AMD in backwards-compatible ways. The syntax is very close to pure JavaScript, but fills gaps in the existing JavaScript language. Now you can start creating modular code that is easy to maintain without waiting for a new version of JavaScript or switching to an alternative language.

Resource optimization

The RaptorJS Optimizer is a server-side tool to build optimized web pages by bundling, compiling, and minifying page dependencies. This tool makes managing complex JavaScript and CSS dependencies almost effortless while helping you produce extremely optimized web pages. 

Unlike other optimizers, the RaptorJS Optimizer does not require that you write your code a certain way. Simply tell the optimize

r which modules your page depends on, and it will take care of the rest—including writing optimized JavaScript and CSS bundles to disk and generating the HTML that is required to include those bundles. Since every application has different requirements, the RaptorJS Optimizer gives you full control over how resources are combined and where to include resources on your HTML pages.

All dependencies are described using RaptorJS package.json files, which are easy to maintain and allow you to leverage the full power of the RaptorJS Packaging system.

Packaging

RaptorJS extends the popular package.json format so that it is better suited for packaging up JavaScript modules and UI components to be delivered to the browser. RaptorJS packages are used by the RaptorJS server-side module loader, as well as to optimally deliver only the required code to the browser. RaptorJS imposes no requirements on the code that is packaged, so you can continue to write code however you prefer. Furthermore, RaptorJS packages are extensible and support any type of dependencies (JavaScript, CSS, LESS, Raptor Templates, etc.).

Here are some of the use cases this packaging model supports:

  • You can create separate RaptorJS packages for JavaScript modules and UI components.
  • JavaScript modules and UI components can be delivered and loaded differently based on the target environment. For example, you can package UI components so that CSS code sent to mobile devices is different from the CSS code sent to desktop browsers.
  • You can make dependencies explicit—including which dependencies are optional or environment-specific.

Client-side asynchronous package loading

RaptorJS includes a lightweight AMD-compatible package/module loader that enables JavaScript and CSS resources to be downloaded asynchronously after the initial page load. This asynchronous package loader works in conjunction with the RaptorJS Optimizer to efficiently download resources from the server.

Server-side JavaScript module loading

RaptorJS provides a server-side module loader that integrates with multiple server-side JavaScript environments, inclu

ding Node and Rhino. The server-side module loader reads modules’ package files to determine which code to load based on the target environment.

The RaptorJS AMD module loader integrates seamlessly with the Node module loader. In addition, RaptorJS provides a fully compliant CommonJS and AMD module loader for Rhino.

You can load JavaScript modules in multiple server-side JavaScript environments, and you can load JavaScript modules differently in Node and Rhino server-side JavaScript environments.

HTML templating

Raptor Templates is a new templating language that elegantly blends powerful templating directives with HTML tags. Unlike most other templating languages, Raptor Templates also enables you to use custom tags to embed high-level UI components into templates.

Raptor Templates is XML-based so that the XML structure of an HTML document can be used to its full advantage, making it simpler for you to write easily readable templates and to provide optimal output. Raptor Templates includes a compiler that converts HTML templates into native and optimized JavaScript code. Equally effective in both the browser and on the server, no other templating language makes it so effortless to produce easily readable templates that are blazingly fast with such a miniscule footprint.

Use Raptor Templates wherever you want to produce HTML.

Widgets

The RaptorJS Widget Framework simplifies the creation of web applications built using UI components. This lightweight framework provides a mechanism for automatically attaching JavaScript behavior to the DOM subtrees associated with rendered UI components—regardless of whether the UI components were rendered on the server or in the web browser.

The RaptorJS Widget Framework does not dictate how a client-side widget is implemented. Furthermore, there is no complex class inheritance hierarchy (only a few mixins that get applied to all initialized widgets).

UI components that use Raptor Templates to render their view will benefit from the bindings that allow widgets to be attached to rendered HTML elements. During the rendering of an HTML template, the widget framework keeps track of which widgets have been rendered and which HTML element each widget is bound to. As a result, widgets can be automatically and efficiently initialized without having to rely on the CPU-intensive task of scanning the final DOM tree to discover rendered widgets.

Conclusion

Lightweight, optimized, modular, portable, and designed for scale, RaptorJS simply makes JavaScript better. To view demos and get started, visit http://raptorjs.org

Kudos to Patrick Steele-Idem (@psteeleidem) of eBay’s presentation platform team, who built RaptorJS.

— Neeraj Khanna (@nk94555), presentation engineer at eBay

{ 0 comments }

I’ve been with eBay Inc. since 2009, and even just in the past four years, I’ve witnessed a significant change in the growth and importance of data centers, and the way we, as an industry, measure their efficiency. The Green Grid’s Power Usage Effectiveness (PUE) metric – as well as the water and carbon equivalents of the same metric – have spurred organizations to track and report on these measurements, demonstrating greater data center efficiency for their businesses and in many cases even sparking competition between companies.

The PUE standard for data center efficiency has brought immeasurable benefit to the industry; however, it’s only one piece of a larger puzzle that transcends data centers and energy efficiency. In fact, if anything has become clear over the past several years, it’s that technology decisions are inextricably linked to business performance. We can see more clearly now than ever before that our designs, purchases and operating decisions have real, tangible effects on the key indicators that are important to running a business: cost, performance, environmental impact and, ultimately, revenue. If we believe this to be true – and I certainly do – then we can no longer make choices regarding our technical infrastructure  in a piecemeal, isolated, component-by-component manner that ignores the larger context of the decision. And yet the way we analyze and optimize the efficiency of our technical infrastructure – our very ability to measure our value to business leaders and stakeholders – hasn’t, until now, connected these indicators in a meaningful way.

At eBay Inc., we’ve spent the past two years developing a new approach to address this gap. Today, we’re releasing the Digital Service Efficiency methodology, or DSE for short. As you’ll see below, DSE is essentially a dashboard for a company’s technical ecosystem – the data centers, compute equipment and software that combine to deliver its digital services to consumers. Much like a dashboard in a car, DSE offers a straightforward approach to measuring the overall performance of technical infrastructure across four key business priorities: performance, cost, environmental impact and revenue. Similar in spirit to the miles-per-gallon (MPG) measurement for cars, DSE enables executives and technical professionals to see the results of how their “engine” performed with real customer consumption – or, rather, how the ebay.com engine performed as our users drove it. Like its vehicular counterpart, the DSE dashboard equips decision-makers to see the results of their technical infrastructure choices to date (i.e., what MPG they achieved with their design and operations), and serves as the flexible tool they need when faced with making new decisions (i.e., what knobs to turn to achieve maximum performance across all dimensions). Ultimately, DSE enables balance within the technology ecosystem by exposing how turning knobs in one dimension affects the others.

Here’s what it looks like:

In the case of ebay.com, what you see is how we’ve combined all of our customer-initiated requests into two distinct services: buy and sell. DSE measures how many business transactions are completed per kilowatt-hour of electric power consumed for these services. Or put another way, it shows how much energy is consumed to deliver each transaction. Transactions divided by kilowatt-hours reveals the performance of each service. Once these two variables are established, a family of interconnected metrics can be derived to provide further insights for optimizations — for example, the cost per transaction, tonnes of CO2e generated per transaction and revenue created per transaction. See more of these details by clicking around the dashboard for yourself.

With this information, we’re able to see and do things that we simply weren’t equipped for previously. As an example, by using the DSE dashboard, some of our software engineers saw that by slightly decreasing the memory allocated for an application in a pool of servers, they could remove 400 servers from the pool. This insight helped us eliminate nearly a megawatt of power consumption and avoid spending more than $2 million to refresh the servers. This simple software tweak helped us lower power consumption, decrease costs, and increase system performance, ultimately increasing our revenue per transaction. And that’s just one example.

Thinking bigger-picture and longer-term, DSE is enabling us to project forward and build tangible plans for what we as a business need to do today to realize the results we’re looking to achieve tomorrow. In essence, we’re able to balance and tune our technical infrastructure to optimize how effectively it’s driving core business performance. And as business priorities fluctuate over time – envision a scenario in which, say, federal or state climate change legislation passes in a couple of years and we’re forced to be even more aggressive in the way we address our environmental impacts – we have the flexibility to model the various choices and trade-offs we’ll need to face, from an IT infrastructure and operations perspective, so that we can help propel the business forward to achieve its goals.

By releasing DSE, and our corresponding data, our intent is to establish a baseline for 2012 and then to set productivity and efficiency goals for 2013. These are our goals for the year ahead:

  • Increase transactions per kWh by 10%
  • Reduce cost per transaction by 10%
  • Reduce carbon per transaction by 10%

We’ll be updating and reporting out on all of the numbers in the dashboard and on our progress toward our improvement goals each quarter. As we continue on our course to fine-tune our own technical infrastructure, we hope to encourage other companies – particularly those that deliver their services digitally – to explore the DSE methodology and see how it can help expose the business value and performance of their own technical infrastructure. And while each company is different in terms of the services they provide and the variables that can be tuned to balance the cost, performance and environmental impact of delivering those services, DSE provides a framework for how other companies can begin.

Our DSE solution paper is just the beginning of what we hope will be an exciting new chapter in how we, as an industry, think about measurement and value. By sharing our methodology and progress, we at eBay Inc. hope to start that conversation. And as other companies are inspired to make their technical infrastructure as efficient and productive as possible, together we as an industry can drive the digital economy to even greater heights.

To comment or continue the conversation, feel free to email dse@ebay.com.

{ 2 comments }

Complex ideas can be conveyed with just a single image. That’s the notion of the common expression, “A picture is worth a thousand words.”

To provide rich user experiences, today’s web sites are incorporating lots of images. In most cases, these images are high resolution and have larger dimensions than in the recent past. Images contribute more to page weight than any other resource; according to httparchive.org, they represent 62% of the weight on an average web page.

Of course, as the use of images has increased so has the challenge of serving them without adversely affecting the user experience. This post addresses some of the most effective techniques for optimizing the perceived and real load times of image-rich web pages.

WebP

WebP is a new image format from Google with a 25-34% smaller file size compared to JPEG images. Currently, Chrome and Opera are the only browsers that support WebP. According to wikipedia, Chrome has a 36% share and Opera a 1% share of browser usage.

This test from webpagetest.org compares the page load time of WebP vs. JPEG. The test has one page with 50 images in the WebP format, and another page with the same 50 images in the JPEG format. Because the WebP page had to download fewer bytes (474484 vs. 757228), it completes loading much earlier compared to the JPEG page.

If you track your web site’s browser usage stats and find that Chrome/Opera users are a sizable chunk, using WebP images will improve the page load time for these users.

Progressive JPEG

A simple or “baseline” JPEG file is stored as one top-to-bottom scan of the image—whereas the progressive JPEG format divides the file into a series of scans. When a progressive JPEG loads, the user sees a low-quality image at first, but gradually sees the image at its full quality. Most of the latest browsers support progressive JPEGs. They improve the perceived loading speed when compared to baseline JPEGs, as this webpagetest.org test demonstrates.

Try progressive JPEGs for your above-the-fold images, and do A/B testing to see the impact on your site’s user experience.

Lazy loading

The most popular image optimization technique, widely used across the web, is lazy loading. This technique involves loading only a subset of the images initially, and loading the rest of the images on the scroll event or after the window on-load event. Using the on-scroll event to lazy-load images will significantly improve the page load time.

This technique might require the use of JavaScript. The only issue I see with lazy loading is that if the user loads a page through the browser’s back or forward button, JavaScript usage might cause some lag before the lazy-loaded images appear.

Image embedding (Data URI / Base64)

Each HTTP request exacts a price. Another popular technique for improving page load time is to minimize the number of HTTP requests. To improve the perceived load time, you can embed above-the-fold images using the Data URI scheme (base64). With this technique, images can be delivered as part of the page’s main payload; the HTTP request can be eliminated completely. Consider, for example, this <IMG> tag, which uses an HTTP request to display Right arrow (a right arrow):

<IMG src="http://pics.ebaystatic.com/aw/pics/help/infohubs/images/imgNaviArrowRgt_12x12.gif">

The following <IMG> tag uses the Data URI scheme to embed the image in the page:

<IMG src="data:image/gif;base64,R0lGODlhDAAMANUmACxw0RZVtClvzhxevA1OqApFoQQ/mxFMqARBnApFnxVWsipxzRZXsyFmwRxfugVIoSNlwx1fvStv0BZWsAA9lAA8lApHoA9PpwNAmSduyilsxx5dvCRlwxxevhRXsiNkwidsyRFOqSFmxQ5Nqidsyxpfuv///wAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAACH5BAEAACYALAAAAAAMAAwAAAY6QJPQJBkajQDB8ShYAJZDUkYDgpo+EE5DBO2UNgNHZKnwBBiBwXIUOhAmUMujcLFiEAmrqWLQmyhQQQA7">

The Data URI approach has two drawbacks:  the main payload of the page is increased, and the images aren’t cached by the browser. To address the caching issue, you could deliver Data URIs via an external JavaScript or CSS file; but you might not see the same perceived speed improvement that you would if you embedded the images directly in the page.

Sprites

Sprites are another way to minimize the number of HTTP requests. With this technique, you “stitch” together different smaller images and display the stitched image on the page, using CSS background styling to show different parts of the image. Sprites are easier to implement in cases where fairly static sets of images are to be displayed. Applying this technique to dynamically constructed pages—such as a search results page, where images to be displayed can vary substantially over time—could be challenging. The stitching process would need to be on demand and very fast. Moreover, if you’re using a content delivery network (CDN), you might lose the CDN benefits, because the stitched images might not be in the CDN cache servers and so the requests would come to your app servers instead.

CDN

By serving all static resources for a web page through a CDN, you can significantly reduce the actual page load times. Instead of the static content being requested from your servers, it would be delivered from CDN cache servers that are closer to the end user. Using the same images across all of your web pages would increase the CDN cache hits.

Domain sharding

Traditionally, web browsers place a limit on the number of simultaneous connections they can make to one domain. To increase the number of simultaneous connections, you can use multiple domains to serve the resources. You would have to pay for extra DNS lookup time for these extra domains, but the benefits outweigh this overhead. Domain sharding is applicable to all external resources—not just the images.

For mobile use cases, domain sharding might not be optimal, because the connection overhead for mobile browsers exceeds the benefit of having additional simultaneous connections.

Image dimensions

Specifying the width and height of every image in the <IMG> tag or in the CSS avoids unnecessary reflows and repaints. For example:

<IMG src=” http://thumbs.ebaystatic.com/d/l225/m/mHpX0W1OpQlEaydbGwCWuOg.jpg” width=”208px” height=”126px”>

This practice improves the rendering time and provides the user with a much smoother rendering experience.

Image scaling

If you need a smaller image, scale the bigger image and create the smaller one. Avoid using width and height specifications to scale images on the fly—a practice that would require downloading unnecessary extra bytes, thereby increasing page loading time.

Here is an example of poor practice:

<IMG src=”bigimage_width300_height300.gif” width=”100px” height=”100px”>

Instead, generate a 100 x 100 image, and use it:

<IMG src=”resized_bigimage_width100_height100.gif” width=”100px” height=”100px”>

The generated image will be smaller in size.

Image pre-fetching

If you can guess the user’s next move, you can pre-fetch some images (maybe the above-the-fold images) for the next page in the idle time or after the current page is loaded. This technique would load the next page faster, since the pre-fetched images would come from the browser’s cache.

Conclusion

To provide rich content, images are king. The optimization techniques discussed in this post are widely used across the web. By adopting all or some of them, you can substantially improve the perceived and real page load times for your site. Personally, I am thrilled about WebP and progressive JPEGs, and I hope more browser providers will come on board to support WebP in the future.

{ 7 comments }

Becoming a World-Class Tester

by Ilari Henrik Aegerter on 01/31/2013

in Testing

 A tester is somebody who knows that things can be different.

That is Jerry Weinberg’s description of a world-class tester, which reminds us that testing is about critical thinking and imagination. It is about not believing anything at face value, but maintaining an investigative mindset.

Testing is questioning a product in order to evaluate it.

That definition of the activity of software testing, from James Bach, includes one of the most important aspects of testing: information discovery and delivery. Software testing is about revealing the unknown.

If quality means value to some person who matters (as Weinberg has defined it), then testing is in the service of discovering anything that threatens this value. To provide this service, a tester needs to be able to do a variety of things, such as asking good questions, commenting on requirements, and interacting with a product in such a way that it yields information that’s useful.

My definition of a world-class tester is a person who is able to rapidly discover highly relevant information about the product, who makes the most use of any resource that is available to him/her, and who has the respect of people involved in a project. It is a person who can be trusted.

So, given these definitions, what skill and mindset dimensions should you invest in to become a world-class tester? What areas do you need to be good at in order to avoid being discredited as a “mouse-click-monkey”?

Technical proficiency

At the higher levels of management, software testing is often seen as only a technical discipline. They think your effectiveness as a tester depends on your ability to write code that does the testing for you. But there are many aspects of software testing that are simply not automatable. Humans are incredibly skilled at thinking and adapting to new information that arises based on context, and since software is a system of ever-changing functions and states, sometimes it is more efficient to do exploration. Automation is the opposite of that, since it involves scripted procedures.

However, automation can free the time of testers to do more exploration, which allows discovery. Anything that is either highly repetitive/boring or that needs to be run over and over again, is a good candidate for automation. Automation is very valuable in these areas.

Developing powerful and maintainable automation is difficult. I am convinced that a mediocre software developer is not able to live up to the requirements for such a job. If you want to do useful test automation, you’d better be an excellent developer in addition to having other testing skills.

Here are some more skills and mindsets that I believe will help you:

Investigative curiosity

If you think that testing is a necessary evil – something to get out of the way until you can do something really cool – you are in danger of missing important problems. World-class testers are constantly exploring the capabilities and limitations of their products while maintaining a healthy skepticism about those products’ capabilities.

If you want to give yourself a fancy title, think of yourself as a “compulsive empiricist,” guided by irresistible impulses to find the truth about what’s happening. World-class testing means following the urge to get to the bottom of things, not giving up until you have experienced enough value for yourself. When you see a tester shying away from unverified assumptions, that’s a good indication they take their job seriously.

Observational skills

If you want to observe something, you need to notice it first. Observation is the ability to notice important or significant details. While testing the product you need to be able to observe situations that are unusual.

Who would be a good teacher in the domain of observation? Go for a walk with a five-year-old and see what grabs his/her attention: a little crack in the pavement, a beautiful flower in a garden, a snail slowly making its way on a wall. Children are natural observers; they are interested in everything that happens around them. Their channels are wide open and they observe with joy.

Condition yourself to keep your channels open. Emulate child-like behavior. Train yourself in general awareness. Look closer. Slow down. Observe.

Communication skills

You will encounter circumstances where you have to defend your findings. You will have to convince people to take action, so you’d better know how to explain why your findings are relevant.

World-class testers actually practice describing their testing. They know their explanations need to go way beyond “Yeah, I tested it, trust me.” If you can describe your testing, your credibility as someone who knows what he/she is doing will grow. One way to get better is to use the Heuristic Test Strategy Model when you explain what you are doing. Or, even better, you will come up with your own models.

As you tell your story, remember that communication is a two-way street. World-class testers follow up on answers they get from their questions and are alert to new information that comes up in conversation.

Writing skills and bug advocacy

The richest communication happens when you are able to talk with somebody face to face. You observe facial expressions, you can react to indicators of confusion, and misunderstandings can be clarified right away. Not so with writing. The channel is narrower and more prone to misunderstandings. Clarifications need more time and effort. 

Bug advocacy often goes along with writing skills. Your ability to describe the problem accurately and understandably influences the probability of the right bugs getting fixed. Not every reported bug should be fixed, but the ones that should had better not escape because of poor wording.

A bug report is a persuasive document that needs to influence the right people to act. Free your bug reports from irrelevant information and don’t make them come across like a tabloid headline screaming at the reader.

You might want to keep a record of your past bug reports. Which ones produced confusion? Could you have written them differently? Would a change of wording have resulted in better understanding?

Domain knowledge

As a tester you are not just comparing specifications with the implementation. There are many other references, such as user expectation, consistency with comparable products, or the history of your product, that you can use to identify problems. Domain knowledge is also one of the references you can develop. Talk to your customers. Have a chat with the marketing guys. Spend a day with Customer Support. The better you know what is important for your business, the better your judgment will be.

Willingness to learn

You are a knowledge worker. Knowledge is not static, especially not in the technical domain. Constant learning is essential in order to become better at what you do. A situation where humans are interacting with software is a situation where software is interacting with humans. It is helpful to try to understand how both work. I recommend exploring the social sciences (cognitive psychology, social interaction theories, etc.) as well as increasing your technical education.

In my opinion, every tester who wants to become really good will profit from participating in the excellent BBST (Black Box Software Testing) courses. Each course takes four weeks, and you will spend 15-25 hours each week on intensive hands-on exercises. You can find more information about these courses on the website of the Association for Software Testing.

Another course I highly recommend is RST (Rapid Software Testing), which focuses on context, exploration, and thinking to find better bugs. It is currently taught by James Bach, Michael Bolton, and Paul Holland. A skillful search on the engine of your choice will lead you to the right place.

Social skills

Testers who constantly whine about their job and how horrible everybody else is get my blood boiling. Self-victimization is poisonous. Sometimes you just have to let go. If you cannot do that, you will be perceived as a sourpuss. If the crowd goes for a Friday evening beer, you won’t be invited. Imagine what influence that has on people’s willingness to collaborate with you.

Humor

Humor helps. Among many other things, it helps to sustain your sanity. As a tester, you often live in a stressful environment. Smiling and laughter reduce the damaging effects of stress hormones. It helps you to stay focused on what you want to be doing: testing.

Practice

And above all, you need to practice your skills. If you don’t practice, you can’t reach the higher levels of proficiency. While you practice, observe yourself, find areas for improvement, and make all of your actions deliberate and self-reflective.

Conclusion

Aiming to become a world-class tester does not mean reaching a destination, but rather being on a constant journey into new territories. The exploration never ends–which is exhilarating, and makes software testing so rewarding.

Happy testing!

{ 18 comments }

Until fairly recently, a single point of failure (SPOF)—a condition that can bring down an entire system—has been predominantly associated with the backend world of server clusters, networking, and operations. However, with the increase in third-party widgets and the complexity of modern web applications, the web community is increasingly aware that a frontend SPOF can also bring down an entire site. As a result of a frontend SPOF, the user sees an entirely or partially blank page, or experiences delayed rendering of the page.

This post describes a tool developed at eBay, called SPOFCheck, which has been open-sourced and is being integrated into engineering life cycles not only at eBay but globally.

Background

The awareness of frontend SPOF has increased tremendously among engineers, thanks to Steve Souders’ initial research on this topic, as well as recent articles and blogs emphasizing its importance; see, for example, Bjorn Kaiser’s post about third-party content causing SPOFs. Numerous utilities and plugins exist that can detect possible SPOF vulnerabilities in a web application, most notably webpagetest.org, the Chrome plugin SPOF-O-Matic, and the YSlow 3PO extension.

At eBay, we wanted to detect SPOF at a very early stage, during the development lifecycle itself. Such early detection would require an additional hook in our automated testing pipeline. The solution we developed and open-sourced is SPOFCheck, a simple tool that works on our test URLs and produces SPOF alerts. The tool is integrated with our secondary jobs, which run daily automations on a testing server where a development branch is deployed. When SPOFCheck detects possible vulnerabilities, engineers receive SPOF alerts and act accordingly. As a result, SPOFs can be contained within the development cycle, rather than sneaking into staging or production.

Command-line interface

SPOFCheck is a command-line interface (CLI) that runs on Node.js. Source code, syntax, and usage documentation are available in GitHub. Running SPOFCheck involves simply specifying the output directory. Command-line options include output format (XML that most CI servers can parse, XML to be consumed by other utilities, or text), console-only output, and the rules (checks) to be run. By default, SPOFCheck runs the following rules:

  1. Always load third-party external scripts asynchronously in a non-blocking pattern.
  2. Load application JS in a non-blocking pattern or towards the end of the page.
  3. Attempt to inline the @font-face style; ensure that the font files are compressed and cacheable.
  4. Ensure that the font files are compressed, cached, and small in size.
  5. Ensure that inlined @font-face styles are not preceded by a SCRIPT tag (which causes SPOFs in Internet Explorer).

Extensibility

New rules can be easily added, either by pushing them to the rules array or by calling SPOFCheck’s registerRules function.

At eBay, we’ve built a thin web client to provide SPOFCheck as a service. eBay engineers can create multiple SPOF projects, which can be triggered on demand or configured to run on a recurring schedule. Project owners designate the teams or individuals who receive SPOFCheck results.

Acknowledgements

Thanks to Steve Souders, whose work (cited above) is the source of SPOFCheck’s default rules.

SPOFCheck reuses code logic from the GitHub projects spof-o-matic and 3po. The design and packaging of the tool is based on csslint, thanks to Nicholas Zakas and Nicole Sullivan.

Thanks also to perfplanet, who featured SPOFCheck in their 2012 performance calendar.

{ 2 comments }

Tomcat/Jasper Performance Tuning

by Sheldon Shao on 01/04/2013

in Software Engineering

JavaServerPages (JSP) is productivity-enhancing technology for building dynamic web pages. With more than 10 years of history, it is popular and mature. JSP 2.0 includes new features that make the code cleaner, such as the Expression Language (EL), the tag file, and simple tags. However, these new features can cause some performance issues.

eBay uses JSP technology to build web pages. More specifically, we use Geronimo 3 as the web container and Geronimo’s JSP engine, Tomcat/Jasper. To handle eBay’s huge volume of requests, the JSP engine must be powerful and must provide exceptionally high performance. Because Tomcat/Jasper shows some weakness in performance when using hundreds of EL expressions and tags, we have focused much effort on profiling and optimizing Jasper to make JSP more efficient. Most of the performance solutions and patches that we have submitted to the Tomcat community have been accepted. As a result, Tomcat users will soon be able to benefit from our work.

This blog post discusses the main issues we have identified and addressed.

EL resolving

When a JSP page includes a large number of EL expressions, EL resolving becomes a bottleneck for page rendering. The following JProfiler screen shot shows a case where 40% of the Java processing time is for EL resolving; the whole JSP page costs 1,865 ms, while the EL resolving costs 722 ms.

This profiling enables us to identify the heavy methods from Jasper:

 

Looking into the code, we found that every expression node needs to call CompositeELResolver.getValue when trying to resolve the value. The bottleneck is that CompositeELResolver contains at least seven EL resolvers:  ImplicitObjectELResolver, MapELResolver, ResourceBundleELResolver, ListELResolver, ArrayELResolver, BeanELResolver, and ScopedAttributeELResolver. The method getValue tries to return the value from each of these resolvers one by one, until one of them returns a not-null value. This logic results in many unnecessary resolver calls.

We proposed an enhancement:  a customized CompositedELResolver in Jasper that avoids many unused resolver calls. This change, accepted in Tomcat 7.0.33, saves 10% of the EL resolving time. The Bugzilla URL for this issue is https://issues.apache.org/bugzilla/show_bug.cgi?id=53896 (The issue was filed by the Geronimo team, who tried to resolve the issue for us).

We’ve filed some related Tomcat bugs as well, such as Bug 53867 (also accepted in Tomcat 7.0.33) and Bug 53869.

JSP compilation

JSP compilation is a very heavy process that includes JSP reading, JSP parsing, Java generation and compilation, and class loading. We have achieved many optimizations in this logic as well.

JspReader was the focus of our initial performance-tuning work related to compilation. JspReader.mark is called too many times during JSP and tag files compilation. The following screen shot illustrates the excessive number of JspReader.mark invocations and the time they consume.

 

By using the JspReader buffer to reduce the number of invocations, JSP reading performance can improve fourfold. Tomcat 7.0.31 includes this enhancement; for more details, see https://issues.apache.org/bugzilla/show_bug.cgi?id=53713.

EL code generation

Even with multiple optimizations, EL evaluation is still a big hotspot. It is not only time-intensive but also CPU-expensive. EL resolving is the root cause. Therefore, if we can bypass EL resolving in some cases, we can improve performance.

The idea is to do code generation for simple EL expressions, such as ${model}, ${model.test}, ${empty model}, and ${model.value++}. Such expressions can be interpreted as simple Java code. We have identified three kinds of EL expressions that can use code generation:

  • Simple EL expression containing only one part of the expression — for example, ${model}. Here is an example of the generated code for this kind of expression :
this.getJspContext().findAttribute("model")
  • EL expression with two nodes where the type of the first node is specified by a tag file attribute—for example, ${model.location}, where model is specified by the following directive in the tag file:
<%@ attribute name="model" required="true" type="com.ebay.raptor.search.pres.common.model.results.ItemModel" %>

   Here is the generated code for this example:

(getModel() != null ? getModel().getLocation() : null)
  • EL expression with logic or arithmetic where the value part can be generated. For example:
${(intlExpansion eq 'true' && not empty model.location) || sortType==7}

   The generated code for this example looks like this:

(org.apache.jasper.runtime.ELRuntimeUtil.equals(getIntlExpansion(), "true")&&(! org.apache.jasper.runtime.ELRuntimeUtil.isEmpty((getModel() != null ? getModel().getLocation() : null))))

We have proposed an extensible ELInterpreter solution to Tomcat and implemented an interpreter for these three kinds of EL expressions. After the extensible interpreter has been accepted, we will contribute the code to Tomcat as well. This customized interpreter improves total page performance by 10%. Application developers will be able to inject their own interpreter to replace the default JSPUtil.interpreterCall.

For more about this enhancement, see https://issues.apache.org/bugzilla/show_bug.cgi?id=54239

JSTL code generation

Tag library performance is another hotspot for page rendering. Among the problems are the numerous method calls in the standard generated code and the high number of context switches. Every tag needs to call doStartTag and doEndTag.

Jasper has a code-generation solution for JSTL, which is based on the TagPlugin interface. Although eBay leverages the implementations in Jasper, there are many bugs. Here are some of the fixes we’ve contributed:

Issue

Bug

Status

Fixed version

Bug in JSTL tagPlugin “Out”

Bug 54011

Fixed

7.0.33

Bug in JSTL tagPlugin “Set”

Bug 54012

Fixed

7.0.33

Configurable tagPlugins.xml

Bug 54240

In-Progress

 

NPE in BodyContentImpl

Bug 54241

In-Progress

 

Bug in ForEach

Bug 54242

In-Progress

 

Bug in When and Choose

Bug 54314

In-Progress

 

Summary

Tomcat/Jasper is eBay’s chosen JSP solution. However, few solutions can fully satisfy eBay’s daunting performance requirements, and Tomcat/Jasper is no exception. Our efforts have identified four major performance issues:  EL resolution, JSP compilation, EL code generation, and JSTL code generation. Working with the Tomcat community, we’ve made substantial progress on some of these issues, and more enhancements and bug fixes are underway.

{ 4 comments }

Intern Pralay Biswas submitted the following blog post at the conclusion of his summer with eBay. Pralay was one of 120 college interns welcomed into eBay Marketplaces this summer.

In pioneer days, they used oxen for heavy pulling, and when one ox couldn’t budge a log, they didn’t try to grow a larger ox. We shouldn’t be trying for bigger computers, but for more systems of computers. — Admiral Grace Hopper (1906-1992)

Like any company where data is of primary importance, eBay needed a scalable way to store and analyze data to help shape the company’s future, formulate strategies, and comprehend customer pain points in a much better way. eBay began its Hadoop journey in 2007 — and yes, eBay has come a long way since then.

I spent the summer of 2012 with eBay’s core Hadoop Platform team, working on quality engineering. Part of eBay’s central Marketplaces business, this team is where eBay manages the shared clusters, augments public-domain Hadoop software, and validates each release of Hadoop at eBay.

The problem

  • How do we validate each new Hadoop release?
  • How do we test the software to decrease the probability that we are shipping bugs?
  • What about cluster-wide, end-to-end tests and integration tests?
  • How do we make sure there are no quality regressions?

These are difficult questions to answer, especially when it comes to Hadoop.

No silver bullet for Hadoop testing

Existing software solutions from the Apache community that orchestrate and automate Hadoop testing and validation are insufficient. JUnit and PyUnit are great for small, self-contained developer (unit) tests. AspectJ works fine for local, single-module fault injection tests. Herriot is also based on JUnit and has some badly designed, buggy extensions. EclEmma is IDE-specific, and inappropriate.

Long story short: there is no existing framework or test harness that can validate a Hadoop platform release. Quick and automatic validation with minimal human intervention (ideally, zero-touch) is of primary importance to any company that needs to test with large-scale, mission-critical data. The quicker we store and analyze data, the quicker we understand our progress and ensure that we can roll out new software or configurations.

Engineer Arun Ramani developed the eBay Hadoop test framework. I was hired as a graduate summer intern to add tests to the framework that he conceived. In the following sections, I’ll elaborate on the test harness (as opposed to the tests that I wrote).

Architecture of the framework

The architecture of the test harness follows the model-view-controller (MVC) design pattern. The framework separates test cases from the test logic, so that the logic can remain almost unchanged; most changes can be made in the test XML only. The driver in the figure below is a Perl script that uses support modules (some more Perl modules) to prepare the test environment, and then pulls in the tests themselves, which reside in an XML file.

The main driver module acts as an orchestrator, and the support modules help determine the environment configuration; they restart daemons (name node, secondary name node, data node, job tracker, task tracker), send email reports of results, look for anomalous and correct exit conditions of processes, etc. The actual tests are in the XML file only. The driver parses these tests, executes them, and checks whether a test passed. Determination of a test’s pass/fail status can be achieved using Perl extended regular-expression comparisons of standard out, standard error, process termination, exit signals, log file data, or other system-level information. The use of Perl affords extraordinary visibility into end-to-end system-level test results (memory, cpu, signals, exit codes, stdout, stderr, and log files).

 

All tests are grouped according to uses cases.  Tests within group tags represent one use case. Every use case (group) has a setup action and a teardown action containing commands to be executed for the purposes of those individual tests. All commands to be executed go between cmd tags, and desc tags enclose test and command descriptions.

Test case example

Take, for example, the following test case, which runs a Pi calculation job on the Hadoop cluster. The setup action prepares the environment required by the subsequent test — in this case, executing  a command that deletes any existing directories so that the job can create new ones. The cmd tag would typically enclose a Hadoop command. However, this is not a restriction; the command could be a UNIX command line, a Java command to execute any jar, a shell script, another Perl script, or any arbitrary command. This flexibility enables the test framework to support tests not only for MapReduce and HDFS but for the entire Hadoop ecosystem.

 

When the test driver executes the command, it stores results in the variable $actual_out. The driver then compares this value to that of the exp_stdout variable specified in the test case; this is a regular expression comparison. The driver also checks that $actual_out has nothing in common with not_exp_stdout_regexp. Furthermore, the return code of the executed command (the Hadoop job) is compared against the expected return code (exp_rc). A strict test driver declares success of a test only if all the checks and assertions are positive.

Once all tests in a group are run, the commands in the teardown phase clean up any data or state that the tests created (in HDFS, HBase, or memory) during their run, leaving the cluster clean for the next test case. Once the end of the XML test file is reached, the driver calculates the total number of cases (groups) executed, passed, and failed, then sends this log to the configured email IDs.

Automation and results

This test framework is kicked off as a Jenkins job that the Hadoop deployment job runs. So, as soon as a new release is deployed and compiled on the QA cluster, this framework runs automatically to validate that the new bits are good. Modular and flexible, the framework has been used successfully to validate Cloudera’s CDH4 and Apache Hadoop 0.22. Going forward, this framework will validate all future releases of Hadoop at eBay.

 

{ 3 comments }

Bandwidth-based Experience

by Raja Bhogi on 10/10/2012

in Software Engineering

When designing new web pages or working on new features for an existing page, developers commonly consider the following factors:

  • Page appearance for different layout sizes
  • Browser-specific implementations
  • Page behavior with and without JavaScript enabled

All of the above points are valid, and developers should indeed think about them. However, there is another factor that developers might not think much about:  how their pages are loading on low-bandwidth connections or slower computers.

When users browse web sites, numerous factors contribute to the speed at which web pages load. For example, some users might have high-bandwidth connections but slower computers, while others might have high-speed computers with low-bandwidth connections. Due to such factors, users of your web site can have vastly different page loading experiences.

Most web sites collect real-time site speed data to see how their web pages are loading on their users’ computers. Typically, these numbers provide median, average, and 95th-percentile loading time. If you look closely into the 95th-percentile data, you might be amazed how slowly your web pages are loading. Can we do anything about these 95th-percentile users?

To analyze your users’ bandwidth experience, collect and segregate data based on the load time of your web pages. Next, determine what percentage of your users are experiencing slow load times. For this determination, it’s up to you to draw the line between fast users and slow users.

Suppose that your segregated data looks like this:

Time

<1sec

<2sec

<3sec

<4sec

<5sec

<6sec

<7sec

<8sec

<9sec

<10+ sec

Users

32%

64%

77%

85%

89%

92%

93%

94%

95%

100%

According to the data, pages are loading in less than 4 seconds for 85% of your users. Let’s say that you determine that slow loading is anything more than 4 seconds.

Now we have a data point to distinguish between slow and fast load times. But how do we detect load times in real time, per user? One approach is to download an image of X bytes, measure the time taken to load the image, and use that time to calculate the user’s bandwidth. However, this approach might not touch all factors—such as JavaScript execution time—that can affect page loading time.

Here is a different approach, which would touch all possible factors for slowness. When the user visits a web page for the first time, at the start of the session calculate the load time of the page and store it in a cookie. Do this calculation only one time, and use it for the rest of the session.

Calculating page load time is very simple. The following script measures purely client-side load times. (Note that it assumes that you have JavaScript functions to read and write cookies.)

<HEAD>
<SCRIPT> var start = new Date().getTime(); //Take the start measurement </SCRIPT>
</HEAD>

<SCRIPT>
    (function(){
        window.onload = function(){ //Use appropriate event binding mechanism
           var end = new Date().getTime(); //Take the end measurement 
           var loadtime = end-start;  //Calculate and store the load time
           var loadtimefromcookie = readCookie(); //Call JS function to read cookie
           if(!loadtimefromcookie){ //If load time not yet stored
              writeCookie(loadtime); //Call JS function to set cookie
           }
        }
    })();
</SCRIPT>

From the second request onwards, you can read the cookie; based on its value, you can strip down the features for lower-bandwidth users and/or add more features for higher-bandwidth users.

How can you improve the loading speed for lower-bandwidth users? Here are some techniques:

  • Show fewer results.
  • Use smaller images.
  • Cut down on heavy ads, or remove all ads.
  • Remove heavy or crazy features that are not required for basic functionality.

Do A/B testing with these changes, assess the gains for the 95th-percentile user, and see whether this approach makes a difference. Once you see positive changes, you can target and fine-tune page features based on user bandwidth.

{ 3 comments }

The Case Against Logic-less Templates

by Sathish Pottavathini on 10/01/2012

in Software Engineering

At a conference I recently attended, two separate topics related to templates sparked a debate off-stage about Logic vs No-logic View Templates. Folks were very passionate about the side they represented. Those discussions were the high point of the conference, since I learned much more from them than from the sessions that were presented.

While everyone involved in the debates had different opinions about the ideal templating solution, we all agreed that templates should…

  • not include business logic
  • not include a lot of logic
  • be easy to read
  • be easy to maintain

In this post I’ll describe what I prefer–rules, best practices, performance considerations, etc. aside. My preferences come from my experience with templates for medium to complex applications.

Full-of-logic, logic-less, and less-logic solutions

Before I go into the details, I would like to make one thing very clear. Although I’m trying to make a case against logic-less templates, that does not mean that I am advocating the other extreme–i.e., a templating language that allows a lot of logic. I find such templating languages, especially those that allow the host programming languages to be used inside the template, to be hard to read, hard to maintain, and simply a bad choice. A JSP template with Java code in it and an Underscore template with JavaScript in it both fall into the category of being a full-of-logic template. JSP and Underscore are not necessarily at fault here; rather, developers often abuse the additional freedom such solutions offer.

What I am advocating is “less-logic” templates in place of “logic-less” templates (thanks, Veena Basavaraj (@vybs), for the term “less-logic templates”!).

Good and great templating languages

A good templating language should offer, at a minimum, the following features:

  1. Clean and easy-to-read syntax (including the freedom to use whitespace that will not show up in output)
  2. Structural logic like conditionals, iterations/loops, recursions, etc.
  3. Text/Token replacement
  4. Includes

A great templating language should also offer the following features:

  1. Ability to be rendered on the server and the client
  2. Easy learning curve with as few new concepts as possible
  3. Simple template inheritance
  4. Easy debugging
  5. Great error reporting (line numbers, friendly messages, etc.)
  6. Extensibility and customizability
  7. Localization support
  8. Resource URL management for images, scripts, and CSS

The case against logic-less templates

I consider a logic-less template to be too rigid and hard to work with due to the restrictions it imposes. More specifically, here are the reasons I am not a big fan of logic-less templates:

  • Writing a logic-less template requires a bloated view model with comprehensive getters for the raw data. As a result, a messy and difficult-to-maintain view model usually accompanies logic-less templates.
  • Every new variation to the view requires updating both the view model and the template. This holds true even for simple variations.
  • Getting the data ready for the template requires too much data “massaging.”
  • Offsetting the rules of a logic-less template requires a lot of helper methods.

Regarding the argument of better performance with logic-less templates: While they might have a simpler implementation, they require additional preprocessing/massaging of the data. You therefore have to pay a price before the data gets to the template renderer. Granted, a templating language that allows more logic might have a more complex implementation; however, if the compiler is implemented correctly, then it can still produce very efficient compiled code. For these reasons, I would argue that a solution involving templates with more logic will often outperform a similar solution based on logic-less templates.

Conclusion

No doubt, there are advantages like simplicity and readability associated with logic-less templates such as Mustache. And arguably, there could be some performance gains. However, in practice I do not think these tradeoffs are enough to justify logic-less templates.

Logic in a template isn’t really a bad thing, as long as it doesn’t compromise the template’s readability and maintainability.

And as an aside, I’m definitely in favor of more debates than sessions in future conferences, since we actually learn more by hearing multiple viewpoints.

I welcome your thoughts!

Credits: Many thanks to my colleague Patrick Steele-Idem (@psteeleidem) for helping me write this post. He is working on some cool solutions like a new templating language; be sure to check them out when they are open-sourced.

{ 13 comments }

Copyright © 2011 eBay Inc. All Rights Reserved - User Agreement - Privacy Policy - Comment Policy