eBay Tech Blog

Data URI Sprites

by Senthil Padmanabhan on 07/12/2011

in Software Engineering

In the inaugural topic of this blog on site speed, Hugh Williams spoke briefly about image spriting as a technique to speed up the rendering of a page. CSS Image Sprites are now commonly used to remove the performance problems created by making separate HTTP requests for every image on a web page. Unfortunately the technique cannot be used on a web page with a dynamic image set such as the eBay search results page: the images on the page change with every request based on the item specifics, therefore combining them into one sprite image is not possible.

When brainstorming alternatives, we found what seemed to be an obvious solution: the image Data URI scheme. It works in all major browsers (except IE6 and IE7), and it solves our problem of too many HTTP calls. The code looks like this:

#item1 {    
background: url(data:image/png;base64,) no-repeat;
}

Though the Data URI scheme solved our original problem, it created two new ones:

  • A much larger HTML payload, since the images are now embedded in the page
  • A loss of browser cache benefits: all images are sent over the wire on every page request

We realized what we really needed was a combination of the two approaches, because we would then be able to create image sprites dynamically. So we wrote urigen, a simple web service that generates a base64-encoded image data URI scheme response in JSON format for a given set of image URLs.

Under the hood
For a page with a dynamic image set, the browser calls the urigen web service immediately after the page load event with the image URLs passed as GET (or POST) query string parameters. The web service then extracts the image URLs from the query string, makes HTTP calls to the image hosting servers, does a base64 encoding on the response, builds a JSON array and returns it as shown below.

Request:

http://localhost/services/urigen?thumbs2.ebaystatic.com/m/mfqVqeVYnfS8MAHPgpTnINQ/140.jpg&thumbs4.ebaystatic.com/m/mZScfBs8aQzwcGokxSUV_mQ/140.jpg
Or
http://localhost/services/urigen?params={“images”:[“http://thumbs2.ebaystatic.com/m/mfqVqeVYnfS8MAHPgpTnINQ/140.jpg",”http://thumbs4.ebaystatic.com/m/mZScfBs8aQzwcGokxSUV_mQ/140.jpg”]}

Response:

{“data“:[
{"url":"http:\/\/thumbs2.ebaystatic.com\/m\/home\/images\/productbrowser\/140.jpg",
"uri":"\/9j\/4AAQSkZJRgABAgAAZABkAAD\/........5AQEBAQEBAQEBAQEBAQEBAQEH\/\/z"},
{"url":"http://thumbs4.ebaystatic.com/m/mZScfBs8aQzwcGokxSUV_mQ/140.jpg",
"uri":"R0lGODlhbgAtANUAALIAJpnMAAAAmczl.......B4NmdQZTJ5QUFBQUFFbEZUa1N1UW1DQw=="}]}

The JavaScript engine then de-serializes the JSON response, creates images with the data URI scheme and displays them to the user. This technique (we call it Data URI sprites) blends image spriting with the Data URI scheme. Multiple image HTTP requests are dynamically combined into one, and there is no need to lay out images ahead of time as is necessary for CSS sprites.

Data URI sprites can be effectively used for dynamic images below the fold on a page, or for images that are shown on demand based on user actions.

A prototype of Data URI sprites with detailed documentation is available in the public github repository ImageURIGen. Websites like Flickr have tried similar techniques and have reported substantial performance gains, as Ross Harmes from the Flickr frontend engineering team explains in the High Performance JavaScript book.

Proposed optimizations:

  • The JSON response should have the appropriate HTTP cache headers to benefit from browser caching, if the same URLs are requested again
  • If the service is hosted on the same web server that is hosting the images, HTTP calls (from web service to image server) and their associated latency can be avoided and images can be read from the disk locally.
  • The web service should have a proper LRU caching mechanism in place to avoid repetitive calls for the same image and instead retrieve data from the server cache
  • The service should also be available in JSONP format with a JavaScript callback to enable browser cross domain communication
  • The urigen web service itself can be a server side call from the page and the data  sent as a separate below-fold chunk after the main page is flushed out

Senthil Padmanabhan
Engineering Lead & Site Speed Evangelist

{ 18 comments… read them below or add one }

raghuram gururajan July 13, 2011 at 1:05PM

With Reference to your above article when the page with dynamic image set loads is it possible to use Lazy loading for calling the urigen and fetching the image urls based on demand ?

Reply

Senthil P July 13, 2011 at 5:08PM

Yes. The last point in the Proposed optimizations section actually means lazy loading of images on server side after flushing the main page content. Since urigen is a web service this can be done easily.

Reply

David He July 13, 2011 at 5:04PM

A good reading from this for the impressive works to combine dynamic images in search result page. For static images (we have many static images in search and other pages), there was another external discussion thread on data uri vs. css sprites: Data URIs make CSS sprites obsolete. The 58 comments also have many good points:
http://www.nczonline.net/blog/2010/07/06/data-uris-make-css-sprites-obsolete/

Reply

raghuram gururajan July 14, 2011 at 10:29AM

Do we use entity tags to improve performance along with caching .Seems a very good approach look at this site for more details

http://developer.yahoo.com/performance/rules.html

Reply

Senthil Padmanabhan July 19, 2011 at 10:07PM

The main use of ETags is to return a 304 HTTP status code when there is a browser cache miss. But this requires proper server configuration in the VIP, as sometimes etags become server specific and when a request goes to a different server the entire response comes back. The safest alternate is to use the “Last-Modified” header in response which does the same task as 304 and very reliable. We are planing to implement that in the service.

Reply

kll July 18, 2011 at 2:58AM

Does this actually improve client-side caching? I’m not sure if AJAX is cacheable, especially when the URL contains query string.

Reply

Senthil Padmanabhan July 19, 2011 at 10:13PM

Hi Kll,
Any HTTP GET request (either AJAX or with query strings) are cacheable if they have the proper cache headers (Cache-Control & Expires). A classic example is the YUI combo handler. More details are available at http://developer.yahoo.com/performance/rules.html#cacheajax and http://www.stevesouders.com/blog/2008/07/17/yuis-combo-handler-cdn-service/

Reply

Adnan July 18, 2011 at 4:46AM

You re already making a call. Whats the use then?

Reply

Senthil Padmanabhan July 19, 2011 at 10:16PM

Hi Adnan,

I am not sure I understood your question. The main intention here is to reduce the number of separate HTTP image calls as much as possible by combining them into 1 data URI call.

Reply

Ryan McGrath July 18, 2011 at 6:32AM

Not to be a nitpicker, but it *is* possible to make this work in IE6/IE7 – much of the work done to figure this out comes from Stoyan Stefanov, with my finishing touch being getting it working under IE7 on Vista (big whoop, etc).

http://venodesigns.net/2010/06/17/you-got-your-base64-in-my-css/

I used this technique back when I worked at Webs.com, got an earlier version of the homepage loading incredibly quick.

Reply

Senthil Padmanabhan July 19, 2011 at 10:52PM

Hi Ryan,

Yes we thought about this method, but it becomes very clumsy when trying to do the same using JS after we get the response from the urigen. Directly embedding in CSS works well.

Reply

James Socol July 18, 2011 at 7:59AM

Another possible optimization: stick the data URIs into localStorage in browsers that support it, keyed on the URL, and possibly with some sort of LRU eviction mechanism.

If I run a similar search, I’m likely to get similar, but not identical, results, and having just one result different would invalidate the browser cache for the whole response, making me redownload all the data that hadn’t changed (because the request URL changes a little). If I can pull images out of localStorage, and only request new ones from urigen, the browser/HTTP cache (which is likely to miss most of the time) doesn’t matter.

Reply

Senthil Padmanabhan July 19, 2011 at 10:53PM

Thanks James, very valuable comment. We will look into this.

Reply

Peter Coles July 18, 2011 at 9:27AM

Cool technique! I like how it has the benefit of loading all the images at once, like a sprite, but not the overhead of managing background positions in CSS. For ie6&7 do you just sniff out the useragent and have the page load the images the oldy moldy way?

Also, another small proposed optimization might be having the images served from a different domain (even though they’re internally on the same server) so the browser doesn’t also send cookies for the image requests (this would be contingent on the jsonp support).

Reply

Senthil Padmanabhan July 19, 2011 at 10:58PM

Hi Peter,

Yes, for IE6&7 we use the old way (separate HTTP request for all images) and do not call the uirgen service.

And also the urigen service will be hosted in a different pic server domain so we get the advantage of not sending cookies and also more parallel connections.

Reply

Rob Benwell August 27, 2011 at 8:05AM

I created a couple of tools to help with data URI creation. These are command line utilities so they can be made part of your build system without external dependencies, so you may find them more useful than Duris.ru

Reply

Jtw December 22, 2013 at 7:58PM

One of the challenges of dealing with data URIs is when all of a sudden you have a ton of them to keep track of. In order to deal with this I’ve created a tool for automatic CSS generation and batch data URI encoding: http://datauri.net .
I’m pretty sure it’s the only tool available that will allow you to create dynamic, programmatic CSS based on a set of data URI images. Hopefully others find this helpful as well!

Reply

Senthil Padmanabhan December 25, 2013 at 7:28PM

The tool looks impressive, we will give it a shot. Thanks for letting us know.

Reply

Leave a Comment

{ 1 trackback }

Previous post:

Next post:

Copyright © 2011 eBay Inc. All Rights Reserved - User Agreement - Privacy Policy - Comment Policy