Author Archives: Senthil Padmanabhan

eBay’s Font Loading Strategy

The usage of custom fonts in web pages have steadily increased in recent years. As of this writing, 68% of sites in the HTTP Archive use at least one custom font. At eBay, we have been discussing custom web fonts for typography for quite some time, but never really pursued it. The main reason was due to uncertainty in end user experiences from a performance standpoint. But this changed recently.

Our design team made a strong case for a custom font to complement our new branding and, after multiple reviews, we all agreed it makes sense. Now it was on the engineering team to come up with an optimized implementation that not only uses the new custom font, but also tackles the performance overhead. This post gives a quick overview of the strategy we use at eBay to load custom web fonts.

Meet “Market Sans”

Our new custom font is rightly named “Market Sans” to denote affiliation with an online marketplace. As observed in the below image, it adds a subtle difference to typography, but in its entirety makes the whole page elegant and creates a unique eBay branded experience. Check out our desktop homepage, where “Market Sans” has been deployed.

Custom Font vs. System Font

Problems

It is well known and documented that custom web fonts come with a cost, and that is performance. They often delay rendering of text (critical to any web page) until the font is downloaded. A recent post from Akamai gives an excellent overview of the problems associated with custom fonts. To summarize there are two major issues, and it varies among browsers:

  • FOUT — Flash of Unstyled Text
  • FOIT — Flash of Invisible Text

As expected, the design and product teams were not happy with the compromise. Yes, custom fonts create a unique branded experience, but not at the cost of delaying the same experience. Additionally, from an e-commerce perspective, custom fonts are a good enhancement and not an absolute necessity. System fonts can still provide a compelling typography. So it was up to the engineering team to come up with an efficient font loading strategy with minimal tradeoffs.

Strategy

Our strategy was pretty simple — avoid FOUT and FOIT: Use the custom font if it is already available (meaning downloaded and cached), else use default system fonts.

Fortunately, there is a CSS Font-Rendering proposal that adds a new @font-face descriptor named font-display. Using font-display, developers can specify how a font is displayed, based on whether and when it’s downloaded and ready to use. There are many values for font-display (checkout this quick video to understand them), and the one that maps to our strategy would be ‘font-display: optional’.

Unfortunately, the adoption of font-display among browsers is not widespread, as it is relatively new. So for now, until the adoption becomes mainstream, we came up with a solution that leverages the localStorage, FontFaceSet APIs and the Font Face Observer utility (as a backup if the FontFaceSet API is not present).

The below illustration gives an overview of how it works:

Flow diagram for Font Loader

To summarize,

  • When users visit an eBay web page, we add a tiny inline CSS and JavaScript snippet in the response HTML <head> tag. We also include a small JavaScript snippet in the footer HTML that incorporates the font loader logic.
  • The JavaScript in the <head> checks the localStorage if a font flag is set. If the flag is set, it immediately adds the CSS class to document root to enable the custom font. The page renders with the “Market Sans” custom font. This is the happy path.
  • The JavaScript in the footer again checks the localStorage for a font flag. If it is NOT set, it calls the font loader function on the document load event.
  • The font loader function loads (downloads) the custom fonts either using the built-in FontFaceSet API (if present) or through the Font Face Observer utility. The Font Face Observer is asynchronously downloaded on demand.
  • Once the font download is complete, a font flag is set on the localStorage. One thing to note — even though the font flag is set, we do not update the current view with the custom font. It is done on the next page visit with Step 2 above kicking in.

We have open sourced this module as ebay-font. It is a small utility that works along with other eBay open source modules Skin, Marko, and Lasso, as well as in standalone mode. We hope others can benefit from it.

Tradeoffs

There are a couple of tradeoffs with this strategy:

  1. First time users: A new user visiting eBay for the first time will get the system font. On navigation or subsequent visits they will get our custom font. This is acceptable, as we have to start the custom font at some point, and we will start it from the second visit of a new user.
  2. Private or incognito mode: When a user starts a new browsing session in private or incognito mode, they get the system font initially. But subsequent browsing in the same session will render the custom font (Safari is an exception, but it is getting fixed). We do not have metrics on how many users fall under this category, but this is something we have to live with.
  3. Cache eviction: In certain rare scenarios we observed that the custom font entity in the browser cache is evicted, but the localStorage entry is still present. Probably browsers clean up cache more frequently than localStorage. In these scenarios, users will experience a FOIT or FOUT based on the browser. This is more of an edge case and hence less concerning.

As a team we agreed that these tradeoffs are acceptable, considering the unpredictable behavior that comes with default font loading.

Conclusion

Custom web fonts do add value to the overall user experience, but it should not be at the cost of delaying the critical content. Each organization should have a font loading strategy based on their application needs. The new built-in CSS property ‘font-display’ makes it very easy to choose one. We should start using it right away, even if the support is minimal and even if there is already an in-house implementation.

Huge thanks to my colleague Raja Ramu for partnering on this effort and help in open sourcing the module ebay-font.

—  Senthil Padmanabhan

Building a UI Component in 2017 and Beyond

As web developers, we have seen the guidelines for building UI components evolve over the years. Starting from jQuery UI to the current Custom Elements, various patterns have emerged. To top it off, there are numerous libraries and frameworks, each advocating their own style on how a component should be built. So in today’s world, what would be the best approach in terms of thinking about a UI component interface? That is the essence of this blog. Huge thanks to folks mentioned in the bottom Acknowledgments section. Also, this post leverages a lot of learnings from the articles and projects listed in the Reference section at the end.

Setting the context

Before we get started, let’s set the context on what this post covers.

  • By UI components, we mean the core, standalone UI patterns that apply to any web page in general. Examples would be Button, Dropdown Menu, Carousel, Dialog, Tab, etc. Organizations in general maintain a pattern library for these core components. We are NOT talking about application-specific components here, like a Photo Viewer used in social apps or an Item Card used in eCommerce apps. They are designed to solve application-specific (social, eCommerce, etc.) use cases. Application components are usually built using core UI components and are tied with a JavaScript framework (Vue.js, Marko, React, etc.) to create a fully featured web page.

  • We will only be talking about the interface (that is, API) of the component and how to communicate. We will not go over the implementation details in depth, just a quick overview.

Declarative HTML-based interface

Our fundamental principle behind building a UI component API is to make it agnostic in regard to any library or framework. This means the API should be able to work in any context and be framework-interoperable. Any popular framework can just leverage the interface out of the box and augment it with their own implementation. With this principle in mind, what would be the best way to get started? The answer is pretty simple — make the component API look like how any other HTML element would look like. Just like how a <div>, <img>, or a <video> tag would work. This is the idea behind having a declarative HTML-based interface/API.

So what does a component look like? Let’s take carousel as an example. This is what the component API will look like.

<carousel index="2" controls aria-label-next="next" aria-label-previous="previous" autoplay>
    <div>Markup for Item 1</div>
    <div>Markup for Item 2</div>
    ...
</carousel>

What does this mean? We are declaratively telling the component that the rendered markup should do the following.

  • Start at the second item in the carousel.
  • Display the left and right arrow key controls.
  • Use "previous" and "next" as the aria-label attribute values for the left and right arrow keys.
  • Also, autoplay the carousel after it is loaded.

For a consumer, this is the only information they need to know to include this component. It is exactly like how you include a <button> or <canvas> HTML element. Component names are always lowercase. They should also be hyphenated to distinguish them from native HTML elements. A good suggestion would be to prefix them with a namespace, usually your organization or project name, for example ebay-, core-, git-, etc.

Attributes form the base on how you will pass the initial state (data and configuration) to a component. Let’s talk about them.

Attributes

  • An attribute is a name-value pair where the value is always a string. Now the question may arise that anything can be serialized as a string and the component can de-serialize the string to the associated data type (JSON for example). While that is true, the guideline is to NOT do that. A component can only interpret the value as a String (which is default) or a Number (similar to tabindex) or a JavaScript event handler (similar to DOM on-event handlers). Again, at the end of the day, this is exactly how an HTML element works.

  • Attributes can also be boolean. As per the HTML5 spec, “The presence of a boolean attribute on an element represents the true value, and the absence of the attribute represents the false value.” This means that as a component developer, when you need a boolean attribute, just check for the presence of it on the element and ignore the value. Having a value for it has no significance; both creator and consumer of a component should follow the same. For example, <button disabled="false"> will still disable the button, even if it is set to false, just because the boolean attribute disabled is present.

  • All attribute names should be lowercase. Camel case or Pascal case is NOT allowed. For certain multiword attributes, hyphenated names like accept-charset, data-*, etc. can be used, but that should be a rare occurrence. Even for multiwords, try your best to keep them as one lowercase name, for example, crossorigin, contenteditable, etc. Check out the HTML attribute reference for tips on how the native elements are doing it.

We can correlate the above attribute rules with our <carousel> example.

  • aria-label-next and aria-label-previous as string attributes. We hyphenate them as they are multiwords, very similar to the HTML aria-label attribute.
  • index attribute will be deserialized as a number, to indicate the position of the item to be displayed.
  • controls and autoplay will be considered as boolean attributes.

A common pattern that used to exist (or still exists) is to pass configuration and data as JSON strings. For our carousel, it would be something like the following example.

<!-- This is not recommended -->
<carousel 
    data-config='{"controls": true, "index": 2, "autoplay": true, "ariaLabelNext": "next", "ariaLabelPrevious": "previous"}' 
    data-items='[{"title":"Item 1", ..}, {"title": "Item 2", ...}]'>
</carousel>

This is not recommended.

Here the component developer reads the data attribute data-config, does a JSON parse, and then initializes the component with the provided configuration. They also build the items of the carousel using data-items. This may not be intuitive, and it works against a natural HTML-based approach. Instead consider a declarative API as proposed above, which is easy to understand and aligns with the HTML spec. Finally, in the case of a carousel, give the component consumers the flexibility to build the carousel items however they want. This decouples a core component from the context in which it is going to be used, which is usually application-specific.

Array-based

There will be scenarios where you really need to pass an array of items to a core component, for example, a dropdown menu. How to do this declaratively? Let’s see how HTML does it. Whenever any input is a list, HTML uses the <option> element to represent an item in that list. As a reference, check out how the <select> and <datalist> elements leverage the <option> element to list out an array of items. Our component API can use the same technique. So in the case of a dropdown menu, the component API would look like the following.

<dropdown-menu list="options" index="0">
    <option value="0" selected>--Select--</option>
    <option value="1">Option 1</option>
    <option value="2">Option 2</option>
    <option value="3">Option 3</option>
    <option value="4">Option 4</option>
    <option value="5">Option 5</option>
</dropdown-menu>

It is not necessary that we should always use the <option> element here. We could create our own element, something like <dropdown-option>, which is a child of the <dropdown-menu> component, and customize it however we want. For example, if you have an array of objects, you can represent each object ({"userName": "jdoe", "score": 99, "displayName": "John Doe"}) declaratively in the markup as <dropdown-option value="jdoe" score="99">John Doe</dropdown-option>. Hopefully you do not need a complex object for a core component.

Config-based

You may also argue that there is a scenario where I need to pass a JSON config for it to work or else usability becomes painful. Although this is a rare scenario for core components, a use case I can think about will be a core analytics component. This component may not have a UI, but it does all tracking related stuff, where you need to pass in a complex JSON object. What do we do? The AMP Project has a good solution for this. The component would look like the following.

<analytics>
    <script type="application/json">
    {
      "requests": {
        "pageview": "https://example.com/analytics?pid=",
        "event": "https://example.com/analytics?eid="
      },
      "vars": {
        "account": "ABC123"
      },
      "triggers": {
        "trackPageview": {
          "on": "visible",
          "request": "pageview"
        },
        "trackAnchorClicks": {
          "on": "click",
          "selector": "a",
          "request": "event",
          "vars": {
            "eventId": "42",
            "eventLabel": "clicked on a link"
          }
        }
      }
    }
    </script>
</analytics>

Here again we piggyback the interface based on how we would do it in simple HTML. We use a <script> element inside the component and set the type to application/json, which is exactly what we want. This brings back the declarative approach and makes it look natural.

Communication

Till now we talked only about the initial component API. This enables consumers to include a component in a page and set the initial state. Once the component is rendered in the browser, how do you interact with it? This is where the communication mechanism comes into play. And for this, the golden rule comes from the reactive principles of

Data in via attributes and properties, data out via events

This means that attributes and properties can be used to send data to a component and events send the data out. If you take a closer look, this is exactly how any normal HTML element (input, button, etc.) behaves. We already discussed attributes in detail. To summarize, attributes set the initial state of a component, whereas properties update or reflect the state of a component. Let’s dive into properties a bit more.

Properties

At any point in time, properties are your source of truth. After setting the initial state, some attributes do not get updated as the component changes over time. For example, typing in a new phrase in an input text box and then calling element.getAttribute('value') will produce the previous (stale) value. But doing element.value will always produce the current typed-in phrase. Certain attributes, like disabled, do get reflected when the corresponding property is changed. There has always been some confusion around this topic, partly due to legacy reasons. It would be ideal for attributes and properties to be in sync, as the usability benefits are undeniable.

If you are using Custom Elements, implementing properties is quite straightforward. For a carousel, we could do this.

class Carousel extends HTMLElement {  
    static get observedAttributes() {
        return ['index'];
    }
    // Called anytime the 'index' attribute is changed
    attributeChangedCallback(attrName, oldVal, newVal) {
        this[attrName] = newVal;
    }
    // Takes an index value
    set index(idx) {
        // First check if it is numeric
        const numericIndex = parseInt(idx, 10);
        if (isNaN(numericIndex)) {
            return;   
        }
        // Update the internal state
        this._index = numericIndex;
        /* Perform the associated DOM operations */
        moveCarousel();
    }
    get index() {
        return this._index;
    }
}

Here the index property gets all its associated characteristics. If you are doing carouselElement.index=4, it will update the internal state and then perform the corresponding DOM operations to move the carousel to the fourth item. Additionally, even if you directly update the attribute carouselElement.setAttribute('index', 4), the component will still update the index property, the internal state and perform the exact DOM operations to move the carousel to the fourth item.

However, until Custom Elements gain massive browser adoption and have a good server-side rendering story, we need to come up with other mechanisms to implement properties. And one way would be to use the Object.defineProperty() API.

const carouselElement = document.querySelector('#carousel1');
Object.defineProperty(carouselElement, 'index', {
    set(idx) {
        // First check if it is numeric
        const numericIndex = parseInt(idx, 10);
        if (isNaN(numericIndex)) {
            return;   
        }
        // Update the internal state
        this._index = numericIndex;
        /* Perform the associated DOM operations */
        moveCarousel();        
    },
    get() {
        return this._index;
    }
});

Here we are augmenting the carousel element DOM node with the index property. When you do carouselElement.index=4, it gives us the same functionality as the Custom Element implementation. But directly updating an attribute with carouselElement.setAttribute('index', 4) will do nothing. This is the tradeoff in this approach. (Technically we could still use a MutationObserver to achieve the missing functionality, but that would be an overkill.) Hopefully as a team, if you can standardize that state updates should only happen through properties, then it should be less of a concern.

With respect to naming conventions, since properties are accessed programmatically, they should always be camel-cased. All exposed attributes (an exception would be ARIA attributes) should have a corresponding camel-cased property, very similar to native DOM elements.

Events

When the state of a component has changed, either programmatically or due to user interaction, it has to communicate the change to the outside world. And the best way to do it is by dispatching events, very similar to click or touchstart events dispatched by a native HTML element. The good news is that the DOM comes with a built-in custom eventing mechanism through the CustomEvent constructor. So in the case of a carousel, we can tell the outside world that the carousel transition has been completed by dispatching a transitionend event as shown below.

const carouselElement = document.querySelector('#carousel1');

// Dispatching 'transitionend' event
carouselElement.dispatchEvent(new CustomEvent('transitionend', {
    detail: {index: this._index}
}));

// Listening to 'transitionend' event
carouselElement.addEventListener('transitionend', event => {
    alert(`User has moved to item number ${event.detail.index}`);
});

By doing this, we get all the benefits of DOM events like bubbling, capture etc. and also the event APIs like event.stopPropagation(), event.preventDefault(), etc. Another added advantage is that it makes the component framework-agnostic, as most frameworks already have built-in mechanisms for listening to DOM events. Check out Rob Dodson’s post on how this works with major frameworks.

Regarding a naming convention for events, I would go with the same guidelines that we listed above for attribute names. Again, when in doubt, look at how the native DOM does it.

Implementation

Let me briefly touch upon the implementation details, as they give the full picture. We have been only talking about the component API and communication patterns till now. But the critical missing part is that we still need JavaScript to provide the desired functionality and encapsulation. Some components can be purely markup- and CSS-based, but in reality, most of them will require some amount of JavaScript. How do we implement this JavaScript? Well, there a couple of ways.

  • Use vanilla JavaScript. Here the developer builds their own JavaScript logic for each component. But you will soon see a common pattern across components, and the need for abstraction arises. This abstraction library will pretty much be similar to those numerous frameworks out in the wild. So why reinvent the wheel? We can just choose one of them.

  • Usually in organizations, web pages are built with a particular library or framework (Angular, Ember, Preact, etc.). You can piggyback on that library to implement the functionality and provide encapsulation. The drawback here is that your core components are also tied with the page framework. So in case you decide to move to a different framework or do a major version upgrade, the core components should also change with it. This can cause a lot of inconvenience.

  • You can use Custom Elements. That would be ideal, as it comes default in the browsers, and the browser makers recommend it. But you need a polyfill to make it work across all of them. You can try a Progressive Enhancement technique as described here, but you would lose functionality in non-supportive browsers. Moreover, until we have a solid and performant server-side rendering mechanism, Custom Elements would lack mass adoption.

And yes, all options are open-ended. It all boils down to choices, and software engineering is all about the right tradeoffs. My recommendation would be to go with either Option 2 or 3, based on your use cases.

Conclusion

Though the title mentions the year “2017”, this is more about building an interface that works not only today but also in the future. We are making the component API-agnostic of the underlying implementation. This enables developers to use a library or framework of their choice, and it gives them the flexibility to switch in the future (based on what is popular at that point in time). The key takeaway is that the component API and the principles behind it always stay the same. I believe Custom Elements will become the default implementation mechanism for core UI components as soon as they gain mainstream browser adoption.

The ideal state is when a UI component can be used in any page, without the need of a library or polyfill and it can work with the page owner’s framework of choice. We need to design our component APIs with that ideal state in mind and this is a step towards it. Finally, worth repeating, when in doubt, check how HTML does it, and you will probably have an answer.

Acknowledgments

Many thanks to Rob Dodson and Lea Verou for their technical reviews and valuable suggestions. Also huge thanks to my colleagues Ian McBurnie, Arun Selvaraj, Tony Topper, and Andrew Wooldridge for their valuable feedback.

References

Experience the Lightning Bolt

 

Three months back we announced how we are transforming the shopping experience at eBay, enabling our users to browse with style and speed. Our goal was to provide an engaging experience not only to users who are within the eBay site, but also to mobile users accessing eBay from external platforms like Google and Twitter. This is where AMP technology comes into play. We implemented an AMP version for our new product browse experience, along with the regular mobile web pages, and launched them in June. At that time we did not make our AMP content discoverable to Google, as we had a few pending tasks to be completed. Also, AMP links surfaced in Google search results only for publisher-based content and not for eCommerce.

Things have changed now. Google announced that they are opening AMP beyond the news industry to include eCommerce, travel and so on. From our end, we wrapped up the pending items and linked the AMP pages from non-AMP pages to make them discoverable. Today we are happy to announce that users around the globe will start seeing eBay AMP links in Google search results and experience the lightning bolt — instant loading. We have close to 15 million AMP-based product browse pages, but not all will appear as AMP right away. This feature is being ramped up and will eventually surface. Check out some of the popular queries in a mobile browser — “iPhone 6 no contract” and “canon digital cameras,” for example. The AMP lightning bolt appears next to links as an indication. AMP for eCommerce is now a reality.

eBay AMP Product Browse Page eBay AMP link in Google search results (left); eBay AMP product browse page (right)

Between now and then

Following our initial launch in June, we did a couple of things to make AMP ready for prime time. We outline a few these efforts here.

Robust analytics system

Understanding how users interact with our pages is critical for us to provide the most optimized experience. The back-end system that powers the new product browse experience is designed in such a way that it constantly collects users’ screen activity, learns from it, and optimizes the experience for subsequent visits. For example, if users interact more often with a module that appears below the fold in the screen, then in future visits to the same browse page, that module will start appearing above the fold. Our non-AMP page has a custom analytics library that does the reporting to the back end.

AMP has a component (amp-analytics) for doing this. In our initial AMP launch, we used this component just to track page impressions. It provides a fairly exhaustive tracking mechanism. But what we wanted was more granular control at an element level, where each element dictates what it wants to track. We started working with the AMP team on this and came up with a spec. We went ahead implemented the spec and contributed it back to the open-source project. With the implementation in place, we were able to achieve a robust and advanced analytic system that reports user interactions like click, scroll, and visibility to our backend, which in turn optimizes the subsequent visits.

Feature parity

We mentioned in our previous blog that most of the code is shared between the AMP and non-AMP pages. Even with this code sharing, there were still small feature inconsistencies between the two versions. We closed these gaps, fixed the inconsistencies, and put a process in place to make sure they do not creep in. Having said that, there were certain UI components and behaviors that we were not able to achieve in the AMP version due to restrictions. Some of these components are eCommerce-specific. We are working with the AMP team to add them to the component list so everyone can benefit. A good example would be tabbed UI component, and there is already a feature request to get this implemented.

Streamlined build process

During the initial launch, we put manual effort into managing assets (CSS and JavaScript) between the AMP and non-AMP versions. In the AMP version, there should be no JavaScript, and all CSS should be inline, whereas in the non-AMP version, both CSS and JavaScript should be bundled and externalized. Doing this manually was not ideal. Our asset pipeline tool, Lasso, had a solution for this —  conditional dependencies. We created an AMP flag that gets initialized to true if the request is AMP and then set as a Lasso flag. The pipeline gets access to it and automatically bundles, externalizes, or inlines resources based on the conditions. This was a big time saver and ended up being very efficient.

The road ahead

We are not done yet; in fact, we are just getting started. We have a bunch of tasks lined up.

  • Beyond AMP — We know AMP pages are fast. But what about the subsequent pages the user visits? Currently when users click on a link in the AMP page, a new tab opens, and the destination page is loaded there. In our case, the mobile web version of the destination page is loaded. We want that experience also to be as fast and consistent as the AMP experience. There is an AMP component (amp-install-serviceworker) to achieve this goal, and our top priority is to leverage this utility and create a seamless transition from the AMP to the target page. We are also discussing with the Google team about how to avoid the new tab and continue the experience in the same window.
  • Cache freshness — AMP content is served from Google AMP cache, and the cache update policy can be found here. What this means to eBay is, for popular product queries, users always see fresh content. But for certain extremely rare queries, a few users may end up seeing stale content. While this is not a common scenario, there is an AMP component (amp-fresh) in the works to fix this. We will be integrating this component as soon as it is ready. In the meanwhile, we have a script that we manually run for a few products to update the AMP content in cache.
  • Unified version — Currently we have two versions of the new browse pages — AMP and non-AMP. The AMP version shows up to users searching in Google, and the non-AMP version shows up to users searching within eBay. Although both of them are highly optimized, look the same, and share most of the code, updating both versions is still a maintenance overhead. In addition, we always need to watch out for feature parity. In the future, based on how AMP pages are performing, we may choose to have one mobile version (AMP) and serve it to all platforms.

We are very excited to provide the AMP experience to our mobile users coming from Google. We have been playing with it for a while, and it is indeed lightning fast. Mobile browsing can be slow and sometimes frustrating, and this is where AMP comes in and guarantees a consistent and fast experience. We hope our users benefit from this new technology.

Senthil Padmanabhan | Principal Engineer at eBay