Monthly Archives: April 2011

Items and Products

You often hear that the difference between eBay and other e-Commerce sites is that eBay sells items, other sites sell products. What does that mean?

Here’s an example. On eBay, a seller lists a specific item, for example a tackle box titled “PLANO MAGNUM HIP ROOF SIX TRAY STORAGE BOX 8616 NEW”. When someone else lists “Plano Hip Roof with 6 Trays”, is it the same product? eBay doesn’t offer much help in answering this question. On Amazon, a Plano Hip Roof has a product number B001ECQPQG assigned by Amazon (an ASIN). Everyone selling this product will be assigned that same ASIN of B001ECQPQG.

So on an abstract level, it’s easy to explain the difference between an item and a product. A product is a way of telling whether two items are the same. Suppose seller A on eBay is offering item X and seller B is offering item Y. You purchase both of them. When they arrive, you throw away the shipping containers they arrived in. If you now can’t tell which is X and which is Y, then these items are the same product.

As mentioned before, Amazon distinguishes different products by assigning each item an ASIN. They are the same product if and only if they have the same ASIN. The ASIN is somewhat like a checksum on an item: it quickly distinguishes whether two items are the same.

The ASIN system is proprietary to Amazon. What can other e-tailers like eBay do? They can employ the widely used system of ISBN for books and UPC (universal product codes) for everything else. So it seems like the product problem is really rather easy: simply use ISBN/UPC as the checksum that distinguishes products.

Well, it’s not that simple for two reasons. First, the question of when two items are the same product is slippery. For example, is the first and second printing of a technical book the same product? The second printing often has errors corrected, but still has the same ISBN. What about a product that changes its packaging? Or Coke made with high fructose corn syrup vs. sugar? So maybe ISBN and UPC don’t precisely define a product.

An even bigger problem is that even if a product has a UPC, it might not be that easy to find out what it is. Here’s an example: I recently purchased a black Kenmore dishwasher, model 1374. I couldn’t find a UPC anywhere on the dishwasher or in the documentation that came with it, so I headed to the web. A web search finds two UPCs for the 1374:

  • 13743 883049019642 (stainless steel)
  • 13742 883049006161 (white)

The extra number at the end is the color: ‘3’ for stainless steel, ‘2’ for white. As expected, the different colors have different UPCs. That’s consistent with our definition: if you received a white dishwasher in one box and a stainless steel one in another box, you could certainly tell them apart!

But I purchased a black 1374, which is 13749. I was unable to find any UPCs on bing. Google had a single hit, to a document that claimed the UPC was 13749 883049006178 (black). Should I believe a UPC that has only 1 hit in Google, especially for a product that is one of the best-selling dishwasher models? As an aside, this gives a googlewhack, which is a pair of search terms that returns a single result in Google. The googlewhack is

13749 883049006178

Another example is the cologne “L’Homme by Yves Saint Laurent Eau De Toilette Spray 2 oz”: Go to Google, click on “Shopping” and search for that title. Click on “Details” and it returns the UPC as 556779982028 and 885892085119. Which (if any) is correct? You can try a UPC validation website,

It has a hit on 885892085119 but not on 556779982028. That suggests the 885 is correct. But go to Amazon and search for “Lhomme by Yves Saint Laurent Eau De Toilette Spray 2 oz Men” (you may have to scroll down in the search results to get this: skip pass the La Nuit De L’homme items). It gives the UPC as 556779982028. Which is correct — the 885 or the 556?

These examples should convince you that although the concept of a product seems simple, it’s anything but. UPCs (and ISBN) at first appear to be an easy way to find the product for an item. But when looking more closely, we discover that we can’t reliably find the UPC, even for common products. It may not be on the product itself, nor reliably available on the web. I hope to return in a later blog posting to talk about some attacks on the problem of associating an item with a product.

Using Behavioral Data to Improve Search

Since 2006, “Best Match” has been the default sort order for users on eBay.  It is designed to balance the many objectives that users have on the site–for example finding relevant items to their query, scoring great deals, finding timely auctions, and locating items by reliable sellers.  For users that want to emphasize one of these areas over others, there are other sort options and eBay’s Advanced Search function, but we feel it’s important to help users have a good experience even without exerting a lot of effort or customization.  As Best Match has evolved over time, it has incorporated more and more information to help it distinguish relevant items (more interesting, more sales worthy, more attractive) from irrelevant ones.  In the past 18 months since we founded the Search Science team, we’ve made substantial strides.  For example, here are snapshots of the results for the query “plasma tv” using the version of Best Match that was used on the site 18 months ago versus the same query using today’s Best Match:

The differences are striking, with no actual TVs in the top 10 results in the old version, compared to a number of TVs at good price points in the new version.  So what is the driver behind these improvements?  One of the biggest differences between these two versions is in how we are using behavioral data.  By behavioral data, I mean observations about how our users behave when shown a listing.  Do they click on it?  Do they place a bid?  Do they Buy It Now?  All of us have seen the “wisdom of crowds” in action.  The best restaurants are frequently crowded and always have a wait, and we’re suspicious if a restaurant is empty in a hot neighborhood on Friday evening.  Similarly, when users have found an item interesting in the past, by clicking on it, bidding on it, or purchasing it, we expect that tells us something about how relevant that item might be to other users.

Best Match has used behavioral data for several years, but in the last 18 months, we’ve learned a lot about how to interpret and clean that data before using it for ranking.  For sales and click data, there is one obvious bias that has been well-studied by search engines before: position bias.  Position bias means the items that appear in the top positions of a search result page are far more likely to be clicked and purchased by users.  Left unchecked, this bias makes it appear that the listings at the top of the search are more relevant, because they’re clicked more frequently than lower results.  This can result in a feedback loop when you use clicks and sales in your ranking function:  an item ranks highly, so it gets more clicks, so it ranks even higher, and gets still more clicks, and . . . well, you get the point.

There are several ways to reduce this bias, the simplest of which is to normalize each click by its Expected Clicks based on its position.  Let’s look at an example.  Let’s assume we live in a world where users on average click on the top-ranked listing 10% of the time, the second-highest listing 9% of the time, and so on until the 10th-best listing, which is clicked on 1% of the time.  Now let’s assume we want to rank a listing which has been clicked on 20% of the time.  If all of those clicks came while the item was in the top position, where Expected Clicks were 10%, we’d say this item performed 2x (20% actual / 10% expected) better than expected.  If on the other hand the item got all its clicks while it was in the 10th position, we’d say it performed 20x (20% actual / 1% expected) better than expected.  You’ll notice that items that are ranked lower get more credit for their clicks–this counterbalances the feedback loop we described above and makes sure all items have an opportunity to improve their ranking if they perform better than expected.

Another bias in behavioral data that is less-well studied in academic literature is a price bias.  Basically, it’s easier for people to buy cheap things than it is to buy expensive things.  If I showed you a fantastic #2 pencil and offered it to you for the great price of 5 cents, how long would it take for you to consider buying it?  Maybe 10 seconds?  Now if I showed you a fantastic plasma TV and offered it to you for the great price of $500, how long would it take to consider buying it?  Well, that’s a lot of money…maybe you should look at some reviews, or talk to that friend of yours who knows a lot about TVs, or do some price comparisons.  It might take you days, weeks, even months to re-assure yourself that you’re using your money wisely.

Let’s get back to behavioral data.  Since pencils sell faster than plasma TVs, is the pencil a better item than the TV?  No, we said above that they’re both fantastic and they’re both offered at a great price.  So we don’t want to penalize the TV just because it has a naturally slower selling rate.  And we definitely don’t want to always show pencils before TVs in our rankings–what if the user searched specifically for “plasma tv”?  We need to correct for this price bias in our sales data.  We can apply the same principle we did for position bias, and figure out the Expected Sales for an item of price X based on historical data.

These two changes (correcting position bias and correcting price bias) combined have led to significant improvements in the relevance of Best Match, and more items are selling on eBay as a result.  We’re experimenting and learning every day about how to better use the feedback our users are already giving us–what they choose to click on and what they choose to buy. Stay tuned for more information about our efforts to enhance eBay’s search relevance.