eBay Tech Blog

From the category archives:

Data Infrastructure and Services

Announcing Kylin: Extreme OLAP Engine for Big Data

by eBay Inc. Kylin Team October 20, 2014 Cloud

We are very excited to announce that eBay has released to the open-source community our distributed analytics engine: Kylin (http://kylin.io). Designed to accelerate analytics on Hadoop and allow the use of SQL-compatible tools, Kylin provides a SQL interface and multi-dimensional analysis (OLAP) on Hadoop to support extremely large datasets. Kylin is currently used in production [...]

8 comments Read the full article →

NoSQL Data Modeling

by Donovan Hsieh October 10, 2014 Data Infrastructure and Services

Data modeling for RDBMS has been a well-defined discipline for many years. Techniques like logical to physical mapping and normalization / de-normalization have been widely practiced by professionals, including novice users. However, with the recent emergence of NoSQL databases, data modeling is facing new challenges to its relevance. Generally speaking, NoSQL practitioners focus on physical [...]

5 comments Read the full article →

Quality of Service in Hadoop

by Chris Li August 21, 2014 Data Infrastructure and Services

At eBay we run Hadoop clusters comprising thousands of nodes that are shared by thousands of users. We analyze data on these clusters to gain insights for improved customer experience. In this post, we look at distributing RPC resources fairly between heavy and light users, as well as mitigating denial of service attacks within Hadoop. [...]

4 comments Read the full article →

Oink : Making Pig Self-Service

by Ruchir Shah July 22, 2014 Cloud

The Platform and Infrastructure team at eBay Inc. is happy to announce the open-sourcing of Oink – a self-service solution to Apache Pig. Pig and Hadoop overview Apache Pig is a platform for analyzing large data sets. It uses a high-level language for expressing data analysis programs, coupled with the infrastructure for evaluating these programs. Pig [...]

1 comment Read the full article →

Using Spark to Ignite Data Analytics

by eBay Global Data Infrastructure Analytics Team May 28, 2014 Data Infrastructure and Services

At eBay we want our customers to have the best experience possible. We use data analytics to improve user experiences, provide relevant offers, optimize performance, and create many, many other kinds of value. One way eBay supports this value creation is by utilizing data processing frameworks that enable, accelerate, or simplify data analytics. One such [...]

4 comments Read the full article →

Delivering eBay’s CI Solution with Apache Mesos – Part II

by The eBay PaaS Team May 12, 2014 Cloud

In part I of this post we laid out in detail how to run a large Jenkins CI farm in Mesos. In this post we explore running the builds inside Docker containers and more: Explain the motivation for using Docker containers for builds. Show how to handle the case where the build itself is a [...]

24 comments Read the full article →

Delivering eBay’s CI Solution with Apache Mesos – Part I

by The eBay PaaS Team April 4, 2014 Cloud

Problem statement In eBay’s existing CI model, each developer gets a personal CI/Jenkins Master instance. This Jenkins instance runs within a dedicated VM, and over time the result has been VM sprawl and poor resource utilization. We started looking at solutions to maximize our resource utilization and reduce the VM footprint while still preserving the [...]

14 comments Read the full article →

Fine-Tuning the eBay Technical Infrastructure, Part 2: Our Q1 Digital Service Efficiency Results

by Sri Shivananda May 24, 2013 Data Infrastructure and Services

As discussed in a previous post, earlier this year we unveiled the Digital Service Efficiency (DSE) methodology, our miles-per-gallon (MPG) equivalent for viewing the productivity and efficiency of our technical infrastructure across four key areas: performance, cost, environmental impact, and revenue. The goal of releasing DSE was to provide a transparent view of how the [...]

2 comments Read the full article →

Fine-Tuning the eBay Technical Infrastructure: A New Methodology

by Dean Nelson March 5, 2013 Data Infrastructure and Services

I’ve been with eBay Inc. since 2009, and even just in the past four years, I’ve witnessed a significant change in the growth and importance of data centers, and the way we, as an industry, measure their efficiency. The Green Grid’s Power Usage Effectiveness (PUE) metric – as well as the water and carbon equivalents [...]

2 comments Read the full article →

Validating Hadoop Platform Releases

by Mitch Wyle October 17, 2012 Cloud

Intern Pralay Biswas submitted the following blog post at the conclusion of his summer with eBay. Pralay was one of 120 college interns welcomed into eBay Marketplaces this summer. In pioneer days, they used oxen for heavy pulling, and when one ox couldn’t budge a log, they didn’t try to grow a larger ox. We [...]

3 comments Read the full article →
Copyright © 2011 eBay Inc. All Rights Reserved - User Agreement - Privacy Policy - Comment Policy