eBay Tech Blog

From the category archives:

Data Infrastructure and Services

Oink : Making Pig Self-Service

by Ruchir Shah July 22, 2014 Cloud

The Platform and Infrastructure team at eBay Inc. is happy to announce the open-sourcing of Oink – a self-service solution to Apache Pig. Pig and Hadoop overview Apache Pig is a platform for analyzing large data sets. It uses a high-level language for expressing data analysis programs, coupled with the infrastructure for evaluating these programs. Pig [...]

0 comments Read the full article →

Using Spark to Ignite Data Analytics

by eBay Global Data Infrastructure Analytics Team May 28, 2014 Data Infrastructure and Services

At eBay we want our customers to have the best experience possible. We use data analytics to improve user experiences, provide relevant offers, optimize performance, and create many, many other kinds of value. One way eBay supports this value creation is by utilizing data processing frameworks that enable, accelerate, or simplify data analytics. One such [...]

2 comments Read the full article →

Delivering eBay’s CI Solution with Apache Mesos – Part II

by The eBay PaaS Team May 12, 2014 Cloud

In part I of this post we laid out in detail how to run a large Jenkins CI farm in Mesos. In this post we explore running the builds inside Docker containers and more: Explain the motivation for using Docker containers for builds. Show how to handle the case where the build itself is a [...]

19 comments Read the full article →

Delivering eBay’s CI Solution with Apache Mesos – Part I

by The eBay PaaS Team April 4, 2014 Cloud

Problem statement In eBay’s existing CI model, each developer gets a personal CI/Jenkins Master instance. This Jenkins instance runs within a dedicated VM, and over time the result has been VM sprawl and poor resource utilization. We started looking at solutions to maximize our resource utilization and reduce the VM footprint while still preserving the [...]

7 comments Read the full article →

Fine-Tuning the eBay Technical Infrastructure, Part 2: Our Q1 Digital Service Efficiency Results

by Sri Shivananda May 24, 2013 Data Infrastructure and Services

As discussed in a previous post, earlier this year we unveiled the Digital Service Efficiency (DSE) methodology, our miles-per-gallon (MPG) equivalent for viewing the productivity and efficiency of our technical infrastructure across four key areas: performance, cost, environmental impact, and revenue. The goal of releasing DSE was to provide a transparent view of how the [...]

2 comments Read the full article →

Fine-Tuning the eBay Technical Infrastructure: A New Methodology

by Dean Nelson March 5, 2013 Data Infrastructure and Services

I’ve been with eBay Inc. since 2009, and even just in the past four years, I’ve witnessed a significant change in the growth and importance of data centers, and the way we, as an industry, measure their efficiency. The Green Grid’s Power Usage Effectiveness (PUE) metric – as well as the water and carbon equivalents [...]

2 comments Read the full article →

Validating Hadoop Platform Releases

by Mitch Wyle October 17, 2012 Cloud

Intern Pralay Biswas submitted the following blog post at the conclusion of his summer with eBay. Pralay was one of 120 college interns welcomed into eBay Marketplaces this summer. In pioneer days, they used oxen for heavy pulling, and when one ox couldn’t budge a log, they didn’t try to grow a larger ox. We [...]

3 comments Read the full article →

Cassandra Data Modeling Best Practices, Part 2

by Jay Patel August 14, 2012 Data Infrastructure and Services

In the first part, we covered a few fundamental practices and walked through a detailed example to help you get started with Cassandra data model design. You can follow Part 2 without reading Part 1, but I recommend glancing over the terms and conventions I’m using. If you’re new to Cassandra, I urge you to [...]

13 comments Read the full article →

Cassandra Data Modeling Best Practices, Part 1

by Jay Patel July 16, 2012 Data Infrastructure and Services

This is the first in a series of posts on Cassandra data modeling, implementation, operations, and related practices that guide our Cassandra utilization at eBay. Some of these best practices we’ve learned from public forums, many are new to us, and a few still are arguable and could benefit from further experience. In this part, [...]

44 comments Read the full article →
Copyright © 2011 eBay Inc. All Rights Reserved - User Agreement - Privacy Policy - Comment Policy