Category Archives: Developer Tools

Platforms, frameworks, utilities, etc. to support and enhance developer productivity

Enhancing the User Experience of the Hadoop Ecosystem

  At eBay, we have multiple large, multi-tenant clusters. Each of these clusters stores hundreds of petabytes of data. These clusters offer tens of thousands of cores to run computations on the data. We have thousands of internal users who use Hadoop in their roles, including data analysts, data scientists, engineers, and product managers. These
Continue Reading »

The API Journey: Or How We Built a New Family of Modern Services

  An API — or application programming interface — is an intermediary that enables applications to interact. It is a contract that specifies how applications talk to one another. Further, an API creates a separation between a service provider and its consumers. Essentially, it decouples their implementations. As long as the contract stays intact, API providers may continue changing
Continue Reading »

Scalable and Nimble Continuous Integration for Hadoop Projects

  Experimentation The Experimentation Platform at eBay runs around 1500 experiments that are responsible for processing over hundreds of terabytes of reporting data contained in millions of files using a 2500+ node Hadoop infrastructure and consuming thousands of computing resources. The entire report generation process contains well over 200 metrics. It enables millions of customers to experience small and
Continue Reading »