/Scale

How Live Is Your Stream? Measure Live Latency At Scale With Mux Data

tl;dr: Beta test the new HLS Live Stream Latency metric in Mux Data for free! Understand the live streaming experience, and find opportunities to improve.

featured in #308

Bottleneck #02: Talent

- Tim Cochran Roni Smith tl;dr: Common tech debt bottlenecks by companies entering growth. The authors cover the common signs you are approaching a scaling bottleneck i.e. frustration from employees, stretching to hit deadlines, dependency on people and more. And strategies on how to get out of the bottleneck i.e. Use your technology and innovation as a hiring differentiator, hire more T-shaped technologists than specialists, utilize non-senior developers, and more. 

featured in #304

On Building Scalable Systems

- Kislay Verma tl;dr: "Scalability is the idea that a system should be able to handle an increase in workload by employing more computing resources without significant changes to its design." The key axes of scalability are latency, throughput & capacity. Kislay discusses each, as well as how to quantify scalability, and more.

featured in #303

Architecture Patterns: Caching

- Kislay Verma tl;dr: "Depending on the type of application, the type of data, and the expectation of failure, there are several strategies that can be applied for caching." Kislay discusses the levels in a systems architecture where caching commonly occurs and various caching strategies, such as read through, write through, write behind. 

featured in #302

Why We Switched Our Data Orchestration Service

- Guillaume Perchais tl;dr: "Within Spotify, we run 20,000 batch data pipelines defined in 1,000+ repositories, owned by 300+ teams — daily. The majority of our pipelines rely on two tools: Luigi (Python) and Flo (Java). The data orchestration team decided to move away from these tools, and in this post, the team details why the decision was made, and the journey they took to make the transition."

featured in #301

Bottleneck #01

- Tim Cochran Carl Nygard tl;dr: This first post in a series studies common technical debt bottlenecks by companies entering growth. The authors look at causes of tech debt bottlenecks, various types of debt, signs you are approaching this bottleneck, and strategies to get out of it, include setting a quality bar, damage limitation to the business, collaboration with product, and more.

featured in #299

Four Rules for Scaling

- Varun Srinivasan tl;dr: Coinbase's Director of Eng: (1) Shape your org chart so your product reflects what you want to ship. (2) Push part of the strategic decision-making process down the org to gain speed and flexibility. (3) Think in 3D to visualize what's missing i.e, think about how the work gets done, not just who does it. (4) Put people over org structures and leverage emerging leaders.

featured in #298

The Four Innovation Phases Of Netflix’s Trillions Scale Real-time Data Infrastructure

- Zhenzhong Xu tl;dr: "I hope this post will help platform engineers develop their cloud-native, self-serve streaming data platforms and scale use cases across many business functions (not necessarily from our success but maybe more from our failures)."

featured in #294

Designing Netflix

- Ankit Sirmorya tl;dr: Ankit guides us through the architecture plan for a Netflix style application, planning for the following scale: 100 million active users registered, 2500 MB uploaded every minute, 10 combinations of resolution and codec formats supported, 3 videos watched daily.

featured in #290

Designing Instagram

- Ankit Sirmorya tl;dr: Ankit shows us how to architect an Instagram scale photo-sharing application where users can upload photos and view those photos, follow other users, view feeds containing posts of the users they follow, like and comment on posts.

featured in #289