How Discord Moved Engineering To Cloud Development Environments
- Denbeigh Stevens tl;dr: “The Internal Developer Experience team is responsible for roughly the first third of the development life cycle. Our main tasks include building and maintaining IDE experiences, managing development environments, shipping tools for building, developing, and testing code, scaling and maintaining CI infrastructure, and owning the change management process and supporting tooling infrastructure. While we could delve deeper into any of these topics, this blog post focuses on how we transitioned all backend and infrastructure development to a Linux-based Cloud Development Environment.”featured in #492
Ledger: Stripe’s System For Tracking And Validating Money Movement
- Ilya Ganelin tl;dr: “Ledger models internal data-producing systems with common patterns, and it relies on proactive alerting to surface issues and proposed solutions. Each day, Ledger sees five billion events and 99.99% of our dollar volume is fully ingested and verified within four days. Of that activity, 99.999% is monitored, categorized, and triaged through rich investigative tooling — while the remaining long-tail is reliably handled through manual analysis.” This post shares technical details on how Stripe built this money movement tracking system, and how teams interact with the data quality metrics that underlie our global payments network.featured in #490
Event-Driven Architecture Fundamentals and Pitfalls (and How to Avoid Them)
- James Higginbotham tl;dr: With over 20 years of API and microservices experience, James summarizes the fundamentals of event-driven architecture, including (1) message types: command, reply, and event (2) what constitutes a message, including body, headers, and network protocol (3) message interaction patterns such as request-reply, fire-and-follow-up, and fire-and-forget (4) event design patterns: event notifications, event-carried state transfer, using hypermedia links in events, and event batching. Throughout, he highlights common pitfalls and the best practices to avoid them.featured in #489
How Disney+ Hotstar Delivered 5 Billion Emojis in Real Time
- Neo Kim tl;dr: This post outlines how Disney+ Hotstar delivered billions of emojis in real-time during the cricket World Cup in India to create a more engaging live experience. The post described how emojis were received from clients, processed and delivered at scale.featured in #489
Authorization In Microservices
tl;dr: When you move from a monolith to a service-oriented architecture, you need to design your authorization accordingly. You'll need to share your Authorization data between your services, and there are many ways to do that. Each design and architecture decision you'll make has trade-offs that you'll need to understand. We'll show you each of those choices and their trade-offs. And, we'll provide you with heuristics that will help you make decisions about your service architecture.featured in #486
How Apple Built iCloud To Store Billions Of Databases
- Leonardo Creed tl;dr: Apple uses FoundationDB and Cassandra for iCloud and CloudKit, their cloud backend service, running one of the largest Cassandra deployments in the world. This deployment includes over 300,000 instances or nodes, managing hundreds of petabytes of data, possibly extending to exabytes. Each cluster in this deployment can handle over two petabytes of data, and there are thousands of such clusters. This indicates a highly distributed and scalable storage system spread across multiple data centers.featured in #480
How Discord Serves 15-Million Users On One Server
- Alex Xu tl;dr: "Internally, each Discord community is called a “guild”. A dedicated Elixir “guild process” handles coordination and routing for each guild. This tracks all connected users to the guild. Every online user has a separate Elixir "session process”. When the guild process gets a new message, event, or update, it fans out this information to the relevant session processes. These session processes then push the update over WebSocket to the Discord clients."featured in #479
An Overview Of Distributed PostgreSQL Architectures
- Marco Slot tl;dr: “What many users notice within the first few minutes of using a distributed database is how unexpectedly slow they can be, because you quickly start hitting performance trade-offs. There are many types of distributed PostgreSQL architectures, and they each make a different set of trade-offs. Let’s go over some of these architectures."featured in #479
The Missing Piece In Enterprise Integration Architectures
tl;dr: There's a critical part of today’s data integration equation that traditional tools don’t solve: How can businesses easily and safely import a wide variety of data files and types? Find out how you can add the missing piece to your data stack.featured in #478
How Uber Finds Nearby Drivers At 1 Million Requests Per Second
tl;dr: H3 is a hexagonal-shaped hierarchical geospatial indexing system created at Uber dividing Earth’s surface into cells on a flat grid, giving each cell a unique identifier with a 64-bit integer. Uber finds nearby drivers by identifying the relevant cells covering the rider's area and then listing the drivers in those cells sorted by the estimated time of arrival (ETA). H3 offers the benefits of a hierarchical indexing system and a hexagonal grid system.featured in #477