How GitHub Indexes Code For Blazing Fast Search & Retrieval
- Shivang Sarawagi tl;dr: “The search engine supports global queries across 200 million repos and indexes code changes in repositories within minutes. The code search index is by far the largest cluster that GitHub runs, comprising 5184 vCPUs, 40TB of RAM, and 1.25PB of backing storage, supporting a query load of 200 requests per second on average and indexing over 53 billion source files.”featured in #458
Executing Cron Scripts Reliably At Scale
- Claire Adams tl;dr: Claire discusses the challenges of managing and executing cron scripts in a reliable manner within large-scale infrastructure. “The Job Queue is an asynchronous compute platform that runs about 9 billion “jobs” or pieces of work per day.“ Claire provides insights into techniques such as distributed execution, retries, and monitoring to ensure the dependable execution of cron jobs at scale, highlighting the need for a systematic approach to handle failures effectively.featured in #457
How Clerk Rolls Infra For Auth & User Management
tl;dr: The complex infrastructure required to build and operate an authentication system. It contrasts self-hosted authentication, where developers manage the infrastructure, with a hosted authentication solution, where all auth-related responsibilities are delegated to a specialized service. The article details the components and integrations involved, including the use of cloud services and third-party platforms like Sendgrid and Twilio. The hosted system ensures secure, scalable, and responsive authentication, with options for developers to bring their own infrastructure or configuration.featured in #441
featured in #431
The Growing Pains Of Database Architecture
- Tim Liang tl;dr: “In 2020, Figma’s infrastructure hit some growing pains due to a combination of new features, preparing to launch a second product, and more users - database traffic grows approximately 3x annually.” Tim discusses the infrastructure changes the team implemented.featured in #420
Trunk and Branches Model for Scaling Infrastructure Organizations
- Will Larson tl;dr: Early on in your company’s lifetime, you’ll form your infrastructure organization: a small team of 4-8 engineers. Later on, you'll have 70 engineers across 8-10 teams. Those are both stable organizational configurations. The transition between the small to the large team can be difficult and unstable, and Will provides us with a playbook on how to execute it.featured in #297
Building Infrastructure Platforms
- Poppy Rowse Chris Shepherd tl;dr: "Infrastructure Platform teams enable organizations to scale delivery by solving common product and non-functional requirements with resilient solutions." The 7 key principles that can help you build the right thing are discussed.featured in #296
Don't Make My Mistakes: Common Infrastructure Errors I've Made
- Mat Duggan tl;dr: "Allow me a moment to go back through some of the most disastrous decisions or projects I ever agreed to (or even fought to do, sometimes):" (1) Don't migrate an application from the datacenter to the cloud. (2) Don't write your own secrets system. (3) Don't run your own Kubernetes cluster. Mathew runs through 6 decisions in total, and provides what should have been done for each.featured in #274
Why Secure Access To Cloud Infrastructure Is Painful
- Ev Kontsevoy tl;dr: Secure access to cloud infrastructure doesn't have to be painful. Access Plane consolidates connectivity, authentication, authorization, and audit in one place.featured in #261
Run Code Faster Than The Speed Of Light
tl;dr: Nanos is a Linux binary-compatible unikernel that runs one and only one application in the cloud. Faster and safer than Linux.featured in #241