/Tests

Snapshot Testing

- Kent Beck tl;dr: Kent explains what Snapshot Testing is and how it scores on the  test desiderata - a list of 12 desirable properties of tests. This list is a useful framework for evaluating different types of tests.

featured in #431

Why We Test In Production (And You Should To)

- Ian Vanagas tl;dr: "Testing in production successfully is a multi-step process, and this post goes over what it is, why we do it, and how to do it well." Ian covers various types of production testing, such as usage tracking, feedback, monitoring, load testing, and integration testing.

featured in #428

I Booted Linux 292,612 Times

- Richard Jones tl;dr: Richard discovered a bug in Linux where it occasionally hangs on boot. He ran guestfish in a loop, performing 10,000 boots and using a test harness with up to 8 threads. After an extensive bisection process between versions 6.0 and 6.4-rc6, he found that a regression in the printk time feature was responsible.

featured in #423

When And How To Run Group-Targeted A/B Tests

- Lior Neu-ner tl;dr: Tests are run when one user interaction with your product impacts how others use it. “Suppose Slack wants to improve the usage of a new video calling feature. Improving the feature's discoverability for a single user will increase their own usage with it, but since they use it with their coworkers, their coworkers will also discover it.”

featured in #421

8 Annoying A/B Testing Mistakes Every Engineer Should Know

- Lior Neu-ner tl;dr: (1) Including unaffected users in your experiment. (2) Only viewing results in aggregate (aka Simpson's paradox). (3) Conducting an experiment without a predetermined duration. Lior discusses these and 5 more anti-patterns.

featured in #416

So You Want To Build End-To-End Tests

- Rebecca Stone tl;dr: A well-built test suite runs faster, gives more reliable results, and makes long-term maintenance easier and cheaper. After thousands and thousands of end-to-end tests, QA Wolf has picked up a few tricks to build fast, stable, and accurate ones. This guide teaches you how.

featured in #414

An Example Of LLM Prompting For Programming

- Martin Fowler tl;dr: Martin shows us how ChatGPT produces useful self-tested code. The initial prompt primes the LLM with an implementation strategy asking for an implementation plan rather than code. Once that plan is in place, it’s refined and the author uses it to generate useful sections of code.

featured in #407

Why Engineering Teams Struggle To Scale Their Test Coverage

- Kirk Nathanson tl;dr: We talk to a lot of engineering leaders about QA and end-to-end testing. Something we hear all the time is how difficult it’s been to scale their automated test coverage beyond a few key workflows. Here are the three obstacles that are faced by companies of all sizes.

featured in #406

Unit Testing Analytics Code

- Matt Kaye tl;dr: “I face lots of pushback when it comes to unit testing. Usually the objections come in the form of either not knowing why you might test, since the code is just so simple and straightforward that nothing could go wrong, or not understanding the value added. In my opinion, both of these objections come from the same place.”

featured in #405

How Lyft Uses Load Testing To Ensure Reliable Service During Peak Events

- Remco Van Bree tl;dr: “We have come to realize that load testing in production is a powerful tool to prepare systems for unexpected bursty traffic and peak events. We’ll explore why Lyft needed a custom performance testing framework that worked in production, how we built a cross-functional solution, and how we’ve continued to improve this testing platform.”

featured in #404