tl;dr:Vicki shares challenges of working with LLMs, offering advice to focus on specific use cases, establishing clear evaluation metrics, building modular systems, and troubleshooting complex issues by getting "close to the metal."
tl;dr:“When humans have no external context as guardrails, we end up recreating what’s already been done or, on the other hand, throwing away things that work and glomming onto hype without substance. This is a real problem in production data systems. In order to do this, we need to understand how to build one.” Vicki believes that we must understand the historical context of our engineering decisions if we are to be successful in this brave new LLM world.
tl;dr:Vicki shares her experience and pain points when building AI applications, highlighting several aspects often not discussed in conversations: (1) Slow iteration times, (2) Build times, (3) Docker images, and more.
tl;dr:"Instead of working on the core of the code and focusing on the performance of a self-contained application, developers are now forced to act as some kind of monstrous manual management layer between hundreds of various APIs..." Vicki shows us how this manifests.
tl;dr:"The critical idea of the blog post is that developers do not work alone: building a product is a team project, and the product itself is a system. And a group of developers working on a single goal also becomes a system that works either together or at cross-purposes with itself."
tl;dr:"For the past two years, I’ve been building and breaking recommender systems at Wordpress and Tumblr. I’m starting a new adventure soon, but this scope of work has been the most meaningful and fun of my career so far, and I wanted to reflect on a few things I’m taking away." Vicki discusses her takeaways about ML, people, culture, communication, and the joys of engineering.
tl;dr:What is a low-friction small thing that most will not notice, but that when they do, is a sign of craftsmanship, expertise, and pride in one's work? Vicki has created a list relevant for ML and Data Science: (1) Python code has type annotations. (2) Accurate documentation of a repo and an easy, reproducible way to run the project. (3) Formatted and linted SQL statements. And more.
tl;dr:An assertion made in the book "The Programmer's Brain" is that confusion around code comes from multiple types of confusion" happening at once i.e. lack of information, knowledge, or "processing power." To overcome this: (1) Write down code syntax so your brain remembers it, no copying and pasting. (2) Read and add "beacons" in the code - "places in the code that your mind can anchor against and do the work of chunking, or processing groups of information."
tl;dr:"Deploying is a ritual, one of my coworkers wrote recently. It’s a sacred place, a quiet place, and a dangerous place, where anything can happen. In deployment, the system is in a fragile state, and you are in a fragile state." Vicki points to a prod deploy as a common, ritualistic moment for engineers.
tl;dr:Vicki only sent 11 emails last year. The reason is that Automattic - behind WordPress.com and Tumblr - use P2s for everything. These are effectively blog posts where you can tag co-workers and cross post to other P2s, Vicki shares an example. Despite the fact that the company still uses Slack, information isn't lost in the same way it is elsewhere.