/LLM

Build And Keep Your Context Window

- Vicki Boykis tl;dr: “When humans have no external context as guardrails, we end up recreating what’s already been done or, on the other hand, throwing away things that work and glomming onto hype without substance. This is a real problem in production data systems. In order to do this, we need to understand how to build one.” Vicki believes that we must understand the historical context of our engineering decisions if we are to be successful in this brave new LLM world.

featured in #448

Lessons From Building A Domain-Specific AI Assistant

- Eric Liu tl;dr: Eric Liu, Engineer at Airplane, discusses how the Airplane team built a domain-specific AI assistant, the lessons they learned along the way, and what's next for the future of AI assistants.

featured in #447

What We Don't Talk About When We Talk About Building AI Apps

- Vicki Boykis tl;dr: Vicki shares her experience and pain points when building AI applications, highlighting several aspects often not discussed in conversations: (1) Slow iteration times, (2) Build times, (3) Docker images, and more.

featured in #432

Building Boba AI

- Farooq Ali tl;dr: “We are building an experimental AI co-pilot for product strategy and generative ideation called “Boba”. Along the way, we’ve learned some useful lessons on how to build these kinds of applications, which we’ve formulated in terms of patterns. These patterns allow an application to help the user interact more effectively with a LLM, orchestrating prompts to gain better results, helping the user navigate a path of an intricate conversational flow, and integrating knowledge that the LLM doesn't have available.”

featured in #429

Emerging Architectures For LLM Applications

- Matt Bornstein Rajko Radovanovic tl;dr: "In this post, we’re sharing a reference architecture for the emerging LLM app stack. It shows the most common systems, tools, and design patterns we’ve seen used by AI startups and sophisticated tech companies. This stack is still very early and may change substantially as the underlying technology advances, but we hope it will be a useful reference for developers working with LLMs now.”

featured in #425

All The Hard Stuff Nobody Talks About When Building Products With LLMs

- Phillip Carter tl;dr: (1) Context windows are a challenge with no complete solution. (2) LLMs are slow and chaining is a nonstarter. (3) Prompt engineering is weird and has few best practices. (4) Correctness and usefulness can be at odds. (5) Prompt injection is an unsolved problem.

featured in #418

Numbers Every LLM Developer Should Know

tl;dr: (1) 40 -90% is the Amount saved by appending “be concise” to your prompt. (2) 1.3:1 is the average tokens per word. (3) ~50:1 is the cost ratio of GPT-4 to 3.5. And more.

featured in #415