/LLM

What We Don't Talk About When We Talk About Building AI Apps

- Vicki Boykis tl;dr: Vicki shares her experience and pain points when building AI applications, highlighting several aspects often not discussed in conversations: (1) Slow iteration times, (2) Build times, (3) Docker images, and more.

featured in #432

Building Boba AI

- Farooq Ali tl;dr: “We are building an experimental AI co-pilot for product strategy and generative ideation called “Boba”. Along the way, we’ve learned some useful lessons on how to build these kinds of applications, which we’ve formulated in terms of patterns. These patterns allow an application to help the user interact more effectively with a LLM, orchestrating prompts to gain better results, helping the user navigate a path of an intricate conversational flow, and integrating knowledge that the LLM doesn't have available.”

featured in #429

Emerging Architectures For LLM Applications

- Matt Bornstein Rajko Radovanovic tl;dr: "In this post, we’re sharing a reference architecture for the emerging LLM app stack. It shows the most common systems, tools, and design patterns we’ve seen used by AI startups and sophisticated tech companies. This stack is still very early and may change substantially as the underlying technology advances, but we hope it will be a useful reference for developers working with LLMs now.”

featured in #425

All The Hard Stuff Nobody Talks About When Building Products With LLMs

- Phillip Carter tl;dr: (1) Context windows are a challenge with no complete solution. (2) LLMs are slow and chaining is a nonstarter. (3) Prompt engineering is weird and has few best practices. (4) Correctness and usefulness can be at odds. (5) Prompt injection is an unsolved problem.

featured in #418

Numbers Every LLM Developer Should Know

tl;dr: (1) 40 -90% is the Amount saved by appending “be concise” to your prompt. (2) 1.3:1 is the average tokens per word. (3) ~50:1 is the cost ratio of GPT-4 to 3.5. And more.

featured in #415