tl;dr:“I hate to be that hype guy, but uhh, can you think of something as impactful to the world as computers and information technology have been since the introduction of personal computers in the 1970s? Well, if LLMs/AI are a new kind of computer, it's not the biggest leap to think that they could also have an enormous impact on the world. I should be clear on my stance here. I'm with Bill Gates when he says that people tend to overestimate what's possible in the short-term and underestimate what's possible in ten years.”
tl;dr:“Like many companies, earlier this year we saw an opportunity with LLMs and quickly but thoughtfully started building a capability. About a month later, we released Query Assistant to all customers as an experimental feature. We then iterated on it, using data from production to inform a multitude of additional enhancements, and ultimately took Query Assistant out of experimentation and turned it into a core product offering. However, getting Query Assistant from concept to feature diverted R&D and marketing resources, forcing the question: did investing in LLMs do what we wanted it to do?”
tl;dr:(1) Context windows are a challenge with no complete solution. (2) LLMs are slow and chaining is a nonstarter. (3) Prompt engineering is weird and has few best practices. (4) Correctness and usefulness can be at odds. (5) Prompt injection is an unsolved problem.