tl;dr:“I haven’t been particularly impressed by most online content about LLMs and security. For instance, the draft OWASP content is accurate but not particularly useful. It portrays LLM security as being a wide array of different threats that you have to familiarize yourself with. Instead, I think LLM security is better thought of as flowing from a single principle. Here it is: LLMs sometimes act maliciously, so you must treat LLM output like user input.”
tl;dr:“I’m a big fan of “sharp tools”. These are tools that are powerful enough to be hugely helpful or harmful, depending on how they’re used. Most forms of direct production access are in this category: like ssh or kubectl access, a read-write prod SQL console. It’s also possible to give “dangerous advice”. Dangerous advice is dangerous because (like sharp tools) it takes competence and judgment to use well. Giving the wrong person dangerous advice is like giving the wrong person production SQL access - they might go off and do something enormously destructive with it.”
tl;dr:“I’m a big fan of “sharp tools”. These are tools that are powerful enough to be hugely helpful or harmful, depending on how they’re used. Most forms of direct production access are in this category: like ssh or kubectl access, a read-write prod SQL console. It’s also possible to give “dangerous advice”. Dangerous advice is dangerous because (like sharp tools) it takes competence and judgment to use well. Giving the wrong person dangerous advice is like giving the wrong person production SQL access - they might go off and do something enormously destructive with it.”
tl;dr:“Empathetic managers care. They are emotionally invested in their employees as human beings, and actively campaign to support their employees’ needs. Ruthless managers are there to do their job. They aren’t necessarily assholes, but they see their main role as communicating the company’s needs to their engineers and vice versa. They will almost never go out on a limb on an employee’s behalf.”
tl;dr:“In the glory days of the 2010s, tech companies were very invested in their employees’ work-life balance. Those glory days are over. Anecdotally, tech company executives are now internally directing their employees to work harder and faster, with the new threat of layoffs adding weight to that directive. Engineers are rightfully scared. What should we do?”
tl;dr:“In the glory days of the 2010s, tech companies were very invested in their employees’ work-life balance. Those glory days are over. Anecdotally, tech company executives are now internally directing their employees to work harder and faster, with the new threat of layoffs adding weight to that directive. Engineers are rightfully scared. What should we do?”
tl;dr:“I realised the other day that I actually have a straightforward heuristic for this. I count the number of times I have this thought:“Oh nice catch, I didn’t think of that!””
tl;dr:“I realised the other day that I actually have a straightforward heuristic for this. I count the number of times I have this thought:“Oh nice catch, I didn’t think of that!””
tl;dr:“It’s a strange time to be a software engineer. Large language models are very good at writing code and rapidly getting better. Multiple multi-billion dollar attempts are currently being made to develop a pure-AI software engineer. The rough strategy - put a reasoning model in a loop with tools - is well-known and (in my view) seems likely to work. What should we software engineers do to prepare for what’s coming down the line?”
tl;dr:“It’s a strange time to be a software engineer. Large language models are very good at writing code and rapidly getting better. Multiple multi-billion dollar attempts are currently being made to develop a pure-AI software engineer. The rough strategy - put a reasoning model in a loop with tools - is well-known and (in my view) seems likely to work. What should we software engineers do to prepare for what’s coming down the line?”