Maybe it is cargo culting at this point, idk. When I first started experimenting with this, about two generations of models back, the role play prompt made a noticeable difference.
Example: with early Claude (pre-Claude Code) if you asked for a Rust program you’d get something that only resembled Rust syntax but was a mix of different languages. “You are a senior software engineer that develops solely with the Rust programming language” or something like this made it generate syntactically correct Rust.
Similar prompts led to better, more focused tests. I find that such prompts are not as necessary anymore, but anecdotally I’ve still felt a difference.
Example: with early Claude (pre-Claude Code) if you asked for a Rust program you’d get something that only resembled Rust syntax but was a mix of different languages. “You are a senior software engineer that develops solely with the Rust programming language” or something like this made it generate syntactically correct Rust.
Similar prompts led to better, more focused tests. I find that such prompts are not as necessary anymore, but anecdotally I’ve still felt a difference.