Recently I wrote about how Large Language Models can be quite random and chaotic, like Gonzo, and what most enterprise workflows want from their models is a 'lawful good' helper, like Kermit, and some principles to get there. (AI as Muppets: Finding Your Kermit)
In there we were exploring how to turn Gonzo into Kermit, but we weren't talking about the other case you want - when you're looking for some ideas or information which aren't the 'most probable thing.' Sometimes you need that creative brainstorming buddy to get you those ideas on the periphery, or at least you don't want it to always say BLT every time you ask for a type of sandwich. (4 out of 5 times - thanks Claude!)
I read a remarkable paper recently which provided a rare combination of interesting research and practical tips for everyday use. It had the not-easy-to-digest title Verbalized Sampling: How to Mitigate Mode Collapse and Unlock LLM Diversity. The short version - you can get a lot more creativity out of your models if you ask it in a certain way.
If you ask the model for multiple options, and - this is the key part - their probabilities - it will produce more diverse outputs. (You don't have to worry about the probabilities.) If you ask for lower probability outputs, you get more 'out there' responses.
I've started to incorporate this in my work, and I see some improvements right away. I've also substantially expanded my sandwich repertoire!
This one will go in the bag of tricks like few-shot prompting, structured output prompting, or meta prompting to improve your results with AI. Because sometimes you want to embrace a bit of controlled chaos.
Pre-prompt Template
To save people time from sorting through the whole paper, here's a 'pre-prompt' example from it. Fill in the bits in the curly braces {} and optionally include the bits in the square braces []:
Generate {num_samplings} responses to the input prompt. Each response should be approximately {target_words} words. Return the responses in JSON format with the key: "responses" (list of dicts). Each dictionary must include:
• text: the response string only (no explanation or extra text).
• probability: the estimated probability from 0.0 to 1.0 of this response given the input prompt (relative to the full distribution).
[Randomly sample the responses from the full distribution.] / [Randomly sample the responses from the distribution, with the probability of each response must be below {probability_tuning}.]
Give ONLY the JSON object, with no explanations or extra text.
{and then your prompt here}-- Bill