Translate a business question into a well-formed prompt for a data query
Read and sanity-check the results an assistant returns
Recognize signs that a query result is wrong, incomplete, or scoped incorrectly
Reframing Business Questions
Business questions and data questions look similar but aren't the same thing. "Why are these orders late?" is a business question as it's really asking about causes, patterns, and next steps. A data query can't answer "why" directly, as it can only return information that may help you answer why. If a question is not reframed, results returned may not be usable even if it is technically providing what you asked for.
Business question
Reframed data question
"Why are these orders late?"
"How many returns did we get last week?"
"Which fulfillments are stuck?"
Effective Prompting for Data Querying
Effective prompts for data querying generally include:
The entity type being asked about (orders, fulfillments, InventoryCatalogues, retailers, etc.)
A status or date filter, where relevant
Clear scope (one retailer? one location? a specific time window?)
Missing any one of these tends to produce a technically-answered-but-wrong-scope result rather than an obvious error, which may impact
Differences between weak prompts and strong prompts
Weak: "show me problem orders"This prompt is missing all three ingredients. The LLM has to guess what "problem" means, over what time window, and for which retailer. Returned results will vary as there is not enough context provided.Strong: List me all orders from Retailer A where the status is not 'Completed'
Checking Results
Before accepting a query result as accurate, check the important points:
Entity match: Does the result match the entity type you asked about?
Scope match: Is the scope (retailer/location/date range) what you intended?
Plausibility: Does the row count and shape of the data look plausible?
Completeness: Did the result actually cover everything the question was asking about?
If one or more of these do not align with what you think the result should be, ask the agent to explain what it did and why so you can gain more insight into where the mistake was made. When iterating on your prompts, remember to change one thing at a time so you are able to know how you can fix or improve things faster next time.If you've tried clarifying your prompt with little success, it might be a good idea to escalate, as there may be more technical differences in the data set, or other issues.
Check your prompts
You can check your potential prompt by asking yourself if a colleague would be able to give you the correct information if you were to ask them the same question.