Trace an Assistant problem without guessing
“The Assistant is wrong” is a symptom, not a diagnosis.
The active profile, OpenAI model, previous turns, runtime, retrieved knowledge, Tool results, visibility, and frontend all contribute to what a visitor experiences. Preserve one failed case and locate the first layer that diverged.
Start with a reproducible conversation
Section titled “Start with a reproducible conversation”Record the exact visitor message, expected behavior, actual response/error, time, page/channel, visitor access context, and whether Basic or Advanced runtime was enabled. Name the source or Tool that should have been used. Without this, “it behaved strangely” invites random changes.
Repeat in a new private browser session. A previous response chain can retain old instructions, clarified intent, Tool results, or specialist context. If the fresh conversation works, the profile may be fine and the original chain/context needs examination.
- Confirm one intended profile shows Active.
- Open it and record Internal Name, local ID, model, Welcome Message, and relevant Instructions.
- Confirm the regular OpenAI key can load models.
- Start a new private conversation with the exact message.
- Open the matching Chat History record and inspect status, model, runtime, retrieval, Tools, route/handoff trace, duration, and errors.
- Identify the first incorrect layer.
- Change one field or system, repeat the same message, and compare evidence.
Separate profile problems from neighboring systems
Section titled “Separate profile problems from neighboring systems”| Evidence | Likely owner | Do next |
|---|---|---|
| No Active badge / “Assistant is not available” | Assistant selection | Create or activate a local profile. Deleting Active clears the selection. |
| Model list or API request fails | OpenAI API/project/model | Verify the regular key, billing, permissions, network, and a Responses-compatible text model. |
| Right tone, wrong company fact; search absent/wrong | Knowledge Base/retrieval | Check source authority, Current/completed state, specialist knowledge permission, and retrieval diagnostics. |
| Correct facts, wrong tone/scope/uncertainty | Assistant or active specialist instructions | Rewrite the operating brief and use a fresh conversation. |
| Unexpected email/webhook/data return | Tool description/assignment/receiver | Disable the Tool first, then inspect arguments, result, Basic/Advanced behavior, and remote log. |
| Wrong specialist or sticky topic | Advanced Agent Flow | Inspect route decisions, handoffs, agent scope, policy, and published revision. |
| Widget missing or wrong users see it | Settings → Access/frontend | Test allowed and denied contexts; do not alter Instructions. |
| Wrong landing title/color/position | Settings or Design | Separate chat content from Assistant greeting and visual controls. |
Symptoms, evidence, and the next check
Section titled “Symptoms, evidence, and the next check”“I edited Instructions, but the old behavior remains”
Section titled ““I edited Instructions, but the old behavior remains””Reload the edit screen to confirm Save persisted. Verify that profile is Active. Start a fresh conversation. In Advanced runtime, check whether a specialist—not the Base Assistant—answered; specialist instructions can own the turn. If old wording appears only in the Welcome Message, edit that separate field.
“The Display Name changed, but another title did not”
Section titled ““The Display Name changed, but another title did not””Display Name belongs to the chat header identity. Landing-screen title, description, avatars, and other content belong in Settings. The Welcome Message is the first conversational message. Identify the exact pixel/text before editing.
“A model appears in the selector but chat fails”
Section titled ““A model appears in the selector but chat fails””The full selector lists models from OpenAI, including entries that may not support this Responses API flow or your project. Choose a verified compatible text model. Do not resync knowledge to repair a model request error.
“Basic works; Advanced does not”
Section titled ““Basic works; Advanced does not””The local profile and shared knowledge are probably available. Inspect the published flow, specialist Knowledge Base setting, Tool ownership, routing policy, and trace. Confirm the workflow is live and the intended specialist has an escape path.
“Advanced works; Basic calls an action too easily”
Section titled ““Advanced works; Basic calls an action too easily””Advanced side-effect actions receive approval/duplicate policy; Basic actions do not have the same enforced visitor confirmation gate. Narrow the Tool, assignment, parameters, Description, and receiver authorization. Design it to be safe in Basic or avoid exposing it there.
Read the profile fields as evidence
Section titled “Read the profile fields as evidence”| Field | Can explain | Cannot explain alone |
|---|---|---|
| Internal Name / local ID | Which card/configuration administrators selected. | Visitor-visible identity or factual accuracy. |
| Display Name | Header identity shown to visitors. | Landing copy, greeting body, or response behavior. |
| Welcome Message | First conversation message. | Later answer scope unless Instructions also say it. |
| Instructions | Tone, scope, source preference, uncertainty, and Tool-use guidance. | Technical authorization, missing source facts, model outage, or frontend visibility. |
| Model | Generation capability, compatibility, latency, token/cost pattern. | Source truth, current synchronization, receiver-side action result. |
| Active badge | Selected local profile for runtime/Base. | That a stale existing conversation contains no prior context. |
Avoid diagnostic damage
Section titled “Avoid diagnostic damage”Do not weaken Security, enable public visibility, delete sources, turn on unsafe HTTP, rotate multiple credentials, or install a larger model merely to see whether the symptom disappears. Each move expands risk and destroys the comparison.
Do not paste complete API keys, private visitor messages, webhook headers, personal data, or raw production logs into support channels. Use log IDs, timestamps, sanitized excerpts, and exact safe error text.
Follow time in the right order
Section titled “Follow time in the right order”Compare the Assistant card’s last-updated time, the conversation timestamp, Knowledge synchronization time, published workflow revision, and remote receiver log. A conversation created before an edit is not evidence that the edit was ignored. A page synchronized after the conversation cannot have grounded that earlier answer.
Use the site’s configured timezone carefully when comparing WordPress screens with OpenAI, PHP, mail, webhook, or Meta logs. Record both displayed time and timezone in an escalation packet. This small discipline often resolves apparent “random” behavior.
Distinguish plausible from proven
Section titled “Distinguish plausible from proven”An answer that sounds like the expected profile is not proof that profile handled it. The same model may produce similar wording under several profiles. Use the Active ID, Chat History Assistant/model fields, and Advanced trace when available. Likewise, a correct answer does not prove Knowledge search ran; it may come from earlier conversation context or general model knowledge.
For action Tools, the Assistant saying “sent” does not prove email delivery or remote business completion. Match the Tool execution record with the receiving system. Troubleshooting ends when evidence agrees across the boundary, not when the wording looks reassuring.
When rollback is the right fix
Section titled “When rollback is the right fix”If a recent active-profile edit causes broad production regressions, reactivate the known-good profile or restore the approved instructions before continuing investigation. Containment is not failure. Preserve the bad case and change record, then reproduce safely in Local/staging.
Do not delete the problematic profile immediately; that can remove the exact configuration needed to understand the incident. Disable related action Tools if their behavior is uncertain, restrict widget access if visitor impact continues, and coordinate with the appropriate content, security, or integration owner.
A compact regression set
Section titled “A compact regression set”Keep five synthetic questions for every profile:
- a simple greeting and identity question;
- a direct company fact with one authoritative source;
- the same fact using visitor vocabulary not in the title;
- a missing or out-of-scope question that should produce honest limitation; and
- a Tool-related phrase that is informational rather than an explicit action request.
If Advanced is enabled, add a clear specialist request, an ambiguous request that should stay with Base, a topic change, and an approval cancellation. Compare traces, not only prose.
Escalate with a useful evidence packet
Section titled “Escalate with a useful evidence packet”Provide plugin release, WordPress/PHP environment, timestamp/timezone, safe conversation log ID, active local Assistant ID/name, model, Basic/Advanced state, expected owner source/Tool, exact sanitized error, and reproduction steps. State what you already changed. This lets a technical owner trace the same execution path without requesting secrets.
Recognize the profile under test
Section titled “Recognize the profile under test”- Capture
- Show the Assistants list with the intended active profile and a nearby sanitized error notice or diagnostic callout.
- Show
- Active badge, model, management actions, notice area
- Viewport
- Desktop, 1440 × 900
- Annotate
- Use numbered callouts only for controls referenced in the procedure.
- Redact
- OpenAI keys, tokens, secrets, personal information, private URLs, IP addresses, and conversation text
Follow the evidence
Section titled “Follow the evidence”- Profile text: write the operating brief
- Active selection: manage alternatives
- Retrieval: investigate a weak knowledge answer
- OpenAI error: troubleshoot the connection
- Full system symptom: maintenance diagnostics