Improve Answer Quality and Troubleshoot Knowledge
Answer quality depends on source quality, synchronization state, retrieval relevance, assistant instructions, and the wording/context of the visitor question.
Knowledge troubleshooting follows the evidence from one failed question backward. The conversation record can show whether retrieval ran, which files were returned, and how relevant the top result appeared. You can then improve the actual source, its metadata, or the assistant’s source-use instruction and repeat the same question in a fresh conversation.
Begin by choose a failed or low-confidence information-seeking question. The work pays off when you compare response, retrieval details, duration, and confidence; that final observation is more meaningful than a saved notice because it shows what an administrator or visitor actually receives.
What this feature does and when to use it
Section titled “What this feature does and when to use it”Use evidence from a repeatable test and Analytics instead of adding broad content blindly.
Use this feature in the following situations:
- A question has a wrong, incomplete, or unnecessarily uncertain answer.
- Knowledge Gaps reports low confidence and you need to determine whether content or retrieval is responsible.
- The expected source is synchronized but a different file is consistently selected.
Where to find it
Section titled “Where to find it”Before you begin
Section titled “Before you begin”- SmartSite Assistant is installed and activated.
- You are signed in with an account that can manage WordPress options.
Set it up step by step
Section titled “Set it up step by step”- Choose a failed or low-confidence information-seeking question.
- Open its Chat History details and note whether knowledge search ran, which files were found, and their scores.
- Verify the expected source is selected, current, and not failed.
- Improve the page itself or add accurate summary/topics/FAQs/key facts.
- Save metadata and synchronize.
- Start a new conversation and ask the same question.
- Compare response, retrieval details, duration, and confidence.
Fields, controls, and important values
Section titled “Fields, controls, and important values”The values here help explain why a response was weak even when synchronization appeared successful. Retrieval metadata and confidence clues show whether the AI searched, what it found, and how strongly it matched. Testing in a new conversation avoids old context hiding a change. Improve the source or metadata when retrieval is weak; improve instructions when the right source was found but used poorly.
| Field, control, or status | What SmartSite Assistant does with it | How to use it and why it matters |
|---|---|---|
| Knowledge search diagnostics | Records whether search was triggered and which files/scores were involved when available. | Use “Knowledge search diagnostics” to isolate the activity relevant to one question. The result can reveal where an answer failed, but the filter itself does not fix anything; improvement comes from the configuration change made after reviewing the evidence. |
| Confidence threshold | Knowledge Gaps default is 70%; available UI thresholds are 50%, 60%, 70%, 80%, and 90%. | Interpret “Confidence threshold” with the unit, range, and default shown beside it. Change one boundary at a time and test just below and above it; tighter numbers may protect cost or safety but can also remove legitimate context or activity. |
| Metadata | Adds retrieval-oriented context but does not repair an inaccurate source. | Use metadata to add the vocabulary and structure that retrieval is currently missing. It can help a correct page match more questions, but it cannot repair a source whose underlying dates, policies, or facts are wrong. |
| New conversation | Avoids a previous response chain masking the effect of source changes. | Start a fresh conversation after changing knowledge so earlier response-chain context does not influence the comparison. This makes it easier to tell whether the new source or metadata actually changed retrieval and the resulting answer. |
How to confirm it is working
Section titled “How to confirm it is working”For Improve Answer Quality and Troubleshoot Knowledge, a saved notice is only the beginning. Confirm the outcome from a fresh session and compare it with the success description above so cached pages or earlier conversation context do not create a false positive.
Practical example
Section titled “Practical example”If visitors ask “returns window” but the page says “withdrawal period,” add the visitor phrase to Topics and a precise FAQ, then sync.
Recommended practice
Section titled “Recommended practice”- Change one part of Improve Answer Quality and Troubleshoot Knowledge at a time and keep a short record of the previous value and test result.
- Judge the change with a fresh question that depends on this source; an admin status cannot show whether retrieval chose it.
Important warnings
Section titled “Important warnings”Common problems and focused checks
Section titled “Common problems and focused checks”| Problem | What to check and what to do next |
|---|---|
| Search was not triggered. | Check whether the question is information-seeking, review instructions, and confirm a vector store is connected. |
| Relevant file has a weak score. | Improve source title/body and concise metadata, then resynchronize and retest. |
| Improve Answer Quality and Troubleshoot Knowledge is missing or does not match this guide. | Confirm the plugin is active and the account can manage WordPress options. Check selection, synchronization, remote processing, and retrieval evidence in that order. |
| A change on Improve Answer Quality and Troubleshoot Knowledge does not produce the expected result. | Keep the exact notice and test case, then review the browser console and WordPress/PHP log. Check selection, synchronization, remote processing, and retrieval evidence in that order. |
Screen reference
Section titled “Screen reference”- Capture
- Open one sanitized low-confidence gap detail with the score, signals, and searched-file section expanded.
- Show
- Question, confidence, reason signals, searched files and scores, response excerpt
- Viewport
- Desktop, 1440 × 900
- Annotate
- Use numbered callouts only for controls referenced in the procedure.
- Redact
- OpenAI keys, tokens, secrets, personal information, private URLs, IP addresses, and conversation text