Feedback Mechanism

Overview

The Feedback Mechanism is a new feature within the GenAI engine that allows you to provide feedback on AI-generated answers. By refining the way GenAI interacts with questions, you can significantly enhance the accuracy and relevance of the responses provided. The Feedback Mechanism is a part of our Quality Assurance features, enabling you to gain greater insights and control over the GenAI engine.

Tip: you can also check out a Discover Video on the Feedback Mechanism here!

Navigating Quality Assurance & Feedback Mechanism

Two new tabs have been added to the left side of the GenAI dashboard: “Conversations” and “Feedback”. The “Conversations” tab displays all the interactions in which GenAI has participated. Here, you can view general metrics of these conversations, such as the topic and recognition rate—the latter being the percentage of messages to which the GenAI engine was able to provide a response.

By clicking on a specific conversation, you can review all the messages exchanged during that conversation. This view allows you to like or dislike the GenAI engine's responses.

 

Types of Feedback

If you are satisfied with a GenAI-generated answer, you can like it. We're expecting to leverage responses marked as "liked" for future features to ensure consistency in your knowledge base over time, and to run model finetunings for your specific environment. Additionally, liking an answer serves as a reminder that you've already rated the response and don’t need to evaluate it again.

If you are dissatisfied with a GenAI answer, you can dislike it and choose from several options to specify why you didn’t like the response: “I don’t like the answer,” “Wrong answer,” or “Do not answer.”

Wrong answer

Select the “Wrong answer” option if the GenAI engine provided an incorrect response. When you choose this option, you’ll be prompted to review the knowledge base used to generate the answer, as an incorrect response often indicates that the knowledge base contains incorrect, vague, or outdated information.

Do not answer

You can also select “Do not answer” as a feedback reason. This option ensures that similar questions in the future will not be answered by the GenAI engine. Use this option when you prefer that certain user questions, such as those on sensitive topics or other company-specific contexts, should not be addressed by AI. When this feedback is applied, future similar questions will either prompt the end user to be transferred to a live agent, or, if you are using the GenAI basic package, the question will simply remain unanswered.

I don’t like the answer

If the answer provided by the GenAI engine is mostly correct, but you are not fully satisfied with how it’s phrased, formatted, or structured, you can select the “I don’t like the answer” feedback type.

When you choose "I don’t like the answer," you’ll be taken to a new view where you can provide detailed feedback using the Feedback Mechanism. Here, you’ll see the original question and answer, along with a field to enter your feedback. In this field, you can specify how you’d like the answer to be modified, for example: “make it shorter,” “remove the last sentence,” or “format as a numbered list.” After providing your feedback, you can use the “Test” button to see how the feedback alters the generated answer. Once you are satisfied with the changes, you can save your feedback.

When you’re satisfied with how the feedback has improved the GenAI response, you can save your feedback. This ensures that the feedback will be applied when a similar question is encountered by the GenAI engine in the future, improving the answer to your liking.

Add Knowledge

Another reason you might be dissatisfied with a response could be due to missing knowledge in your Knowledge base. Previously, adding missing knowledge required modifying a PDF, website, or other document. Now, it’s much easier to add knowledge directly to the GenAI engine via the Conversations Screen. When viewing a conversation, you will now find an “Add Knowledge” button next to the Like and Dislike buttons beneath a GenAI response. Clicking this button allows you to enter new knowledge into a text field and save it to your Knowledge base, ensuring better responses in the future.

After adding your first piece of knowledge this way, you can view it in the updated 'Knowledge' tab. From this tab, you can also manually add additional pieces of knowledge.

Please note that manually added knowledge is treated the same as knowledge added through your website, a PDF, or another source. It is considered a piece of Knowledge, not Feedback. This means it may be used for any relevant question, not just those similar to the one where the knowledge was initially added.

View and edit your provided Feedback

In the Feedback tab, you can view all the feedback you’ve provided to the GenAI engine. Here, you can see the type of feedback given, review the details, and use the three dots under the “Actions” column to edit or delete your feedback.

How to provide proper Feedback for “I don’t like the answer”

Here are several tips to ensure your feedback is useful for our GenAI engine:

• Provide feedback only if the answer is mostly factually correct but you’re not completely satisfied with it. but you’re not completely satisfied with it. If the answer is incorrect, it’s better to update your Knowledge base rather than trying to correct the mistake with Feedback.
• Do not reference the customer question when giving Feedback. It's our job to match your feedback with new incoming questions, so you don’t need to worry about this. For example, feedback like “On similar questions, do not mention our email address” isn’t helpful. Instead, simply state: “Do not mention our email address.”
• State how you want your answer to be changed, instead of rewriting the answer. For instance, if you want the last sentence removed from a GenAI response, provide feedback like: “remove the last sentence” rather than rewriting the entire answer with the last sentence removed.
• Avoid referring to your Knowledge base in the Feedback. Our Feedback Mechanism doesn’t “see” the Knowledge used when processing feedback. Its sole purpose is to improve the given answer by incorporating your feedback without having knowledge of the entire conversation context.
• Use simple, imperative language in your feedback. Examples of good, usable feedback include: “Write GenAI as Generative AI,” “Format as a numbered list,” “Do not mention our Email Channel,” “Use simpler language,” or “Add an emoji to the end of the sentence.”