Support Ticket Quality Review: Fair Scores and Coaching That Gets Booked

Thirty tickets, one rubric and plenty of room for argument

A weekly QA review samples thirty closed tickets and scores each against the rubric: greeting, accuracy, resolution, tone and professionalism. The scores feed coaching, so reps read them closely. A score that is wrong, or a ticket that should never have been in the sample, costs more trust than the review earns.

The tickets live in HubSpot, the QA sheet is a Notion database, the rubric is a document in Google Drive, disputes arrive by Gmail and coaching lands on Google Calendar. When the rubric is revised midweek or the sample window turns out to be wrong, someone re-scores by hand, rewrites rows that never changed, and the summary that reaches team leads drifts away from the sheet.

Who works in the room

Unaddressed messages in the QA room go to Nova, the default agent, so a question about a ticket's score lands with it. Myco takes anything about the summary. Ryan takes coaching, because it is the only agent with a grant to the team's coaching calendar. When a re-score pushes a rep below the coaching line, Nova mentions Ryan with the ticket and the new total, and Ryan leads from there. For a full re-score, Nova reads the thirty conversations through parallel helpers and gets one set of scores back.

The weekly review, step by step

The rubric and the sample rules live in the QA folder, and the room guidelines say which fields an agent may change. Each step ends with the QA lead or a team lead.

A rubric revision in the middle of the week

A rubric revision is where QA reviews usually go wrong. Re-scoring all thirty tickets and rewriting every row makes the sheet's history useless, because a rep can no longer see which scores actually moved. Nova re-scores the full sample against the new rubric but proposes updates only for tickets whose score or status changed, and lists the rest as unchanged.

A window correction works on the sample itself. If the window should have started on Monday, tickets closed the Sunday before come out, a ticket sampled twice counts once, and every total that included them is recomputed. Disputes come last and are the narrowest change: when the QA lead upholds one, only that criterion on that ticket goes back to its original sheet value, and Ryan drafts a single reply on the dispute thread stating the new total.

What the checks catch

• A row rewritten in Notion even though its score and status did not change
• A ticket outside the corrected sample window, or sampled twice, still counted in a total
• A summary line that is missing, belongs to an out-of-scope ticket, or carries the wrong total
• An upheld dispute that resets the whole ticket instead of the one criterion in question
• A second reply on a dispute thread that already has one

What stays with people

• Approving every score change before it is written to the Notion sheet
• Ruling on each dispute and sending the reply, which Ryan prepares as a Gmail draft
• Setting the coaching line and deciding who gets coaching
• Approving each coaching time before it is booked

Integrations in this workflow

Kylon is not a help desk and it does not replace one. It sits across HubSpot, the QA sheet and the calendars the support team already uses, and keeps the review's coordination in one room.

• Gmail
• Google Calendar
• Google Drive
• HubSpot
• Notion
• Slack

• HubSpot holds the sampled tickets and their conversations, read for scoring
• Notion holds the QA sheet, with one row per sampled ticket
• Google Drive keeps the rubric, its revisions and the weekly summary
• Gmail carries disputes in and replies out, as drafts the QA lead sends
• Google Calendar carries each approved coaching session
• Slack is where team leads flag a dispute or ask for a session

Where to start

Begin with one week's sample. Connect the Notion QA sheet, point Nova at the rubric folder in Google Drive, and put one rule in the room guidelines: only tickets whose score or status changed are written, and only after the QA lead approves.

“When a rep asks why a score moved, the answer is already in the thread.”

Set it up with your agent

Open a new Room, add your agent and paste this prompt. It connects the tools, builds the App with you and asks for your rules one step at a time, and nothing goes out until a person approves it.