Designing the shift from search to conversation

Why we were moving toward an assistant

KNOWRON already had semantic search: users selected a product line, entered a few keywords, and found technical information. As conversational AI became familiar, customers started asking for a way to ask questions naturally and follow up without reformulating searches.

We also saw the assistant as a foundation for future workflows, so Engineering built an early version and put it in front of users. That gave us real queries and exposed where the interaction was breaking down.

Users were leaving out important context

A request like "service manual" could return dozens of valid documents because the assistant did not know which product or machine the user meant. To the user, it simply looked unreliable.

The old search flow had trained users to select a product line before searching. They carried that mental model into the assistant and assumed the product context was already known.

The final approach

The assistant searched first. Clear requests went straight to results. When the result set became too broad, it acknowledged the ambiguity and asked for the missing product or machine context.

Why we kept filters inside a conversational experience

Technicians already knew filters from the existing product, and they usually needed only two or three. Selecting a known option was often quicker than writing another sentence. It also gave users direct control while our AI-generated filter suggestions were still too unreliable to ship.

The assistant also had to understand what kind of response someone wanted

The assistant had two retrieval skills. Search synthesised an answer from the knowledge base; Document Match returned specific files. The wording of a request did not always make that distinction clear.

I reviewed more than 50 real queries and mapped their elements against the skill we expected to respond. That helped us see where rules might work, and where they would become brittle because of shorthand, multilingual requests and missing context.

Making the assistant's interpretation visible

We considered making users choose Search or Document Match upfront, but that added a decision before every request and assumed intent would stay fixed throughout a conversation.

We evaluated the trade-off between giving users more control upfront and keeping the interaction flexible as their intent changed.

Instead, the assistant evaluated each turn independently and showed which skill it had selected. If the interpretation was wrong, users could rephrase or change direction without restarting the conversation.

What changed

Document Match accuracy improved by around 20% across more than 30 benchmark prompts. User interviews showed higher trust in the assistant, and Customer Success reported fewer questions about how it worked.

This version covers the main product story. The full case study includes the rejected concepts, query analysis, skill-routing trade-offs, benchmark setup, and detailed interaction flows.