Customer Support Bot With GPT-4 on Your Own Knowledge Base

I built a support bot for a cosmetics brand — 6 months later it's handled 12,000+ questions. Accuracy 92%, customer satisfaction rating (from the post-chat survey) 4.6/5. This post is the architecture, the decisions I made, and the problems I hit.

Customer support bot = RAG + persona + escalation

Three components working together. RAG — pulls accurate info from the company's knowledge base (see my earlier RAG post). Persona — how the bot talks, tone, what it must and mustn't say. Escalation — handoff to a human when the bot doesn't know.

Persona — how the bot should talk

Underrated step. Just write 'You are a helpful assistant' and the bot answers cold and stilted. Users feel it, satisfaction drops. For the cosmetics brand I wrote: 'You're Alina, our brand assistant. Patient, friendly, you love the products. You understand customer feelings. Always say 'we', never 'the company'. Keep answers short and personal.'

Decompose complex questions

Customer writes: 'Hi, I'm 35, dry skin, what serum do you recommend, what does it cost, and how does shipping work?'. That's three questions. The first version answered all three in a single jumbled paragraph. Now I first send to GPT-4o-mini: 'How many separate questions are in this message? Return JSON.' Then RAG-search each question separately, synthesise, send.

Escalation — let the bot admit it doesn't know

Biggest mistake: a bot that makes things up instead of saying 'I don't know'. When the RAG context doesn't have the answer, models hallucinate. To prevent this I put it explicitly in the prompt: 'Only use information from the context below. If the context doesn't contain the answer, tell the customer 'I'll route this to our operator team for a more accurate answer' and prepend [ESCALATE] to your response.'

response = await llm.chat(messages=[{"role": "system", "content": SYSTEM_PROMPT}, ...])

if response.content.startswith("[ESCALATE]"):
    # Operator-a ötür
    await create_support_ticket(user_id, conversation_id, response.content)
    await send_message(user_id, "Bizim operator komandamız 15 dəqiqə ərzində sizə yazacaq.")
else:
    await send_message(user_id, response.content)

Conversation memory — keep the last 6

Sending the full conversation history every turn gets expensive fast. I keep the last 6 messages verbatim and summarise everything older into one short paragraph. Customer profile data (skin type, purchase history) is stored separately and added to the context on every turn.

Multi-channel — one bot, many surfaces

The bot core is channel-agnostic — exposes a REST API. On top sit adapters: Telegram, WhatsApp Business API, web chat widget (Next.js + Socket.io). Every adapter hits the same endpoint and gets the same response. Adding a new channel takes about a day.

Customer satisfaction — measuring it

After every conversation we send 'Did this help? 👍 / 👎 / Talk to an operator'. 👎 routes the conversation to an admin panel — me and the client team review them once a day. Almost always it's a gap in the knowledge base. We fill the gap, the question stops coming back.

Real cost — cosmetics brand example

• GPT-4o-mini per conversation: ~$0.008
• Embeddings & vector DB: ~$15/month (15,000 chunks)
• Server (Hetzner CX22): $5.18/month
• 12,000 conversations × $0.008 = $96/month
• Total: ~$120/month vs. one full-time support rep at $1,500+/month