Documentation
Home
Getting Started
Overview Token Counting Plans & Limits
Core Guides
AI System Prompts Q&A Generation Chatbot Overview Knowledge Base Vector Stores Design & UI Intent Logic Deployment

Chatbot - Compare AI vs RAG

Chatbot - Compare AI vs RAG

Overview

The Compare AI vs RAG Chatbot feature lets you test both versions of your chatbot side-by-side. This helps you understand the difference between general AI responses and RAG-enhanced responses that use your knowledge base.

What's the Difference?

Basic AI Chatbot

Uses:
  • General AI knowledge
  • Your system prompt
  • Intent responses (if enabled)
  • No document retrieval

Best For:

  • General questions
  • FAQ-style responses
  • Simple queries
  • When you don't have specific documents

Limitations:

  • Doesn't know your specific content
  • May give generic answers
  • Can't reference your documents

RAG-Based Chatbot

Uses:
  • Your uploaded documents (via vector store)
  • Semantic search of knowledge base
  • Context from your specific content
  • Combines retrieval + AI generation

Best For:

  • Questions about your specific content
  • Document-based queries
  • Accurate, source-backed answers
  • When you have relevant documents

Advantages:

  • Knows your specific information
  • Provides accurate, sourced answers
  • Can quote from your documents
  • More reliable for your content

Accessing Comparison Page

From Project Page: 1. Go to your Chatbot project 2. Find "Step 2: Customize Your Chatbot" 3. Click "Compare AI vs RAG Based AI Chatbot" button 4. Opens side-by-side comparison interface

Prerequisites:

  • Chatbot must be designed
  • Vector store should be built (for RAG to work)
  • Knowledge base items completed

Comparison Interface

Side-by-Side Layout

Left Side: Basic AI Chatbot

  • Shows standard AI responses
  • Uses system prompt and intents
  • No vector store involved

Right Side: RAG-Based Chatbot

  • Shows RAG-enhanced responses
  • Uses vector store + knowledge base
  • Retrieves relevant documents

Interface Features

Chat Windows:

  • Two identical chat interfaces
  • Same appearance and styling
  • Side-by-side for easy comparison
  • Independent conversations

Controls:

  • Toggle intentions on/off (affects both)
  • Independent message inputs
  • Real-time responses
  • Clear conversation buttons

Understanding Intentions Toggle

How It Works

Enable Intentions:

  • Both chatbots check intent responses first
  • If intent matches, use your response
  • If no match, fall back to AI/RAG
  • Controlled by single toggle (affects both)

Disable Intentions:

  • Both chatbots ignore intent responses
  • Basic: Uses only general AI
  • RAG: Uses only vector store retrieval + AI

Toggle Location:

  • Above Basic AI Chatbot
  • Button shows current state
  • Click to toggle
  • Reflects in both chatbots

When to Use

Enable When:

  • You have important intents defined
  • Want specific responses
  • Testing intent matching

Disable When:

  • Testing pure AI vs RAG
  • Comparing without intents
  • Seeing raw capabilities

Testing Scenarios

Scenario 1: General Knowledge

Question: "What is artificial intelligence?"

Basic AI:

  • Provides general definition
  • Uses common knowledge
  • Comprehensive answer

RAG-Based:

  • May use your documents if relevant
  • Could provide context from your content
  • May be more specific if you have AI docs

Observation:

  • Similar responses if no relevant docs
  • RAG may add your perspective if available

Scenario 2: Your Specific Content

Question: "What are your business hours?"

Basic AI:

  • Uses intent response (if enabled)
  • Or generic answer if no intent
  • May not know your specific hours

RAG-Based:

  • Uses intent response (if enabled)
  • Or searches documents for hours
  • May find specific info from your docs

Observation:

  • Intent response same in both
  • RAG better if hours in documents but no intent

Scenario 3: Document-Specific Question

Question: "What does your refund policy say about returns?"

Basic AI:

  • Generic refund policy answer
  • May not match your policy
  • General information

RAG-Based:

  • Searches your documents
  • Finds relevant refund policy content
  • Provides specific, accurate answer
  • May quote from your documents

Observation:

  • Clear difference in accuracy
  • RAG gives your actual policy
  • Basic gives generic answer

Scenario 4: Complex Query

Question: "Can you explain how your API authentication works?"

Basic AI:

  • General API authentication explanation
  • May not match your specific API
  • Generic technical info

RAG-Based:

  • Searches your API documentation
  • Finds your specific authentication method
  • Provides accurate, relevant answer
  • Matches your actual implementation

Observation:

  • RAG clearly superior for specific tech
  • Basic may confuse users with generic info
  • RAG uses your actual docs

When to Use Each Type

Use Basic AI When:

✅ General questions are common ✅ You don't have specific documents ✅ Intent responses cover most questions ✅ Simple FAQ use case ✅ No need for document retrieval

Use RAG When:

✅ Users ask about your specific content ✅ You have comprehensive documentation ✅ Need accurate, sourced answers ✅ Content changes frequently ✅ Technical or specialized domain

Best Practice:

Use RAG - It combines best of both:
  • Uses intents when they match
  • Falls back to document search
  • Provides most accurate answers
  • Best user experience

Comparison Tips

What to Compare

Response Quality:

  • Accuracy of answers
  • Relevance to your content
  • Completeness of responses

Response Source:

  • Where answer comes from
  • Can RAG cite sources?
  • Is basic AI guessing?

User Experience:

  • Which feels more helpful?
  • Which answers faster?
  • Which is more accurate?

Testing Checklist

□ General Questions - Compare both responses □ Specific Questions - See RAG advantage □ Intent Questions - Verify both use intents □ Unknown Questions - See fallback behavior □ Complex Questions - Test document retrieval

Understanding Vector Store Status

Status Indicators

On Comparison Page:

  • Shows vector store status
  • "Completed" = RAG fully functional
  • "Processing" = Building, wait
  • "Unavailable" = Need to build first

Impact on RAG:

  • Completed: RAG works, retrieves documents
  • Processing/Unavailable: RAG may not work properly
  • Need to Build: RAG won't retrieve documents

Checking Status

Before Comparing:

  • Verify vector store is "Completed"
  • Ensure knowledge base items are processed
  • Wait if status shows "Processing"

Account Considerations

All Accounts

  • Can use comparison feature
  • Same functionality
  • No account restrictions

Differences:

  • Vector store limits based on account
  • Document limits vary
  • But comparison works the same

Troubleshooting

RAG Side Not Working?

  • Check vector store status
  • Verify documents are completed
  • Ensure vector store is built
  • Try rebuilding if needed

No Difference in Responses?

  • Try questions about your documents
  • Use specific content queries
  • Check if vector store is working
  • Verify documents contain relevant info

Intentions Not Working?

  • Check toggle is enabled
  • Verify intents are defined
  • Test with specific intent questions
  • Check intent wording matches query

Best Practices

✅ Build Vector Store First - Essential for RAG comparison

✅ Test Specific Questions - Ask about your documents

✅ Compare Multiple Scenarios - Test various question types

✅ Use Real Queries - Test actual user questions

✅ Document Differences - Note when RAG is better

✅ Choose Based on Results - Use what works best

Next Steps

After comparing:

1. Decide on Deployment Type - RAG or Basic 2. Deploy Your Chatbot - Go live 3. Monitor in Production - See which works better 4. Optimize Based on Usage - Improve based on feedback

Comparison helps you make informed decisions about which chatbot type to use!

Was this page helpful?
Chatbot Icon
AIInSync Chatbot