Background
This is the first of three projects this month: building a customer service AI for the company website.
Process
Before starting the project, I had a rough goal in mind: the AI should be able to answer most of the questions visitors ask about the company, and it needs to be both accurate and fast.
At first, I wasn’t familiar with Dify, so I copied the official documentation structure into a mind map. When I was unclear, I would use the mind map to focus on the key points.
I began by practicing with an example project from the official website. However, when I tried to adapt it to my own use, I found the AI’s responses were slow and chaotic. After a whole day of work, the issue was still unresolved. It wasn’t until I got home that I realized: I shouldn’t have used multiple branches! This is a very simple customer service AI scenario—there was no need for multiple branches. Additionally, the knowledge base was definitely problematic.
The next morning, after removing the multiple branches, the response time improved significantly. I also reorganized the knowledge base structure and selected a different large model. The final result was OK: fast and able to handle light usage.
Conclusion
When building applications with Dify, keep the following points in mind:
Knowledge Base and its Processing: Don’t always rely on crawlers. When there’s no budget, using PDFs directly makes more sense. Also, Dify’s document splitting still has issues; after splitting, always check and revise it. Adding tags and writing good knowledge base descriptions will make the AI indexing more accurate.
Logical Structure: Logic that doesn’t fit the specific scenario can increase response time.
Choosing LLMs: Select the LLM based on the business scenario. Don’t blindly trust a few specific LLMs.
Continuous Debugging: Quick failures → quick adjustments → quick failures also applies here. After making a change (e.g., to the knowledge base or LLM), test it quickly, see how it works, and analyze the core issue.
There are definitely patterns to building these kinds of applications. Figure out the Minimum Viable Product (MVP) and focus on implementing that first.
Pending Issues
I don’t know how to stress test an AI application. What happens when more than 100 people use it at once? I’m not sure. But does Dify really have use cases for AI applications with over 100 users? I’m skeptical.
I don’t know the essence of AI. Even though I only need to call the interface, understanding it would allow me to see the core issues, like why switching the logic resulted in faster response times, etc.
I don’t understand why, even though the knowledge is in the knowledge base, the AI can’t find it. For example, the company address is in the knowledge base—I've added all the descriptions and tags—but the AI still can’t find it. In the end, I had to copy and paste the address into several other split sections, and then it worked.