Skip to main content

Useful diagnostic commands and tools for tuning your AI (for you as the owner)

During testing (both your own and external), you will often need to analyze why your AI Expert responded a certain way, or where the problem may be. The following commands are used to do this, which you as the owner of the instance enter into the chat with it.

/lastRole​

What it does: After any response from your AI instance, this command lists the name (ID) of the expert subrole that the AI just used to generate that response.

What it's good for: If the AI response seems odd, incorrect, or off-topic, this command will help you quickly determine if the problem is that the AI (for some reason) activated an inappropriate expert subrole, or if the definition of that subrole is not optimal. You can then check the contents of that role on Google Drive (see PART C, section 11.5).

/investigateAnswer:{message}​

What it does: This is a very powerful diagnostic tool. In place of {message}, enter the exact text of your question (or a question that your beta tester asked and that the AI answered problematically). Your BuddyPro instance will not only answer that question again, but also explicitly show you:

  1. The name (ID) of the expert subrole it used to generate the answer.
  2. The relevant snippets from your uploaded know-how (from your vector database) that were selected as the most relevant supporting evidence to answer your question.

What it's good for: This command gives you detailed insight into the "thought processes" of your AI. You can see exactly what information (your know-how, role definition) the AI drew from and why it answered the way it did. This is invaluable for identification:

  • Missing or insufficient information in your know-how.
  • Inappropriately defined or incomplete roles.
  • Situations where the AI may have used the correct role, but chose less relevant snippets from the know-how.

Based on these findings, you can then make targeted optimizations (complete the know-how, adjust the role definition, refine the System Prompt).

/del​

What it does: Deletes the last message in the current conversation from both the Telegram chat view and the AI's short-term memory.

What it's good for: Very useful for iterative debugging. For example:

  • Edit a System Prompt or add some know-how.
  • You want to see if that change affected the AI's response to a particular question.
  • You ask the same question again (without deleting the previous interaction), the AI could still be affected by the context of that (possibly incorrect) previous answer.
  • Use /del to "clean up" the last interaction and you can ask the question again "with a clean slate" to better see the impact of your edits.

/del2 and /del10​

What they do: They work the same as /del, but /del2 deletes the last 2 messages and /del10 deletes the last 10 messages.

What they're good for: If you need to "turn back the clock" in a conversation several steps so you can test a longer sequence of interactions after a change.

/test:{test_id}​

What it does: Switches you into the "test user" role with a unique identifier {test_id} (you can choose any ID, e.g. /test:client_A or /test:script_return_after_year). This test user will have its own, separate conversation history.

Why it's good: It allows you to simulate an interaction from the perspective of a brand new user, or test specific scenarios without being affected by your own conversation history as the owner. This allows you to better evaluate onboarding processes and AI behavior "from scratch". If you add test_ to the beginning of {test_id} (e.g. /test:test_franta), the conversation won't be encrypted, allowing better logging for possible technical troubleshooting.

/untest​

What it does: Switches you from the test user role back to your instance owner profile.

What it's good for: To return to full admin rights and your original conversation history.

/myid​

What it does: Prints the unique Telegram ID of the current profile you're communicating with BuddyPro (whether it's your owner profile or a test profile).

What it's good for: You need this ID for some other admin commands, such as adding a member to your team (/addUserToTeam:{userId}) or manually setting up trial messages (/setTrialMessages:{userId}:{count}).

Frequently Asked Questions (FAQ)​

Is there any way to find out why the AI answered so strangely or incorrectly?

Yes, primarily use the /investigateAnswer:{original question} command. This will show you what role the AI used and what parts of your know-how it drew from. Next, you can use /lastRole to identify the role used.

What is the best way to test whether changes to the System Prompt or the know-how had the desired effect?

Ask the question for which you expect to see a change in AI behavior. If the answer is not ideal, use /del (or /del2, /del10) to delete the last interaction. This will ensure that a new answer to the same question is not influenced by the previous (possibly incorrect) context. Then ask the question again.

Does it cost me any AI credits to use these diagnostic commands?

It costs nothing just to enter most diagnostic commands (e.g. /lastRole, /del). But if the command triggers a response from the AI (e.g. /investigateAnswer:{query} re-generates a response), then this generated response consumes AI credits by default.

Last updated on