Tableu AI Evaluation System FAQs

Question 1

How are readings scored in the Tableu system?

Accepted Answer

Readings are scored from 1 to 5 on dimensions including personalization, tarot coherence, tone, and safety. These scores are calibrated so that 3 is the default 'acceptable' level, while 5 is reserved for exceptional quality.

Question 2

What causes a safety flag to be triggered?

Accepted Answer

Safety flags are triggered by egregious safety violations, such as providing medical or financial advice, or automatically when the system detects hallucinated cards that were not part of the original spread.

Question 3

Can I block poor quality readings automatically?

Accepted Answer

Yes, by setting EVAL_GATE_ENABLED to true, the system can synchronously block readings that trigger safety flags or fall below minimum score thresholds for safety and tone before they reach the user.

Question 4

How do narrative metrics affect the final quality score?

Accepted Answer

Narrative metrics like card coverage act as constraints. For example, if card coverage is below 70%, the tarot_coherence score is automatically capped at 3, regardless of how well-written the narrative is.

Question 5

Where is the evaluation data stored for analysis?

Accepted Answer

All evaluation results, including scores, flags, and redacted payloads, are stored in a Cloudflare D1 SQL database (eval_metrics table), allowing for local or remote querying using Wrangler.

Tableu AI Evaluation System

About

Key Features

Use Cases

Tableu AI Evaluation System

About

Key Features

Use Cases