DeepEval Configuration Guide

This guide will help you configure DeepEval and submit evaluation metrics using the Evaluation API.

1. Required Fields

Field NameDescription
metric_keyStandardized name (e.g. AnswerRelevancyMetric)
valueRaw metric value from DeepEval (float)

2. Sample DeepEval Code (Python)

from deepeval.metrics import (
    AnswerRelevancyMetric, HallucinationMetric, BiasMetric,
    RoleAdherenceMetric, ToolCorrectnessMetric
)
from deepeval.test_case import LLMTestCase

# Sample test case
test_case = LLMTestCase(
    input="What is the capital of Germany?",
    actual_output="Berlin is the capital of Germany.",
    expected_output="Berlin",
    retrieval_context=["Germany is a country in Europe. Berlin is its capital."]
)

# Instantiate metrics
metrics = [
    AnswerRelevancyMetric,
    HallucinationMetric,
    BiasMetric,
    RoleAdherenceMetric,
    ToolCorrectnessMetric
]

# Run evaluations
metric_results = {}
for m in metrics:
    m.measure(test_case)
    metric_results[m.__class__.__name__] = m.score

Code Sample: Google Drive Link

3. Metric-to-Pillar Mapping (Partial)

Metric Name Canonical Space Pillar Better High
AnswerRelevancyMetric relevance_and_accuracy Performance Yes
ContextualPrecisionMetric relevance_and_accuracy Performance Yes

4. API Submission Payload

{
  "metric_metadata": {
    "application_name": "chat-application",
    "version": "1.0.0",
    "resource_name": "chat-completion",
    "resource_id": "R-756",
    "url": "https://api.example.com/chat",
    "provider": "deepeval",
    "use_case": "transportation"
  },
  "metric_data": {
    "resource_id": "res_123456",
    "resource_name": "chat-completion",
    "deepeval": {
        "AnswerRelevancyMetric": 85,
        "ContextualPrecisionMetric": 92,
        "ContextualRecallMetric": 78,
        "ContextualRelevancyMetric": 88,
        "ConversationCompletenessMetric": 95,
        "ConversationRelevancyMetric": 82
    }
  }
}

Send via Curl

curl -X POST https://api.cognitiveview.com/v1/evals \
  -H "Content-Type: application/json" \
  -d @eval_payload.json

Summary Steps

StepAction
✅ 1Select relevant DeepEval metrics
✅ 2Run metrics and gather scores
✅ 3Submit to /v1/evals or mcp://... endpoint