---
title: "Evaluate AI agents with IBM CLEAR & EvalHub on OpenShift AI"
description: "Reliable AI agents can fail in production through incorrect tool use, off-topic reasoning, or plausible but incorrect answers that unit tests, static benchmarks, and logs may not expose. This tutorial builds an eval-driven trace-evaluation pipeline on Red Hat OpenShift AI 3.4 or later, using MLflow to store structured agent traces and evaluation artifacts, IBM CLEAR to score reasoning quality, relevance, and tool usage with an LLM judge, and EvalHub to orchestrate OpenShift jobs. It uploads eight prerecorded multistep research-agent traces, each containing at least 30 spans, then verifies them in MLflow before deploying EvalHub, registering the ibm-clear provider, and configuring judge-model credentials. Standard mode evaluates steps holistically, while SPARC mode gives tool calls separate scoring rows; the resulting workflow produces critiques and ranked failure patterns with frequency, severity, and step or agent attribution."
---

# Evaluate AI agents with IBM CLEAR & EvalHub on OpenShift AI

[Red Hat](https://yomu.fyi/company/red-hat) · Hema Veeradhi, Surya Pathak · Sep 3, 2026

**Type:** Tutorial

## Summary

Reliable AI agents can fail in production through incorrect tool use, off-topic reasoning, or plausible but incorrect answers that unit tests, static benchmarks, and logs may not expose. This tutorial builds an eval-driven trace-evaluation pipeline on Red Hat OpenShift AI 3.4 or later, using MLflow to store structured agent traces and evaluation artifacts, IBM CLEAR to score reasoning quality, relevance, and tool usage with an LLM judge, and EvalHub to orchestrate OpenShift jobs. It uploads eight prerecorded multistep research-agent traces, each containing at least 30 spans, then verifies them in MLflow before deploying EvalHub, registering the ibm-clear provider, and configuring judge-model credentials. Standard mode evaluates steps holistically, while SPARC mode gives tool calls separate scoring rows; the resulting workflow produces critiques and ranked failure patterns with frequency, severity, and step or agent attribution.

## Context

AI agents can exhibit production failures that unit tests, static benchmarks, and logs do not fully reveal, including incorrect tool use, off-topic reasoning, and plausible but incorrect answers. Trace evaluation is used to analyze each execution step and identify recurring failure patterns, their frequency, severity, and origin.

## Approach / What changed

The pipeline uses MLflow to capture and store structured agent traces and evaluation artifacts, IBM CLEAR for LLM-as-a-judge error analysis, and EvalHub to submit and orchestrate CLEAR evaluation jobs as OpenShift jobs. It demonstrates both holistic Standard mode and SPARC mode, which separates tool calls for finer-grained scoring.

## Takeaways

- MLflow stores agent execution traces containing LLM calls, tool invocations, node transitions, inputs, and outputs; it also stores CLEAR evaluation scores and artifacts.
- IBM CLEAR generates critiques for reasoning quality, relevance, and tool usage, then clusters them into ranked failure patterns with frequency, severity, and step or agent attribution.
- Standard mode scores each agent step holistically, whereas SPARC mode places tool calls in separate scoring rows to distinguish tool-use correctness from reasoning quality.

**Tags:** [AI Agents](https://yomu.fyi/topic/ai-agents), [MLflow](https://yomu.fyi/topic/mlflow), [Model Evaluation](https://yomu.fyi/topic/model-evaluation), [OpenShift AI](https://yomu.fyi/topic/openshift-ai)

- Source: [Red Hat](https://developers.redhat.com/articles/2026/09/03/evaluate-ai-agents-ibm-clear-evalhub-openshift-ai)
- Source URL: https://developers.redhat.com/articles/2026/09/03/evaluate-ai-agents-ibm-clear-evalhub-openshift-ai
- Ingested by Yomu: 2026-09-03T08:00:50.543Z

[Read original post](https://developers.redhat.com/articles/2026/09/03/evaluate-ai-agents-ibm-clear-evalhub-openshift-ai)