Esha Roy

I built an in-house, pip-installable Python evaluation library for LangGraph-based agents. A project only needs a lightweight agent configuration and a golden-script file describing expected tools, expected outputs and where in the interaction those tools should trigger.

The library runs the evaluation workflow, prints metric values in the terminal and generates output files — removing repetitive evaluation setup across internal agent projects.

Evaluates

Tool accuracy, redundant tool behaviour, tool-call rate, safety / PII, helpfulness-oriented LLM checks and latency.

Method

Golden-script scenarios, tool-call validation and LLM-as-judge style checks.

Current scope

Implemented for LangGraph, with a design that can be extended to other agent frameworks.

Impact

Reduced repeated evaluation work for teams building and testing agentic applications.

For CollateralIQ, I developed an XGBoost regression model using roughly 30,000 historical auction records to support construction-equipment and commercial-asset valuation.

The model combines asset age, usage, make/model, geography and market features to generate valuation ranges that support — rather than replace — underwriter judgement.

Outcome

A more structured, data-driven baseline for collateral review and less repetitive manual valuation effort.

I engineered multi-scale temporal features from high-frequency electricity-consumption data, then applied UMAP and KMeans to segment premises by behavioural consumption patterns.

Approach

Temporal feature engineering → dimensionality reduction → behaviour-based clustering.

Validation

Variance-ratio analysis to assess separation rather than accepting clusters at face value.