
Realtor.com: In a New Test, AI Mortgage Assistants Got Nearly 1 in 4 Answers Wrong
In a new article for Realtor.com, reporter Allaire Conte spoke with Tidalwave Co-Founder and CEO Diane Yu about MortarBench, an open-source benchmark developed by Columbia University researchers and Tidalwave to evaluate AI on realistic mortgage origination tasks. MortarBench tests the kinds of questions loan officers ask when reviewing applications and bank statements. On the benchmark’s strictest measure, the strongest general-purpose model produced a fully correct answer 77.1% of the time. The research also found a troubling bias: deposits associated with non-English names were flagged as potentially foreign 77% of the time, compared with 13.3% for English names. The article places those findings alongside an earlier Tidalwave-Columbia benchmark comparing Tidalwave’s mortgage-trained SOLO with Claude 4.5 across 90 questions and 10 synthetic borrower scenarios. SOLO scored 84% overall compared with 71% for Claude, and 95% compared with 42% on yes-or-no compliance checks. As Yu told Conte, “The key difference is not just about utilizing AI,” but using it correctly. The findings reinforce the need for mortgage-specific systems, transparent testing, clear guardrails, and human oversight when AI is used in high-stakes lending workflows.
Read the full post


