Why Current Text-to-SQL Benchmarks Fail Real-World Data Stores

Any text-to-SQL benchmark should address difficulties of real-world data stores

I argue that existing benchmarks for Text-to-SQL models are misleading because they rely on simplified datasets that ignore the complexity of real-world data stores. True evaluation must account for messy schemas, ambiguous natural language, and the intricate constraints found in actual production environments to ensure these tools are genuinely useful.