Tool Definition Quality Score (TDQS)
Tool Definition Quality Score (TDQS) is an open-source library designed to evaluate and quantify the quality of tool definitions used in AI systems, with a particular focus on Model Context Protocol (MCP) servers. It provides a systematic approach to assessing how well a tool definition is structured, documented, and aligned with best practices, generating a numeric score that reflects its overall quality. The library examines various aspects of a tool definition, including the completeness and clarity of its description, the validity and richness of its parameter schema, the specification of return types, the presence of examples, error handling definitions, and consistency with community standards. By applying a set of customizable rules and heuristics, TDQS produces an objective measurement that can be used to compare, rank, and improve tool definitions. TDQS is particularly useful for developers building MCP servers or integrating function-calling capabilities into large language models (LLMs). It helps ensure that the tools exposed to AI models are well-defined, reducing the risk of misinterpretation or incorrect usage. The library can be easily integrated into development pipelines, such as pre-commit hooks, CI/CD workflows, or testing suites, allowing teams to enforce quality gates before deployment. Its output can serve as a benchmark for continuous improvement and as a metric for acceptance criteria. In addition to raw scoring, TDQS provides detailed feedback on each aspect of the definition, highlighting strengths and areas for improvement. This feedback is invaluable for developers who want to refine their tool definitions to maximize AI performance. The library supports extensibility, enabling users to define custom scoring rules that align with their specific project or organizational requirements. As part of the awesome-mcp-servers curation process, TDQS helps maintain a high standard of quality across curated servers. Built with simplicity and portabi
- Category
- Developer Tools
- Pricing
- Open Source
Problem solved
Developers and teams building AI tools or MCP servers often struggle to objectively assess whether their tool definitions are robust, clear, and likely to produce correct AI behavior. Poor definitions lead to misinterpretation by AI models, causing errors or suboptimal performance. TDQS solves this by providing an automated, quantifiable quality score, replacing subjective manual reviews with a consistent metric that can be integrated into development workflows to enforce standards and drive continuous improvement.
Use cases
- Automatically scoring tool definitions in CI pipelines to reject low-quality submissions.
- Curating high-quality MCP servers for the awesome-mcp-servers list.
- Analyzing an existing server's tool definitions to identify areas for refinement.
- Comparing tool definition quality across different servers or versions.
- Enforcing internal coding standards for function-calling definitions in enterprise AI projects.
- Tracking improvement of tool definition quality over time in an active development project.
Audience
- Developers
- ML Engineers
Pros
- Provides objective, reproducible quality metrics for tool definitions.
- Easy to integrate into existing development workflows (CLI or library).
- Enhances reliability and performance of AI tools interacting with definitions.
- Community-driven standard that can be tailored with custom rules.
- Open-source and lightweight, promoting wide adoption and extensibility.
Cons
- Scoring rules may not capture all subjective aspects of definition quality.
- Effectiveness depends on the chosen rule set; may not align with every use case.
- Limited to tool definitions; does not evaluate runtime behavior or model compatibility.
Official website: https://github.com/glama-ai/tool-definition-quality-score
Canonical LambdaBase page: https://www.lambdabase.com/tools/tool-definition-quality-score-tdqs