feat: add You.com API tools for search, content extraction, and research#5563
Open
EdwardIrby wants to merge 16 commits intocrewAIInc:mainfrom
Open
feat: add You.com API tools for search, content extraction, and research#5563EdwardIrby wants to merge 16 commits intocrewAIInc:mainfrom
EdwardIrby wants to merge 16 commits intocrewAIInc:mainfrom
Conversation
Add two new tools for the You.com API: - YouSearchTool: Web search with advanced operators, filters, and livecrawl support - YouContentsTool: Extract content from URLs in markdown, HTML, or metadata format Features: - Support for search operators (site:, filetype:, boolean logic) - Configurable parameters (count, country, freshness, safesearch) - Livecrawl options for full content extraction - Multiple output formats (markdown, html, metadata) - Comprehensive error handling and parameter validation - Full backwards compatibility with search_query parameter Implementation: - Follows crewAI tool patterns (BaseTool, EnvVar, Field) - Uses core requests dependency (no additional packages) - Includes README documentation with usage examples - API key available at https://you.com/platform/api-keys Testing: - 22 comprehensive tests covering initialization, functionality, and error handling - All tests use mocked API responses for reliability - Follows pytest patterns from existing tools
Addresses Cursor Bugbot review feedback by removing runtime kwargs support from _run methods and using class properties exclusively. This matches the established crewAI tool pattern (TavilySearchTool, SerperDevTool) where: - Tool schema defines agent-callable parameters (query/urls) - Configuration options are class properties set during initialization - _run methods use class properties directly, not runtime kwargs Changes: - YouSearchTool._run: accepts only query parameter, uses class properties - YouContentsTool._run: accepts only urls parameter, uses class properties - Updated tests to set configuration via tool initialization - Removed backwards compatibility for search_query parameter This ensures agents can properly interact with the tools while maintaining flexibility through tool configuration at initialization.
tools from root package
Addresses Cursor Bugbot review #3799879658 by adding YouSearchTool and
YouContentsTool to the root crewai_tools package exports.
This enables the documented import path:
from crewai_tools import YouSearchTool, YouContentsTool
Previously the tools were only available via:
from crewai_tools.tools import YouSearchTool, YouContentsTool
Changes:
- Add imports to crewai_tools/__init__.py
- Add tools to __all__ list in alphabetical order
Addresses Cursor Bugbot review #3799966503 by fixing two validation issues
in YouSearchTool:
1. Missing EN-US language code
- README documented EN-US as supported language
- Language Literal only included EN and EN-GB
- Added EN-US to match documented behavior
2. Unvalidated count parameter
- API expects count in 1-100 range
- No validation caused API errors with invalid values
- Added clamping: max(1, min(self.count, 100))
Changes:
- Add EN-US to Language Literal type
- Clamp count to valid range (1-100) before API request
Test coverage: 25/25 tests passing
Addressed multiple issues identified in Cursor Bugbot reviews: 1. Added EN-US language code to YouSearchTool Language Literal - Language was documented in API but not available in enum - Enables proper BCP 47 language filtering 2. Added count parameter validation (1-100 range) - Prevents API errors from out-of-range values - Clamps count to valid range before API request 3. Fixed API endpoint URLs - Search: api.ydc-index.io/search → ydc-index.io/v1/search - Contents: api.ydc-index.io/contents → ydc-index.io/v1/contents - Verified against canonical source (api.constants.ts) 4. Fixed livecrawl_formats conditional logic - Now only sends livecrawl_formats when livecrawl is set - Prevents unnecessary parameter in API requests 5. Updated test assertions to match new endpoint URLs All tests passing (25/25), linting and formatting checks clean. Resolves: crewAIInc#4481 (reviews #3799966503, #3800020934) Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>
PT-BR is a locale/language code, not an ISO 3166-1 alpha-2 country code. It is absent from the You.com API spec's country enum and Brazil is already represented by BR. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
EN-US is not present in the You.com Search API OpenAPI spec Language enum. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Implements YouResearchTool wrapping the POST /v1/research endpoint, supporting configurable research_effort levels (lite/standard/deep/exhaustive) with full test coverage. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
_run now falls back to self.research_effort when no per-call value is provided, consistent with YouSearchTool/YouContentsTool patterns. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
…gs across all You.com tools - Change YouSearchTool URL from ydc-index.io/v1/search to api.you.com/v1/agents/search (100 free searches/day without API key) - Make YOU_API_KEY optional for YouSearchTool (required=False, no __init__ validation) - Fix safesearch='off' being silently dropped due to truthy check; use checks throughout - Fix livecrawl_formats default from 'markdown' to None (only sent when livecrawl is active) - Remove unreachable KeyError catch blocks from all three tools - Update YouContentsTool URL from ydc-index.io/v1/contents to api.you.com/v1/contents - Update tests to match new URL, response shape, and add coverage for free-tier and safesearch=None Co-authored-by: factory-droid[bot] <138933559+factory-droid[bot]@users.noreply.github.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Reopen of #4481 with upstream/main merged and review fixes applied.
Changes from original PR (#4481)
Bug Fixes (from code review)
safesearch="off"silently dropped: Changed all parameter guards from truthy checks (if self.safesearch:) to explicit None checks (if self.safesearch is not None:). The old code would skipsafesearch="off"since"off"is falsy.KeyErrorcatch blocks: All three tools caughtKeyErrorin_run()but no code path can produce one. The only dict access isos.environ["YOU_API_KEY"]which is validated in__init__.livecrawl_formatsdefault: Changed from"markdown"toNone—livecrawl_formatsis only meaningful whenlivecrawlis set, so the default was misleading.Free-Tier Search (breaking change)
YouSearchToolendpoint fromhttps://ydc-index.io/v1/searchtohttps://api.you.com/v1/agents/search— this endpoint provides 100 free searches per day without an API key.YOU_API_KEYoptional forYouSearchTool: Removed the__init__validation that required the key. The header is only sent when the key is present. SetYOU_API_KEYfor higher rate limits.YouContentsToolURL fromhttps://ydc-index.io/v1/contentstohttps://api.you.com/v1/contents.Test Updates
{"hits": [...]}to{"results": {"web": [...]}}matching the new endpoint response formatsafesearch="off"andsafesearch=Nonelivecrawlwithoutlivecrawl_formatstest_you_search_tool_missing_queryto acceptValueError(raised by BaseTool validation)Original Summary
This PR adds three new tools that integrate You.com's API: