Recipes
For Publishers
How we index your recipes, what we keep, and how to get something corrected or removed.
Last updated: September 19, 2026
What we index
Recipes stores discovery metadata only: title, publisher, author, canonical URL, image URL, cuisine, category, total time, yield, rating metadata, normalized ingredient signals, and timestamps. We do not store article prose, personal stories, subscriber-only content, or cooking instructions. We do not proxy or cache your images; we keep only the source image URL.
Every result shows your publisher name and links to your canonical page. Your site is the destination for the recipe.
How sources are approved
A source is never crawled on its own. Each publisher gets a source record that identifies the approved hostname, sitemap scope, rights basis, crawl delay, and a review owner. Approval requires reviewing the publisher's terms and documenting one of the following:
- Direct permission from the publisher
- A content or data license
- A documented review confirming the intended indexing use is permitted
Robots rules are treated as required crawl controls, not as a content license.
How the crawler behaves
The crawler identifies itself as RecipesBot/0.1 (+https://projott.com/recipes/publishers). To block it site-wide, add User-agent: RecipesBot with Disallow: / to your robots.txt, or email us for a domain opt-out.
- Honors robots.txt rules for RecipesBot and for *
- Restricts requests to the approved host and sitemap scope
- Honors the stricter of our configured delay and any publisher-provided delay
- Uses conditional requests and sitemap modification dates to avoid re-fetching
- Backs off on errors and stops on repeated 429 or 403 responses
- Never bypasses login walls, paywalls, bot challenges, or other technical controls
- Does not fetch comments, user profiles, or unrelated site sections
Corrections, removals, and opt-outs
Email info@projott.com with one of these subject lines:
- Recipes correction -- a result shows the wrong title, author, image, time, or link. Include the canonical URL and what should change.
- Recipes URL removal -- remove one or more specific pages. Include the URLs.
- Recipes domain opt-out -- remove your whole site and block future crawling. Include the domain.
Send the request from an address on the publisher's domain, or include another way for us to verify you control the site. Removal covers primary storage, search indexes, caches, and future crawl queues, so a removed page will not reappear.
Contact
Projott LLC · info@projott.com