Complete Guide to Applebot-Extended — Apple Intelligence Training Control Token
What is Applebot-Extended
Applebot-Extended is a robots.txt policy token with the same structure as Google-Extended. It is not an independent crawler and has no separate User-Agent string. It is a signal controlling whether content collected by Applebot (Apple's default crawler) may be used to train Apple's generative AI models.
TL;DR
Applebot-Extended = token controlling consent for Apple Intelligence (generative AI) training. Not a bot, so IP blocking is meaningless. Blocking does not affect Siri, Spotlight indexing, or search exposure. Same structure and effect as Google-Extended.
Applebot vs Applebot-Extended
| Item | Applebot | Applebot-Extended |
|---|---|---|
| Type | Independent crawler | robots.txt policy token |
| Own User-Agent | ✅ Yes | ❌ No (reuses Applebot UA) |
| Primary Use | Siri, Spotlight, Safari search indexing; Apple foundation model training | Controls consent for generative AI model training |
| Block Effect | Excluded from Siri and Spotlight index | Restricts generative AI training only; indexing maintained |
| Configuration | User-agent: Applebot | User-agent: Applebot-Extended |
Apple's official documentation (support.apple.com/en-us/119829, verified June 2026) states that "Applebot-Extended does not crawl web pages directly; it only controls how crawled data is used."
What Applebot-Extended controls
According to Apple's official documentation, Applebot-Extended controls:
- Apple generative AI model training: Using content collected by Applebot to train Apple Intelligence and other generative AI features
Not affected by blocking:
- Applebot's general crawling (continues as normal)
- Siri, Spotlight, and Safari search results exposure
- Inclusion in Siri Knowledge
Three robots.txt examples
Scenario A. Full allow (default)
# No separate configuration required.
# Applebot crawls as usual and content is also used for Apple Intelligence training.
Scenario B. Block Apple Intelligence training only (maintain Siri and Spotlight exposure)
# Block Apple generative AI model training
User-agent: Applebot-Extended
Disallow: /
# Applebot continues crawling → Siri and Spotlight exposure maintained
Scenario C. Block specific paths only
# Exclude /private/ path from Apple Intelligence training only
User-agent: Applebot-Extended
Disallow: /private/
Comparison with Google-Extended
The two tokens have nearly identical structure and behavior. The difference is only the company and AI product they control.
| Item | Google-Extended | Applebot-Extended |
|---|---|---|
| Operator | Apple | |
| AI Controlled | Gemini models, Vertex AI | Apple Intelligence |
| Search Impact When Blocked | None | None |
| Token Structure | Same | Same |
| Official Documentation | Google Search Central | Apple Support |
Blocking both tokens excludes your content from generative AI training by Google and Apple.
User-agent: Google-Extended
Disallow: /
User-agent: Applebot-Extended
Disallow: /
Recommended scenarios
Most SMBs: Scenario B recommended. Restrict Apple Intelligence training data provision without affecting Siri and Spotlight exposure. Apple Intelligence service availability varies by region, so check Apple's official announcements periodically.
Apple ecosystem exposure priority: Scenario A. Avoid blocking Applebot itself to keep Siri and Spotlight working normally. Allowing Applebot-Extended is an additional choice.
Verification
Applebot-Extended has no separate User-Agent, so it cannot be identified directly in server logs. General Applebot traffic can be checked in server logs using the Applebot keyword.
# Check Applebot traffic in server logs
grep -i "Applebot" /var/log/nginx/access.log | awk '{print $4, $7}' | tail -20
Frequently asked questions
Q. Can I block Applebot-Extended by IP?
A. It is meaningless. Applebot-Extended is a policy token, not a separate bot. Blocking Applebot IPs affects all Siri and Spotlight indexing. There is no way to selectively IP-block only Applebot-Extended.
Q. Is Apple Intelligence available in my region?
A. Apple Intelligence language support and rollout scope should be confirmed against Apple's official announcements. This article describes only features and behavior stated in Apple's official documentation and does not guarantee availability in any specific region.
Q. If I block Applebot-Extended, will I disappear from Siri?
A. No. Apple's official documentation states that "blocking Applebot-Extended does not prevent standard Applebot crawling and inclusion in search results." Siri and Spotlight results are handled by Applebot, which is separate from Applebot-Extended.
References
- Apple official Applebot documentation: https://support.apple.com/en-us/119829 (verified June 2026)