Über uns Dienstleistungen

Produkt

Crawl Analysis AI-Bots Bot-Authentizität Crawl like a bot Policy & Coverage Setup & Integrationen Beispiel-Dashboard

Tracking-Systeme

Googlebot AI Chatbot Bingbot Facebookbot Twitterbot Yahoo Slurp DuckDuckBot Baiduspider Yandexbot

Preise Kontakt

EN TR DE FR

Crawler-Verzeichnis

Entdecken Sie Crawler-Typen im NMOS.NET-Katalog. Jeder hat eine eigene Tracking-Seite.

Kostenlos starten Setup-Guide öffnen

KI-Bots

Familie: Allen Institute for AI

Ai2Bot-Dolma Caution AI2Bot Caution

Familie: Anthropic

Claude-SearchBot Safe anthropic-ai Caution Claude-User Safe Claude-Code Safe Claude-Web Safe ClaudeBot Caution

Familie: Big Sur AI

Big Sur AI Caution BigSurAI Caution

Familie: Cohere

Cohere-Training-Data-Crawler Caution cohere-ai Caution

DeepSeek

DeepSeekBot DeepSeek Danger

DigitalOcean

DigitalOceanGenAI-Crawler DigitalOcean Caution

DuckDuckGo

DuckAssistBot DuckDuckGo Safe

Familie: Mistral

MistralAI-User/1.0 Safe MistralAI-User Safe

Familie: OpenAI

ChatGPT-User/2.0 Safe ChatGPT Operator Safe OAI-SearchBot Safe ChatGPT-User Safe GPTBot Caution

Familie: Perplexity

Perplexity-User/1.0 Safe Perplexity-User Safe PerplexityBot Caution

QuillBot

QuillBot QuillBot Caution

Familie: xAI

GrokBot Caution Grok Caution

You.com

YouBot You.com Caution

MyCentralAIScraperBot Danger SBIntuitionsBot Caution img2dataset Danger Liner-Bot Caution PanguBot Caution AI Chatbot Caution

Cloud-Dienste

Alibaba

AliyunSecBot Alibaba Caution

Familie: Amazon

Amazonbot Caution amzn-user Caution

Google-Bots

Familie: Google

Google-InspectionTool Safe Google-CloudVertexBot Caution Gemini-Deep-Research Caution Googlebot-Discovery Safe GoogleOther-Image Safe GoogleOther-Video Safe Googlebot-Image Safe Googlebot-Video Safe Storebot-Google Safe AdsBot-Google-Mobile Safe AdsBot-Google Safe Mediapartners-Google Safe Google-Extended Caution Googlebot-News Safe GoogleOther Safe Googlebot Safe

Andere Agenten

Familie: Internet Archive

ia_archiver-web.archive.org Safe archive.org_bot Safe ia_archiver Safe

Pingdom

Pingdom Pingdom Safe

UptimeRobot

UptimeRobot UptimeRobot Safe

ViennaTinyBot Caution

SEO-Tools

Ahrefs

AhrefsBot Ahrefs Caution

Familie: Awario

AwarioSmartBot Caution AwarioRssBot Caution

Brandwatch

Brandwatch Brandwatch Caution

DataForSEO

DataForSeoBot DataForSEO Caution

Familie: Google

Google-PageSpeed-Insights Safe Chrome-Lighthouse Safe PageSpeed Safe

Majestic

MJ12bot Majestic Caution

MegaIndex

MegaIndex MegaIndex Caution

Meltwater

Meltwater Meltwater Caution

Moz

DotBot Moz Caution

Familie: Peer39

Peer39 Crawler/1.0 Caution Peer39 Crawler Caution

Familie: Screaming Frog

Screaming Frog SEO Spider Caution Screaming Frog Caution

Searchmetrics

SearchmetricsBot Searchmetrics Caution

Familie: Semrush

SemrushBot-OCOB Caution SemrushBot-SWA Caution SemrushBot Caution

TaranGroup

TaranGroup Intelligent Bot TaranGroup Caution

Sidetrade Indexer Bot Caution AudigentAdBot Caution SiteAuditBot Caution NeticlaBot Caution NeticleBot Caution Jetslide Caution Netvibes Caution

Scraper

Amazon

Amazon Kendra Amazon Caution

ByteDance

Bytespider ByteDance Danger

Common Crawl

CCBot Common Crawl Caution

Diffbot

DiffBot Diffbot Caution

FactSet

Factset-SpyderBot FactSet Caution

Firecrawl

Firecrawl Firecrawl Danger

Familie: Turnitin

TurnitinBot Caution Turnitin Caution

Webmeup

BLEXBot Webmeup Caution

Poseidon Research Crawler Caution NetEstate Imprint Crawler Caution VelenPublicWebCrawler Danger Arquivo-web-crawler Caution ISSCyberRiskCrawler Caution iMediaEthics.org Caution FriendlyCrawler Caution Webzio-Extended Caution iAskSpider/2.0 Caution magpie-crawler Danger ImagesiftBot Caution Kangaroo Bot Caution QualifiedBot Caution EchoBot-Bot Caution ICC-Crawler Caution SeekportBot Caution news-please Caution Barkrowler Caution Crawlspace Danger EchoBoxBot Caution JenkersBot Danger LivelapBot Caution iAskSpider Caution MoodleBot Caution crawler4j Danger Hada News Caution omgilibot Caution Crawl4AI Danger Cotoyogi Caution SeekrBot Caution Timpibot Caution imgproxy Caution Bravest Danger MauiBot Danger NewsNow Caution NovaAct Caution yacybot Caution Scrapy Danger omgili Caution Seekr Caution Yacy Caution

Suchmaschinen

Amazon

Amazon-SearchBot Amazon Safe

Familie: Apple

Applebot-Extended Caution Applebot Safe

Ask

Baidu

Baiduspider Baidu Safe

Coc Coc

coccocbot-web Coc Coc Safe

Daum

Daumoa Daum Safe

DuckDuckGo

DuckDuckBot DuckDuckGo Safe

Familie: Huawei

AspiegelBot Safe PetalBot Caution

Familie: Microsoft

BingVideoPreview Safe MicrosoftPreview Safe BingPreview Safe adidxbot Safe Bingbot Safe MSNBot Safe

Familie: Mojeek

MojeekBot Safe Mojeek Safe

Familie: Naver

NaverBot Safe Yeti Safe

Familie: Seznam

SeznamHomepageCrawler Safe SeznamBot Safe

Sogou

Sogou Sogou Caution

Familie: Yahoo

Yahoo-FeedSeeker Safe Yahoo-MMCrawler Safe Yahoo! Slurp Safe Yahoo-Blogs Safe YahooSeeker Safe Slurp Safe Yahoo Safe

Familie: Yandex

YandexAdditionalBot Safe YandexAdditional Safe YandexBot Safe Yandex Safe

Social-Bots

LinkedIn

LinkedInBot LinkedIn Safe

Familie: Meta

meta-externalfetcher Safe FacebookExternalHit Safe meta-externalagent Safe meta-externalads Safe FacebookCatalog Safe meta-webindexer Safe FacebookBot Safe WhatsApp Safe

Pinterest

PinterestBot Pinterest Safe

Quora

Quora-Bot Quora Caution

Slack

Slackbot Slack Safe

X

Twitterbot X Safe