User-agent: Facebot Crawl-delay: 10000 User-agent: Twitterbot Crawl-delay: 10000 User-agent: MSNBot Crawl-delay: 10000 User-agent: Slurp Crawl-delay: 10000 User-agent: Yahoo! Slurp Crawl-delay: 10000 User-agent: MS Search Crawl-delay: 10000 User-agent: Ocelli Crawl-delay: 80000 User-agent: Twiceler Crawl-delay: 80000 User-agent: Speedy Crawl-delay: 80000 User-agent: aipbot Crawl-delay: 80000 User-agent: 008 Crawl-delay: 80000 User-agent: Alexibot Crawl-delay: 80000 User-agent: Aqua_Products Crawl-delay: 80000 User-agent: asterias Crawl-delay: 80000 User-agent: b2w/0.1 Crawl-delay: 80000 User-agent: BackDoorBot/1.0 Crawl-delay: 80000 User-agent: Baiduspider Crawl-delay: 80000 User-agent: becomebot Crawl-delay: 80000 User-agent: BlowFish/1.0 Crawl-delay: 80000 User-agent: Bookmark search tool Crawl-delay: 80000 User-agent: BotALot Crawl-delay: 80000 User-agent: BotRightHere Crawl-delay: 80000 User-agent: BuiltBotTough Crawl-delay: 80000 User-agent: Bullseye/1.0 Crawl-delay: 80000 User-agent: BunnySlippers Crawl-delay: 80000 User-agent: CheeseBot Crawl-delay: 80000 User-agent: CherryPicker Crawl-delay: 80000 User-agent: CherryPickerElite/1.0 Crawl-delay: 80000 User-agent: CherryPickerSE/1.0 Crawl-delay: 80000 User-agent: Copernic Crawl-delay: 80000 User-agent: CopyRightCheck Crawl-delay: 80000 User-agent: cosmos Crawl-delay: 80000 User-agent: Crescent Crawl-delay: 80000 User-agent: Crescent Internet ToolPak HTTP OLE Control v.1.0 Crawl-delay: 80000 User-agent: DittoSpyder Crawl-delay: 80000 User-agent: DotBot Crawl-delay: 80000 User-agent: EmailCollector Crawl-delay: 80000 User-agent: EmailSiphon Crawl-delay: 80000 User-agent: EmailWolf Crawl-delay: 80000 User-agent: EroCrawler Crawl-delay: 80000 User-agent: ExtractorPro Crawl-delay: 80000 User-agent: FairAd Client Crawl-delay: 80000 User-agent: Fasterfox Crawl-delay: 80000 User-agent: Flaming AttackBot Crawl-delay: 80000 User-agent: Foobot Crawl-delay: 80000 User-agent: Gaisbot Crawl-delay: 80000 User-agent: GetRight/4.2 Crawl-delay: 80000 User-agent: Harvest/1.5 Crawl-delay: 80000 User-agent: hloader Crawl-delay: 80000 User-agent: httplib Crawl-delay: 80000 User-agent: HTTrack 3.0 Crawl-delay: 80000 User-agent: humanlinks Crawl-delay: 80000 User-agent: IconSurf Crawl-delay: 80000 User-agent: InfoNaviRobot Crawl-delay: 80000 User-agent: Iron33/1.0.2 Crawl-delay: 80000 User-agent: JennyBot Crawl-delay: 80000 User-agent: Kenjin Spider Crawl-delay: 80000 User-agent: Keyword Density/0.9 Crawl-delay: 80000 User-agent: larbin Crawl-delay: 80000 User-agent: LexiBot Crawl-delay: 80000 User-agent: libWeb/clsHTTP Crawl-delay: 80000 User-agent: LinkextractorPro Crawl-delay: 80000 User-agent: LinkScan/8.1a Unix Crawl-delay: 80000 User-agent: LinkWalker Crawl-delay: 80000 User-agent: LNSpiderguy Crawl-delay: 80000 User-agent: lwp-trivial Crawl-delay: 80000 User-agent: lwp-trivial/1.34 Crawl-delay: 80000 User-agent: Mata Hari Crawl-delay: 80000 User-agent: Microsoft URL Control Crawl-delay: 80000 User-agent: Microsoft URL Control - 5.01.4511 Crawl-delay: 80000 User-agent: Microsoft URL Control - 6.00.8169 Crawl-delay: 80000 User-agent: MJ12bot Crawl-delay: 80000 User-agent: MIIxpc Crawl-delay: 80000 User-agent: MIIxpc/4.2 Crawl-delay: 80000 User-agent: Mister PiX Crawl-delay: 80000 User-agent: moget Crawl-delay: 80000 User-agent: moget/2.1 Crawl-delay: 80000 User-agent: Mozilla/4.0 (compatible; BullsEye; Windows 95) Crawl-delay: 80000 User-agent: MSIECrawler Crawl-delay: 80000 User-agent: NetAnts Crawl-delay: 80000 User-agent: NICErsPRO Crawl-delay: 80000 User-agent: Nigma Crawl-delay: 80000 User-agent: Nigma.ru Crawl-delay: 90000 User-agent: Offline Explorer Crawl-delay: 80000 User-agent: Openbot Crawl-delay: 80000 User-agent: Openfind Crawl-delay: 80000 User-agent: Openfind data gatherer Crawl-delay: 80000 User-agent: Oracle Ultra Search Crawl-delay: 80000 User-agent: PerMan Crawl-delay: 80000 User-agent: ProPowerBot/2.14 Crawl-delay: 80000 User-agent: ProWebWalker Crawl-delay: 80000 User-agent: psbot Crawl-delay: 80000 User-agent: QueryN Metasearch Crawl-delay: 80000 User-agent: Radiation Retriever 1.1 Crawl-delay: 80000 User-agent: RepoMonkey Crawl-delay: 80000 User-agent: RepoMonkey Bait & Tackle/v1.01 Crawl-delay: 80000 User-agent: RMA Crawl-delay: 80000 User-agent: ScoutJet Crawl-delay: 80000 User-agent: searchpreview Crawl-delay: 80000 User-agent: SEOkicks-Robot Crawl-delay: 80000 User-agent: SiteSnagger Crawl-delay: 80000 User-agent: SpankBot Crawl-delay: 80000 User-agent: spanner Crawl-delay: 80000 User-agent: Spinn3r Crawl-delay: 80000 User-agent: SurveyBot Crawl-delay: 80000 User-agent: suzuran Crawl-delay: 80000 User-agent: Szukacz/1.4 Crawl-delay: 80000 User-agent: Teleport Crawl-delay: 80000 User-agent: TeleportPro Crawl-delay: 80000 User-agent: Telesoft Crawl-delay: 80000 User-agent: The Intraformant Crawl-delay: 80000 User-agent: TheNomad Crawl-delay: 80000 User-agent: TightTwatBot Crawl-delay: 90000 User-agent: toCrawl/UrlDispatcher Crawl-delay: 90000 User-agent: True_Robot Crawl-delay: 90000 User-agent: True_Robot/1.0 Crawl-delay: 90000 User-agent: turingos Crawl-delay: 90000 User-agent: TurnitinBot Crawl-delay: 90000 User-agent: TurnitinBot/1.5 Crawl-delay: 90000 User-agent: URL Control Crawl-delay: 90000 User-agent: URL_Spider_Pro Crawl-delay: 90000 User-agent: URLy Warning Crawl-delay: 90000 User-agent: VCI Crawl-delay: 90000 User-agent: VCI WebViewer VCI WebViewer Win32 Crawl-delay: 80000 User-agent: WBSearchBot Crawl-delay: 80000 User-agent: Web Image Collector Crawl-delay: 80000 User-agent: WebAuto Crawl-delay: 10000 User-agent: WebBandit Crawl-delay: 80000 User-agent: WebBandit/3.50 Crawl-delay: 80000 User-agent: WebCapture 2.0 Crawl-delay: 80000 User-agent: WebCopier Crawl-delay: 80000 User-agent: WebCopier v.2.2 Crawl-delay: 80000 User-agent: WebCopier v3.2a Crawl-delay: 80000 User-agent: WebEnhancer Crawl-delay: 80000 User-agent: WebSauger Crawl-delay: 80000 User-agent: Website Quester Crawl-delay: 80000 User-agent: Webster Pro Crawl-delay: 80000 User-agent: WebStripper Crawl-delay: 80000 User-agent: WebZip Crawl-delay: 80000 User-agent: WebZip Crawl-delay: 80000 User-agent: WebZip/4.0 Crawl-delay: 80000 User-agent: WebZIP/4.21 Crawl-delay: 80000 User-agent: WebZIP/5.0 Crawl-delay: 80000 User-agent: WWW-Collector-E Crawl-delay: 80000 User-agent: Xenu's Crawl-delay: 80000 User-agent: Xenu's Link Sleuth 1.1c Crawl-delay: 80000 User-agent: YandexBot Crawl-delay: 80000 User-agent: Zeus Crawl-delay: 80000 User-agent: Zeus 32297 Webster Pro V2.9 Win32 Crawl-delay: 80000 User-agent: Zeus Link Scout Crawl-delay: 80000