# Block undesirable pages User-agent: * Disallow: /report/ Disallow: /ar/report/ Disallow: /profile/facebook/connect/* Disallow: /profile/* Disallow: /accounts/login/* Disallow: /ar/accounts/login/* Disallow: /accounts/activate/* Disallow: /m/accounts/login/* Disallow: /m/accounts/edit-number/ Disallow: /m/ar/accounts/login/* Disallow: /m/ar/accounts/edit-number/ Disallow: /m/ar/settings/?added__gte=* Disallow: /1000931/* Disallow: /sitemap-* Disallow: /subscribe/ Disallow: /estate/create Disallow: /account/ Disallow: /search/complexes Disallow: /mysearches/ Disallow: /email_alerts/ Disallow: /mylistings/ Disallow: /profile/ Disallow: /chat/ Disallow: /m/myfavorites/ Disallow: /dashboard/ Disallow: /shorter/ Disallow: /api/ Disallow: /m/api/ Sitemap: https://uae.dubizzle.com/sitemaps/sitemap.xml Sitemap: https://uae.dubizzle.com/en/property-for-sale/new-projects/uae-sitemap.xml # Rules for adsense bot User-agent: Mediapartners-Google Allow: / # Block crawling software User-agent: sitecheck.internetseer.com Disallow: / User-agent: Zealbot Disallow: / User-agent: MSIECrawler Disallow: / User-agent: SiteSnagger Disallow: / User-agent: WebStripper Disallow: / User-agent: WebCopier Disallow: / User-agent: Fetch Disallow: / User-agent: Offline Explorer Disallow: / User-agent: Teleport Disallow: / User-agent: TeleportPro Disallow: / User-agent: WebZIP Disallow: / User-agent: linko Disallow: / User-agent: HTTrack Disallow: / User-agent: Microsoft.URL.Control Disallow: / User-agent: Xenu Disallow: / User-agent: larbin Disallow: / User-agent: libwww Disallow: / User-agent: ZyBORG Disallow: / User-agent: Download Ninja Disallow: / User-agent: WebReaper Disallow: / User-agent: Screaming Frog SEO Spider Disallow: /