1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62
|
User-agent: Googlebot
Disallow: /search*
Disallow: /preview*
Disallow: /register*
Disallow: /login*
Disallow: /requestresetpass*
Disallow: /*/video/
Disallow: /shortcode/
Disallow: /preview/
Disallow: /api/
Disallow: /rtmobile/
Disallow: /listing/
Disallow: /widget/
Disallow: /schedulejson/
Disallow: /vote/
Disallow: *?
Allow: /static/fonts/icon/
Allow: /static/css/
Allow: /static/js/
User-agent: bingbot
Disallow: /search*
Disallow: /preview*
Disallow: /register*
Disallow: /login*
Disallow: /requestresetpass*
Disallow: /*/video/
Disallow: /shortcode/
Disallow: /preview/
Disallow: /api/
Disallow: /rtmobile/
Disallow: /listing/
Disallow: /widget/
Disallow: /schedulejson/
Disallow: /vote/
Disallow: *?
Allow: /static/fonts/icon/
Allow: /static/css/
Allow: /static/js/
User-agent: *
Disallow: /search*
Disallow: /preview*
Disallow: /register*
Disallow: /login*
Disallow: /requestresetpass*
Disallow: /*/video/
Disallow: /shortcode/
Disallow: /preview/
Disallow: /api/
Disallow: /rtmobile/
Disallow: /listing/
Disallow: /widget/
Disallow: /schedulejson/
Disallow: /vote/
Disallow: *?
Allow: /static/fonts/icon/
Allow: /static/css/
Allow: /static/js/
Sitemap: https://www.rt.com/sitemap.xml
Sitemap: https://www.rt.com/newssitemap.xml
|