File: www.mtv.com

package info (click to toggle)
python-protego 0.1.16%2Bdfsg-2
  • links: PTS, VCS
  • area: main
  • in suites: bullseye
  • size: 30,204 kB
  • sloc: python: 1,426; perl: 190; cpp: 33; sh: 12; makefile: 3
file content (35 lines) | stat: -rw-r--r-- 905 bytes parent folder | download | duplicates (4)
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
User-agent: *
Disallow: /search/
Disallow: /*source=SEM_
Disallow: /*partnersearch=
Disallow: /*searchterm=
Disallow: /*sicontent=
Disallow: /news/droplets/
Disallow: /beta/
Disallow: /*cmp=
Disallow: /*loomia_si
Disallow: /think/
Disallow: /games/arcade/addicting_games/
Disallow: /tools/
Disallow: /*kw=
Disallow: /inhouse/
Disallow: /community/
Disallow: /mobile/checkout/
Disallow: /tve/
Disallow: /dynamo/droplets/test/Misc.jhtml
Disallow: /feeds/content/
Disallow: /news/newsArtistRelated/*
Disallow: /include/
Disallow: /news/utag/
Disallow: /news/tag/
Disallow: /overdrive/
Disallow: /vma/social-presence
Disallow: /vma/artist-of-the-year-voting
Disallow: /vma/backstage-pass
Sitemap: http://www.mtv.com/xmlsitemap/events
Sitemap: http://www.mtv.com/xmlsitemap/showvideos
Sitemap: http://www.mtv.com/xmlsitemap/episodes
Sitemap: http://www.mtv.com/xmlsitemap/shows

User-agent: Twitterbot
Allow: /