1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
|
GIF Creator
-----------
Shodan keeps a full history of all the information that has been gathered on an IP address. With the API,
you're able to retrieve that history and we're going to use that to create a tool that outputs GIFs made of
the screenshots that the Shodan crawlers gather.
The below code requires the following Python packages:
- arrow
- shodan
The **arrow** package is used to parse the *timestamp* field of the banner into a Python `datetime` object.
In addition to the above Python packages, you also need to have the **ImageMagick** software installed. If you're
working on Ubuntu or another distro using **apt** you can run the following command:
.. code-block:: bash
sudo apt-get install imagemagick
This will provide us with the **convert** command which is needed to merge several images into an animated GIF.
There are a few key Shodan methods/ parameters that make the script work:
1. :py:func:`shodan.helpers.iterate_files()` to loop through the Shodan data file
2. **history** flag on the :py:func:`shodan.Shodan.host` method to get all the banners for an IP that Shodan has collected over the years
.. code-block:: python
#!/usr/bin/env python
# gifcreator.py
#
# Dependencies:
# - arrow
# - shodan
#
# Installation:
# sudo easy_install arrow shodan
# sudo apt-get install imagemagick
#
# Usage:
# 1. Download a json.gz file using the website or the Shodan command-line tool (https://cli.shodan.io).
# For example:
# shodan download screenshots.json.gz has_screenshot:true
# 2. Run the tool on the file:
# python gifcreator.py screenshots.json.gz
import arrow
import os
import shodan
import shodan.helpers as helpers
import sys
# Settings
API_KEY = ''
MIN_SCREENS = 5 # Number of screenshots that Shodan needs to have in order to make a GIF
MAX_SCREENS = 24
if len(sys.argv) != 2:
print('Usage: {} <shodan-data.json.gz>'.format(sys.argv[0]))
sys.exit(1)
# GIFs are stored in the local "data" directory
os.mkdir('data')
# We need to connect to the API to lookup the historical host information
api = shodan.Shodan(API_KEY)
# Use the shodan.helpers.iterate_files() method to loop over the Shodan data file
for result in helpers.iterate_files(sys.argv[1]):
# Get the historic info
host = api.host(result['ip_str'], history=True)
# Count how many screenshots this host has
screenshots = []
for banner in host['data']:
# Extract the image from the banner data
if 'opts' in banner and 'screenshot' in banner['opts']:
# Sort the images by the time they were collected so the GIF will loop
# based on the local time regardless of which day the banner was taken.
timestamp = arrow.get(banner['timestamp']).time()
sort_key = timestamp.hour
screenshots.append((
sort_key,
banner['opts']['screenshot']['data']
))
# Ignore any further screenshots if we already have MAX_SCREENS number of images
if len(screenshots) >= MAX_SCREENS:
break
# Extract the screenshots and turn them into a GIF if we've got the necessary
# amount of images.
if len(screenshots) >= MIN_SCREENS:
for (i, screenshot) in enumerate(sorted(screenshots, key=lambda x: x[0], reverse=True)):
open('/tmp/gif-image-{}.jpg'.format(i), 'w').write(screenshot[1].decode('base64'))
# Create the actual GIF using the ImageMagick "convert" command
os.system('convert -layers OptimizePlus -delay 5x10 /tmp/gif-image-*.jpg -loop 0 +dither -colors 256 -depth 8 data/{}.gif'.format(result['ip_str']))
# Clean up the temporary files
os.system('rm -f /tmp/gif-image-*.jpg')
# Show a progress indicator
print(result['ip_str'])
The full code is also available on GitHub: https://gist.github.com/achillean/963eea552233d9550101
|
