remote website screenshots

Recently I wanted to check, if and what kind of webpages are available in a specific ip-address range. So I decided to scan the ips and make screenshots of the found services. Not as professional as archive.org or similar .. just a short look to get an idea. Problem was, that there was no tool which I just could fire up. So I started frickling some scripts ..

Step 1: scan the ip-range. I used nmap (what else) and logged the results in a file.

 nmap --open -p80,443 --host-timeout 3 --max-retries 2 172.16.0.0/16 > ll

That went quite fast.. Took only a few minutes. After that I started a small script for cleaning the result:

#!/bin/bash
grep -i "skipping" *ll > sl
awk '{ print $4}' < sl | sed s/\(// | sed s/\)// > sl2
grep "report for" *ll > l
awk '{ print $NF}' < l | sed s/\(// | sed s/\)// > l2
for i in ` cat sl2`
do
 grep -v $i l2 > xx
 mv xx l2
done
 
./l.sh
./i.sh

Okay .. here is the l.sh and i.sh:
l.sh: (start TOR first .. don’t want to annoy someone..)

for i in `cat l2`
do
torsocks wget –convert-links -B http://$i –no-check-certificate -t 1 -T 2 -O $i.html $i ; xvfb-run — wkhtmltopdf $i.html $i.pdf
torsocks wget –convert-links -B https://$i –no-check-certificate -t 1 -T 2 -O $i-443.html https://$i ; xvfb-run — wkhtmltopdf $i-443.html $i-443.pdf
done

Needed some tries with wget until I had an acceptable result. Played around with “-p” and “-r -l 1” and “-E” and  “-K” .. that one with just the -B worked best for me. So had the html-files.. but I wanted to have a quick look at them and did not want to start browsing local files. Therefore I transformed the html-files to pdf, and then (in the next step) I used convert to get png – files. (Did not find any html-to-png tools)

i.sh :

for i in `ls *pdf`
do
  convert $i `basename -s .pdf $i`.png 
done

(After that: copy the png-files to a place of your choice.., generate thumbnails..scroll around…)

There are some commcial vendors for services like this with much better quality (including zoomable thumbnails, galeries..you name it) but I wanted to have a quick’n dirty solution for free..

Though I am pretty sure that there are much better tools and hundred better solutions this worked for me.

Leave a Reply

You must be logged in to post a comment.