Monitoring iPad network traffic, and blocking advert download.
An article on monitoring iPhone traffic by Craig Dunn got me wondering what the iPad is sending over the wire. That led me to blocking many of the adverts apps show. Here’s how.
1. Setup a proxy (squid on ubuntu)
First you need to setup a proxy, and send all your iPad’s network traffic through that. On Ubuntu squid is easy to setup:
sudo apt-get install squid
Edit the config file in
/etc/squid/squid.conf and comment in or out the following lines:
# Comment in - change if your local IP's don't start with 192.168 acl localnet src 192.168.0.0/16 # Comment out #http_access deny CONNECT !SSL_ports # Comment in http_access allow localnet # Make sure you have this line access_log /var/log/squid/access.log squid # Add this line in the log_fqdn section log_fqdn on # Set to off strip_query_terms off # We'll play with this later hosts_file /etc/hosts
sudo restart squid. Tail the log file:
sudo tail -f /var/log/squid/access.log
2. Set your iPad to use the proxy
On your iPad go into
Wi-Fi, and tap the white / blue arrow by your network. In the
HTTP Proxy section switch to
Manual and type in the local IP address (192.168…) of your machine running squid, and port
3128 (squid’s default).
Open an app. You should see some traffic come through in your tail of the log. Not all apps use the network, but most do.
3. View the POST requests (tcpdump)
Your squid log will show the GET requests, but not the data of the POST request. You’re probably seeing lots of this:
POST http://data.flurry.com/aap.do. Flurry seems to be the DoubleClick of iOS.
To view the POST contents, you need tcpdump:
sudo apt-get install tcpdump. Then monitor TCP packets on port 80:
sudo tcpdump -i eth0 -A 'tcp port 80 and (((ip[2:2] - ((ip&0xf)<<2)) - ((tcp&0xf0)>>2)) != 0)'
(I copied that line from tcpdump’s man page)
Here’s what some of the apps on my iPad are sending
POST http://data.flurry.com/aap.do - DIRECT/188.8.131.52 application/octet-stream GET http://svrsecure-g2-aia.verisign.com/SVRSecureG2.cer - NONE/- text/plain GET http://ax.init.itunes.apple.com/bag.xml? - DIRECT/184.108.40.206 text/xml CONNECT p17-buy.itunes.apple.com:443 - DIRECT/220.127.116.11 -
Here’s Talking Tom’s POST request to flurry.
For some reason data.flurry.com’s IP reverses to perureisen.com, a seemingly unrelated travel site. Maybe they share some virtual hosting.
turok is the name of my machine running the proxy. Notice the string
IPHONEf0842dbXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXbce (I obscured part of it). That seems to be a unique id for my iPad, which all post’s to flurry share, and is presumably used to track me across apps.
Here’s the POST:
turok.local.39475 > perureisen.com.www POST /aap.do HTTP/1.0 Host: data.flurry.com User-Agent: TalkingCatIpad/1.7 CFNetwork/485.13.9 Darwin/11.0.0 Content-Type: application/octet-stream Accept: */* Accept-Language: en-us Accept-Encoding: gzip, deflate Pragma: no-cache Content-Length: 501 Via: 1.1 turok:3128 (squid/2.7.STABLE9) X-Forwarded-For: 192.168.1.79 Cache-Control: max-age=259200 Connection: keep-alive turok.local.39475 > perureisen.com.www E...h.@.@......@.J)..3.P....j.G7P.9.W....................0...3......R8THSKSDBS4YZQTXWLDP..1.7....IPHONEf0842dbXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXbce...1..z.... 3........device.model.1..iPad1,1......1.7...3.._.........GridPageShown.....mericaUITouches......AppLaunched........AppLaunched....launchedVia.. GridPageShown....group..TalkingFriends.................GridClose.........f......... UITouches....touches..head.......4.................
Some of these POST’s to data.flurry.com include my location. The response is just an empty 200.
Most of the apps’s network traffic looks like those two, although sometimes they do weird things. An app called My Horse does this:
POST http://data.flurry.com/aap.do - NONE/- text/html GET http://ax.init.itunes.apple.com/bag.xml?ix=2 - DIRECT/18.104.22.168 text/xml GET http://itunes.apple.com/WebObjects/MZStore.woa/wa/fetchSoftwareAddOns?appAdamId=421167112&bvrs=1.1.2&sfId=143441-1,9&offerNames=com.naturalmotion.h0rs3.110Gems,com.naturalmotion.h0rs3.1200Gems,com.naturalmotion.h0rs3.20Gems,com.naturalmotion.h0rs3.224Gems,com.naturalmotion.h0rs3.2500Gems,com.naturalmotion.h0rs3.42Gems,com.naturalmotion.h0rs3.450Gems&bid=com.naturalmotion.h0rs3&icuLocale=en_US&appExtVrsId=4160270 - DIRECT/22.214.171.124 text/xml GET http://svrsecure-g2-aia.verisign.com/SVRSecureG2.cer - NONE/- text/plain CONNECT p17-buy.itunes.apple.com:443 - DIRECT/126.96.36.199 - PUT http://mf-horse-upload-rel.s3.amazonaws.com/audit/uid=1345522/directoryListing.txt - DIRECT/188.8.131.52 - PUT http://mf-horse-upload-rel.s3.amazonaws.com/logFiles/uid=1345522/Fri_Sep_23_12:58:26_2011_144189.txt - DIRECT/184.108.40.206 - PUT http://mf-horse-upload-rel.s3.amazonaws.com/logFiles/uid=1345522/Fri_Sep_23_16:30:00_2011_375253.txt - DIRECT/220.127.116.11 - PUT http://mf-horse-upload-rel.s3.amazonaws.com/logFiles/uid=1345522/Sat_Oct__1_17:26:19_2011_317371.txt - DIRECT/18.104.22.168 - [snip - lots and lots more logFiles PUTing]
Pretty weird huh? I suspect someone forgot to remove their debugging code, or else they are being exceptionally nosy.
Ad Blocking, privacy enhancement
You’ll notice a few apps hitting ad servers, then downloading advert images. You probably also don’t want all that monitoring nonsense. The best way to block those requests would be to keep running your iPad through a proxy, and use
/etc/hosts to set relevant sites to 127.0.0.1. That requires a local PC running all the time, so instead I used a setting on my router to block certain sites.
Here’s a list of sites I’m blocking:
All apps still work fine. On one I get the words ‘Request failed’ where the advert usually appears.
Some of the apps (Tap Zoo for example) seem to download their state from the server, which comes as a cleartext JSON response. Using your proxy you might be able to substitute your own JSON and change things in interesting ways.