Monitoring iPad network traffic, and blocking advert download.

An article on monitoring iPhone traffic by Craig Dunn got me wondering what the iPad is sending over the wire. That led me to blocking many of the adverts apps show. Here’s how.

1. Setup a proxy (squid on ubuntu)

First you need to setup a proxy, and send all your iPad’s network traffic through that. On Ubuntu squid is easy to setup: sudo apt-get install squid

Edit the config file in /etc/squid/squid.conf and comment in or out the following lines:

# Comment in - change if your local IP's don't start with 192.168
acl localnet src 192.168.0.0/16

# Comment out
#http_access deny CONNECT !SSL_ports

# Comment in
http_access allow localnet

# Make sure you have this line
access_log /var/log/squid/access.log squid

# Add this line in the log_fqdn section
log_fqdn on

# Set to off
strip_query_terms off

# We'll play with this later
hosts_file /etc/hosts

Restart squid: sudo restart squid. Tail the log file: sudo tail -f /var/log/squid/access.log

2. Set your iPad to use the proxy

On your iPad go into Settings then Wi-Fi, and tap the white / blue arrow by your network. In the HTTP Proxy section switch to Manual and type in the local IP address (192.168…) of your machine running squid, and port 3128 (squid’s default).

Open an app. You should see some traffic come through in your tail of the log. Not all apps use the network, but most do.

3. View the POST requests (tcpdump)

Your squid log will show the GET requests, but not the data of the POST request. You’re probably seeing lots of this: POST http://data.flurry.com/aap.do. Flurry seems to be the DoubleClick of iOS.

To view the POST contents, you need tcpdump: sudo apt-get install tcpdump. Then monitor TCP packets on port 80:

sudo tcpdump -i eth0 -A 'tcp port 80 and (((ip[2:2] - ((ip[0]&0xf)<<2)) - ((tcp[12]&0xf0)>>2)) != 0)'

(I copied that line from tcpdump’s man page)

Examples

Here’s what some of the apps on my iPad are sending

Angry Birds

POST http://data.flurry.com/aap.do - DIRECT/216.74.41.14 application/octet-stream
GET http://svrsecure-g2-aia.verisign.com/SVRSecureG2.cer - NONE/- text/plain
GET http://ax.init.itunes.apple.com/bag.xml? - DIRECT/63.80.4.50 text/xml
CONNECT p17-buy.itunes.apple.com:443 - DIRECT/17.154.66.17 -

Talking Tom

POST http://data.flurry.com/aap.do - DIRECT/216.74.41.14 application/octet-stream
GET http://outfit7-affirmations.appspot.com/rest/v1/news/ipad/talking/en/? - DIRECT/74.125.127.141 application/json
POST http://outfit7-affirmations.appspot.com/rest/talkingFriends/v2/iPad/? - DIRECT/74.125.127.141 application/json
GET http://apps.outfit7.com/static/icon/news/IconNewsFreeA72.png - DIRECT/74.125.127.121 image/png
GET http://apps.outfit7.com/static/buttons/rect/news/RectWhite120.png? - DIRECT/74.125.127.121 image/png
GET http://apps.outfit7.com/rest/talkingFriends/v1/push-notification/put/iPad/? - DIRECT/74.125.127.121 application/json
GET http://apps.outfit7.com/static/icon/news/IconNewsFreeA72.png - DIRECT/74.125.127.121 image/png
GET http://apps.outfit7.com/static/buttons/rect/news/RectWhite120.png? - DIRECT/74.125.127.121 image/png
GET http://apps.outfit7.com/rest/talkingFriends/v1/push-notification/put/iPad/? - DIRECT/74.125.127.121 application/json
GET http://apps.outfit7.com/ad/ad.jsp? - DIRECT/74.125.127.121 application/json
POST http://i.w.inmobi.com/showad.asm? - DIRECT/174.140.140.33 text/html
GET http://www.gstatic.com/afma/sdk-core-v40.js - DIRECT/74.125.127.120 text/javascript
GET http://googleads.g.doubleclick.net/mads/gma? - DIRECT/74.125.127.154 text/html
GET http://media.admob.com/gmsg.js - DIRECT/74.125.127.120 text/javascript
GET http://mmv.admob.com/p/i/4d/59/4d59f1cc8bcdb1e30dfa346b08195126-728x90.png - DIRECT/63.80.242.35 image/png

Here’s Talking Tom’s POST request to flurry.

For some reason data.flurry.com’s IP reverses to perureisen.com, a seemingly unrelated travel site. Maybe they share some virtual hosting.

turok is the name of my machine running the proxy. Notice the string IPHONEf0842dbXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXbce (I obscured part of it). That seems to be a unique id for my iPad, which all post’s to flurry share, and is presumably used to track me across apps.

Here’s the POST:

turok.local.39475 > perureisen.com.www
POST /aap.do HTTP/1.0
Host: data.flurry.com
User-Agent: TalkingCatIpad/1.7 CFNetwork/485.13.9 Darwin/11.0.0
Content-Type: application/octet-stream
Accept: */*
Accept-Language: en-us
Accept-Encoding: gzip, deflate
Pragma: no-cache
Content-Length: 501
Via: 1.1 turok:3128 (squid/2.7.STABLE9)
X-Forwarded-For: 192.168.1.79
Cache-Control: max-age=259200
Connection: keep-alive

turok.local.39475 > perureisen.com.www
E...h.@.@......@.J)..3.P....j.G7P.9.W....................0...3......R8THSKSDBS4YZQTXWLDP..1.7....IPHONEf0842dbXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXbce...1..z....
3........device.model.1..iPad1,1......1.7...3.._.........GridPageShown.....mericaUITouches......AppLaunched........AppLaunched....launchedVia..
GridPageShown....group..TalkingFriends.................GridClose.........f......... UITouches....touches..head.......4.................

Some of these POST’s to data.flurry.com include my location. The response is just an empty 200.

Most of the apps’s network traffic looks like those two, although sometimes they do weird things. An app called My Horse does this:

My Horse

POST http://data.flurry.com/aap.do - NONE/- text/html
GET http://ax.init.itunes.apple.com/bag.xml?ix=2 - DIRECT/63.80.4.24 text/xml
GET http://itunes.apple.com/WebObjects/MZStore.woa/wa/fetchSoftwareAddOns?appAdamId=421167112&bvrs=1.1.2&sfId=143441-1,9&offerNames=com.naturalmotion.h0rs3.110Gems,com.naturalmotion.h0rs3.1200Gems,com.naturalmotion.h0rs3.20Gems,com.naturalmotion.h0rs3.224Gems,com.naturalmotion.h0rs3.2500Gems,com.naturalmotion.h0rs3.42Gems,com.naturalmotion.h0rs3.450Gems&bid=com.naturalmotion.h0rs3&icuLocale=en_US&appExtVrsId=4160270 - DIRECT/63.80.4.65 text/xml
GET http://svrsecure-g2-aia.verisign.com/SVRSecureG2.cer - NONE/- text/plain
CONNECT p17-buy.itunes.apple.com:443 - DIRECT/17.154.66.17 -
PUT http://mf-horse-upload-rel.s3.amazonaws.com/audit/uid=1345522/directoryListing.txt - DIRECT/207.171.189.81 -
PUT http://mf-horse-upload-rel.s3.amazonaws.com/logFiles/uid=1345522/Fri_Sep_23_12:58:26_2011_144189.txt - DIRECT/207.171.189.81 -
PUT http://mf-horse-upload-rel.s3.amazonaws.com/logFiles/uid=1345522/Fri_Sep_23_16:30:00_2011_375253.txt - DIRECT/207.171.189.81 -
PUT http://mf-horse-upload-rel.s3.amazonaws.com/logFiles/uid=1345522/Sat_Oct__1_17:26:19_2011_317371.txt - DIRECT/207.171.189.81 -
[snip - lots and lots more logFiles PUTing]

Pretty weird huh? I suspect someone forgot to remove their debugging code, or else they are being exceptionally nosy.

Ad Blocking, privacy enhancement

You’ll notice a few apps hitting ad servers, then downloading advert images. You probably also don’t want all that monitoring nonsense. The best way to block those requests would be to keep running your iPad through a proxy, and use /etc/hosts to set relevant sites to 127.0.0.1. That requires a local PC running all the time, so instead I used a setting on my router to block certain sites.

Here’s a list of sites I’m blocking:

• data.flurry.com
• mob.adwhirl.com
• media.admob.com
• mmv.admob.com
• a.admob.com
• i.w.inmobi.com
• iphone.playhaven.com
• ingameads.gameloft.com
• ads.mobclix.com
• s.mobclix.com
• thisadworks.com
• c33.smaato.net
• c26.smaato.net
• i.xx.openx.com
• ads.mp.mydas.mobi

All apps still work fine. On one I get the words ‘Request failed’ where the advert usually appears.

Some of the apps (Tap Zoo for example) seem to download their state from the server, which comes as a cleartext JSON response. Using your proxy you might be able to substitute your own JSON and change things in interesting ways.

Happy blocking!