At iBrothers, we have generally stuck with Telstra when it comes to both mobile phone coverage as well as mobile broadband. The reason for this is that Telstra’s Next G network is supposedly the best of all providers for speed as well as coverage. Of late however, we have been noticing that, at times, the speeds we were getting from Telstra seemed to be slower. Coupled with anecdotal evidence from customers and family of similar issues, we decided it was time to conduct some actual real world tests of our own.

The idea for the speed test was to visit a few locations around the Hume and Moonee Valley area and compare Telstra, Optus and Vodafone from both a mobile device as well as a portable wifi hotspot connected to a Netbook. We selected locations based on distance from the city, indoor and outdoor locations as well as locations in built up areas as well as “out of the way”.

Testing Tools

Our testing tools consisted of:

iPad2 3G – representing the mobile device into which we slipped a micro sim for each carrier to run the tests.

Netbook – We used an MSI U100 netbook with Windows 7 and 2GB RAM connecting to each of the hotspots via Wi-Fi.

Wi-Fi Hotspots – We used two types of hotspots in the test, mostly due to not being able to obtain one that would work on all carriers. For Vodafone and Optus, we used the PocketWiFi (Huawei E585 as supplied by Vodafone) unlocked to accommodate the Optus Sim. For Telstra we used their Mobile Wi-Fi Modem (ZTE-MF30).

Software – We used Nirsoft’s download speedtest tool. This tool provided time based tests which were simple to replicate and gave a good indication of overall speed.

How We Tested

For each carrier we performed a standard set of tests.

On the iPad2, we used the Speedtest.net Mobile Speed Test app. The app is probably one of the best ways to test speed from an iPhone or iPad. For the tests we ensured that  the server it connected to was the same each time to ensure impartiality in each test.

The test results are provided in three measurements:

Ping: Ping is achieved by sending a small piece of data and testing the time it takes to reach the destination server and come back. Think of a ping like sonar used on a boat or submarine.

Download: This test downloads data from the server and determines the speed in which this is done. A download in the real world could be opening up a website or downloading music from iTunes or video from Youtube.

Upload: The reverse of download, this test measures the speed in which data is sent to the server. This would affect things like sending email.

On the netbook, we ran the download speedtest tool configured to retrieve data from three different locations. The results of these tests were purely to measure Download speed.

Download location 1 was a file on one of our hosting accounts at micron21. Micron21 servers are located in Australia and we felt that is was a good impartial place to pull a file from.

Download location 2 was from Apple’s servers downloading a copy of iTunes. These servers consistently provide speeds upwards of 60Mbps from our Bigpond Cable Unlimited account and as such, deemed an acceptable high speed server.

Download location 3 was from Microsoft’s server which is generally on par with the speeds we saw from Apple’s.

All download scores were then averaged to produce a final score.

It is often difficult to test raw speed over a network. Often, people will suggest downloading a file from within the carrier’s network, for example, when connected to Vodafone, download a file located at www.vodafone.com.au. Initially, we tried this test using a download from Optus, Bigpond (for Telstra) and Vodafone only to find that inconsistencies occurred with each and that for every carrier, the Optus server proved the fastest. This is why we ended up using three high availability servers for the test.

Locations tested

We selected five locations for this test.

The first was in the heart of Craigieburn seated at the Ferguson Plarre.  We found this to be the busiest coffee shop in the area and at the heart of the shopping centre and as such a good indicator of a “fringe” suburb at its busiest for human traffic and hopefully lots of people using their mobiles.

The second location was at Gellibrand Hill near the radar tower. Up on a hill, away from most human beings and infrastructure (bar the radar tower) should provide an indicator of speeds achievable away from a built up area.

The third location was at Gladstone Park shopping centre. The centre is generally known locally for its weaker mobile reception.

The fourth location was at DFO at The Coffee Club. Close to all carrier’s towers and near the freeway as well as a multitude of people, it represented a good mix of achievable speeds in a prime network location.

Lastly we tested at Moonee Ponds Central (again, at a Coffee Club), chosen as a good representative location closer to the CBD that would have a lot of people possibly accessing the various networks.

Test Results

Craigieburn

Higher number is better

iPad2 Optus Vodafone Telstra
Download 2.02 1.06 0.53
Upload 1.57 0.53 0.61
Ping 152 225 188

Netbook Optus Vodafone Telstra
Score: 2.41 4.09 2.08
Test 1 – micron21 3.23 3.87 1.85
Test 2 – Apple iTunes 2.12 4.21 2.27
Test3 – Microsoft 1.89 4.19 2.13

First up at Craigieburn, we saw Optus come out on top for the iPad2 speedtest and Vodafone for the netbook download test.

At first it appears puzzling as to why there would be such large discrepancies between the two tests but it does highlight a very interesting point and one worth considering before we go foward.

Two different hardware devices: The hardware found in both the Wi-Fi hotspot and the iPad2 are different. Different manufacturers with different antennae designs using different internal chipsets will provide different results.

Test Server Location: This is possibly the trickiest one of the bunch to get right. The location of the test server (in the iPad’s case it was Aussie HQ, with the netbook there were three we listed earlier) will factor into a download result based on how one provider gets to the server vs another. For instance, think of having to drive to say Albury. If you live in Melbourne, that will be a distance of 300kms. If you are in Sydney however, you are looking at double that. The same idea applies here Vodafone’s network may take longer to reach AussieHQ where Optus may be closer. The ping times are often indicative of this too however, network speed and congestion (the amount of people using the network) could also influence this.

Congestion: SpeedTest is available not only on the iPad but also the iPhone, Android handsets and pretty much any computer with a web browser. There is no telling how many people at one time are running a similar test and if many people are hitting the servers at once, you may get slower speeds reported back.

Phew! With that said, let’s keep on moving forward.

Gellibrand Hill

Higher number is better

iPad2 Optus Vodafone Telstra
Download 3.57 2.38 5.43
Upload 1.52 0.54 2.02
Ping 167 196 158

Netbook Optus Vodafone Telstra
Score: 3.06 4.19 2.57
Test 1 – micron21 4.02 4.79 2.75
Test 2 – Apple iTunes 3.46 4.78 2.37
Test3 – Microsoft 1.71 3.00 2.59

Telstra with the iPad2 shows a clear lead here in its speed test where Vodafone, yet again proves dominant in the Netbook test.

Gladstone Park

Higher number is better

iPad2 Optus Vodafone Telstra
Download 1.88 0.78 4.14
Upload 0.22 0.38 1.24
Ping 158 249 209

Netbook Optus Vodafone Telstra
Score: 0.21 0.00 4.85
Test 1 – micron21 0.36 0 4.94
Test 2 – Apple iTunes 0.22 0 4.77
Test3 – Microsoft 0.06 0 4.84

Gladstone Park proved to be interesting. As you can see, the iPad2 managed to perform all tests however; we were unable to get a decent enough connection on Vodafone to complete any of them.

This could have been due to the iPad2’s 3G chip or a software algorithm that pushed more power to the device in a weak signal area.

DFO Essendon

Higher number is better

iPad2 Optus Vodafone Telstra
Download 5.14 3.65 4.99
Upload 1.48 0.37 1.69
Ping 188 214 150

Netbook Optus Vodafone Telstra
Score: 3.56 4.45 4.99
Test 1 – micron21 4.77 5.12 4.82
Test 2 – Apple iTunes 3.01 5.05 5.12
Test3 – Microsoft 2.91 3.18 5.03

Optus and Telstra are neck and neck here in the iPad2 test. The netbook test shows Telstra pulling ahead.

Moonee Ponds

Higher number is better

iPad2 Optus Vodafone Telstra
Download 2.76 1.17 3.33
Upload 1.45 0.66 1.50
Ping 179 427 157

Netbook Optus Vodafone Telstra
Score: 1.40 4.33 3.10
Test 1 – micron21 1.2 3.97 1.49
Test 2 – Apple iTunes 1.76 4.49 3.43
Test3 – Microsoft 1.25 4.53 4.39

Here we saw a sizeable disparity between the iPad2 and the netbook tests.

It should be noted though that we had to restart the Vodafone test three times before it would even start. There seemed to be some issues either with the Wi-Fi modem or the network itself. The iPad2 test could be proving a problem in the area with Vodafone at the time of testing.

Anomalies

It should be noted that the way we tested is not indicative of any hard ruling on a network carrier. There are many variables which would provide very different outcomes such as network congestion (when there are a lot of people all vying for network access on the same tower), the network penetration of a building (Telstra’s nextG is generally noted as being currently better at getting signal inside of buildings however, that may shift as the other carriers improve their networks) and distance from the carrier’s towers.

The distance issue is an interesting one. In Craigieburn, we decided to do a quick test of moving a few kms down to road to the McDonalds car park to conduct a quick test to determine any differences in speed from the shopping district. The results were interesting.

Netbook Optus Vodafone Telstra
Score: 3.65 3.09 3.13
Test 1 – micron21 4.66 2.93 3.91
Test 2 – Apple iTunes 2.63 3.16 3.47
Test3 – Microsoft 3.67 3.17 2

What we can see here is an increase in speed from both Optus and Telstra with Vodafone dropping down.

Of course, no one really wants to be travelling around in a car to find the best location for downloading. Some people need the best speed possible from their home. Others require data connection wherever they are heading.

Of course the further you move towards the CBD, the stronger the showing on most providers.

A quick test in Brunswick came back with the following results:

Netbook Optus Vodafone Telstra
Score: 3.20 4.06 4.29
Test 1 – micron21 2.67 4.06 4.44
Test 2 – Apple iTunes 3.65 4.68 3.83
Test3 – Microsoft 3.28 3.44 4.61

Conclusion

Overall it appears that both Optus and Vodafone are putting in a pretty good show, often beating Telstra in terms of raw performance. This pretty much reflects what we have been noticing of late.

Some are putting it down to Telstra’s newer, more competitive, pricing which is bringing more people onto the network but creating more congestion.

While Telstra seem to be offering an overall slower experience in some of the tests (notably in the areas iBrothers have witnessed as well as our clients), they are at least consistent in their presence. The most notable is at Gladstone Park where the other carriers are virtually nonexistent.

Averaging out all the scores we see the following:

Final Results

Higher number is better

iPad2 Optus Vodafone Telstra
Download 3.07 1.81 3.68
Upload 1.25 0.50 1.41
Ping 168.8 262.2 172.4

Netbook Optus Vodafone Telstra
Score: 2.13 3.41 3.52

Tell someone who cares

If you happen to experience poor service in your area, either with slow speed or poor signal strength, it’s worth using the feedback options supplied by the respective carrier

Telstra
https://www.telstra.com.au/mobile/networks/feedback.cfm

Vodafone
http://community.vodafone.com.au/t5/Network/bd-p/network

Optus
call 1300 307 937