From a user’s perspective, it’s difficult to know exactly how accurate your wearable is. For example, if you’re in the market for the best running watch, what are the chances that you’ll have the opportunity to strap on two different models at once and run the same route to compare that data? And even if you do manage to find a way to do so, how do you determine which of the two is more accurate?
These are the questions that keep me up at night. As TechRadar’s Fitness and Wearables Editor, it’s my job to figure out how to reliably test wearables in order to determine how accurate they are. My two gold standards are the Garmin Epix Pro (one of the best Garmin watches) and the Apple Watch Ultra 2 (which we’ve ranked the best smartwatch, period); but I’ve never before pitted them against each other, nor have I had a more reliable third party to test them against. So, I decided to do just that – running the TCS London Marathon.
Every London Marathon runner is given a bib that comes with a built-in GPS chip, which gives each runner an official time. Thereafter, that time is considered as a qualifying time for other races, such as the exclusive Boston Marathon, or in some cases, even world records.
As such, that GPS chip has to be accurate, verifiably so, in order to ensure runners have an accurate measurement of their race time. And so that GPS chip is the perfect reliable metric against which to measure two of the best smartwatches money can buy, while also evaluating how the popular run-tracking app Strava treats the information from my Garmin watch.
Of course, at the time I signed up I’d totally disregarded the 16 grueling weeks – well, 13 actually, with one week ill, and a further two significantly more chilled-out recovery and tapering weeks – of training that would be required to get me in shape. However, running the race was a singular experience; there’s nothing quite like the wall of noise from the crowd that buffets you as you run over the iconic Tower Bridge. It made all the pain worthwhile. I finished the race with a chip time of 4:00:54 – respectable, but slightly shy of my sub-four-hour goal.
So, what do Garmin, Strava and Apple have to say about one of the most famous foot races of all time? I’ve taken a look at the GPS files and information provided by all three services, in order to show you the difference between them. I tried to end the workout on both wearables at the point I crossed the finish line, and successfully did so, so there should only be a few seconds’ difference from a time perspective.
Garmin Epix Pro: statistics
- Recording time: 4:01:04
- Distance: 42.81km
- Average pace: 5:38/km
- Average heart rate: 166 BPM
- Total calories burned: 3,005.
The Garmin Epix Pro is my everyday running watch of choice. I welcome Garmin’s complex interface, in-depth running statistics, multitude of graphs plus recovery metrics, such as its Training Readiness Score. Its super-long battery life not only enables the collection of a decent amount of sleep and recovery data, but it also allows me to go on multiple long runs with the battery-sucking GPS mode active, without having to worry non-stop about having to recharge the watch.
My Garmin recorded my total distance as 42.81km. Although the race course was just over 42.2km (or 26.2 miles) in length, Garmin shows the distance I actually travelled, including weaving in and out of other runners and taking wide corners. While this might not sound like it would make up much extra distance, it adds up over four hours.
The Epix Pro recorded my average pace at 5 minutes, 38 seconds per kilometer, or 5:38/km. As well as all the other stats recorded, my average heart rate was 166 BPM during the race, measured from my left wrist, and total calories burned was 3,005 – the equivalent to around two large Domino’s original cheese & tomato pizzas, give or take a slice.
Apple Watch Ultra 2: statistics
- Recording time: 4:01:07
- Distance: 42.38km
- Average pace: 5:41/km
- Average heart rate: 168 BPM
- Total calories burned: 2,888
This is where things get interesting. Garmin and Apple use different GPS files, different heart rate sensors and different algorithms to process information, and as such also end up with slightly different results over the same race. The Apple Watch Ultra 2 uses what Apple calls a precision dual-frequency GPS system, combining L1 and L5 satellite bands to more accurately determine your position when you run through tall buildings – London’s more densely packed areas, for example.
Garmin also uses a multiband GPS technology it calls SatIQ – Satellite Intelligence – that determines the optimal GPS mode based on your environment. If you’re close to tall buildings or in a dense forest, for example, it will use multiband GPS. However, when a low-power GPS mode can achieve the same accuracy – in more open areas, for example, it will auto-switch to single band.
The end result is a difference of around 400 meters, or 0.4km – which doesn’t appear to be much on the face of it, but for elite athletes it could be the difference between running a marathon and disqualification.
One thing I noticed is that even though the Apple Watch Ultra is able to see where I am with pinpoint accuracy, when I dive into my splits (the time I spent over each kilometer of the race), Garmin gives me the numbers down to the nearest 10 milliseconds. Apple, in its customer-friendly approach, provides the data only to the second. Maybe this is where my extra 0.4km was registered. The Apple Watch also underestimated my heart rate by 117 calories (less than half a slice of pizza) compared to the Garmin watch.
Honestly, the difference is minor, especially over four hours of effort. Discovering which of the two is more accurate is difficult; but for the average person running 10km on a Saturday, the variances between the two watches are small enough to chalk them up to algorithmic differences.
Strava: statistics
- Recording time: 4:01:04
- Distance: 42.80km
- Average pace: 5:36/km
- Heart rate: 166 BPM
- Total calories burned: 3,005
Strava uses the information from the Garmin Epix Pro – including my Garmin’s raw GPS information – to arrive at its measurements, which is the reason the health stats such as heart rate and calories are so similar. However, Strava processes GPS data with its own algorithm, so there are slight differences.
Strava tends to slightly underestimate Garmin’s data: in a community post, it’s said the reason this happens is that Garmin (both the device and Garmin Connect) rounds up distances, whereas Strava rounds down. It’s only 0.01km slower, but considering both use the same GPS file, you may see differences on your device and on Strava when looking at recorded workouts.
There are a few other changes: distances are always stored in meters on devices and then shown on Strava in the athlete’s preferred units. The first number Strava gives you on its summary is Moving Time, not total time, so it discounts any quick stops.
My Moving Time was actually slightly under four hours; to discover the full time, I had to dive into the statistics under the “Analysis” section. Garmin, as a serious tool for serious people, doesn’t bother with protecting my fragile ego with a faster time, and simply provides the total time recorded on the watch.
Conclusion: Are the differences significant enough to matter?
Both these watches are equipped to handle serious races and distances, representing the cream of Garmin and Apple’s respective crops. They have long battery lives, tough casings, top-of-the-range sensor arrays and GPS offerings, and come in at similar premium price points. Since I’m not an elite runner, an annual marathon is the only way I can test both the watches limits – and my own.
For running, I don’t believe the differences between the two models will prove statistically significant for anyone other than elite runners. Regular amateur runners, even those running marathons or ultra-marathons, use watches to provide accurate locations and good information for training – and in this regard, both the Apple and Garmin watches are more than accurate enough to provide the information such runners will need.
And Strava? Well, if you’ve ever wondered why there are differences between your watch’s measurements and the information on your Strava account, now you know.