Login Or Subscribe?
Home Rulings VODACOM / SA’S BEST NETWORK / MTN / 2017-6178F

VODACOM / SA’S BEST NETWORK / MTN / 2017-6178F

MTN lodged a competitor complaint against the respondent’s claim "SA’s Best Network” as used on several advertising executions. The complainant provided a YouTube link as well as a copy of the advertising appearing on the respondent’s Facebook page as at 22 June 2017.

The YouTube commercial tells the tale of "Joseph”, a blind sculptor, who is reunited with a long-lost love with the assistance of a man relying on his Vodacom phone to search and ultimately find "Thandi”. At the end, when the pair are reunited, the voice-over states "Write the next chapter, with South Africa’s best network …” During this, the words "South Africa’s Best Network*” appear on-screen in large, white lettering against a red background. The asterisk at the bottom in small white lettering reads "*MyBroadband 28 March 2017. MyBroadband’s Internet speed test results show that Vodacom has the highest average mobile broadband speeds in South Africa”.

The Facebook advertisement offers the Huawei P10 and states, inter alia, "Master the art of portrait photography on SA’s Best Network*”.

Complaint

The complainant submitted that in some instances the respondent’s claim of having "SA’s Best Network*” is complemented by disclaimers reading "Network Guarantee”, "T’s & C’s apply” and/or "MyBroadband 28 March 2017. MyBroadband’s Internet speed test results show that Vodacom has the highest average mobile broadband speeds in South Africa”.

It argued that recent OOKLA data (between 1 March 2017 and 30 June 2017) shows that MTN and Vodacom’s network performance (in terms of average mobile download speeds) are on par, thus negating the respondent’s superiority claim. It submitted a letter from OOKLA in support of this argument. In addition, it referred to a document from Citizen Surveys, a SAMRA accredited market research entity. This document sets out Citizen Surveys’ analyses of the OOKLA data over the period 1 March 2017 – 30 June 2017, and confirms that MTN and Vodacom performed on par.

The complainant argued that the ASA has demonstrated a clear preference for OOKLA data over that of other data speed sources such as MyBroadband. This suggests that the respondent’s claim should be supported by OOKLA data, rather than MyBroadband data, failing which, the claim would not only be unsubstantiated, but also misleading. 

The complainant submitted that the general impression created is that the superiority is based on OOKLA data, when in reality it is based on MyBroadband’s research. However, the disclaimer relied on to clarify this is in small, illegible font. In addition, the reference to offering a "Network Guarantee” suggests that customers are guaranteed to receive the best network speeds on Vodacom. As this is not the case, the claim to offer "SA’s Best Network” in conjunction with a "Network Guarantee” exploit’s consumers’ credulity. This exploitation is exacerbated by the terms and conditions on the respondent’s website which fail to indicate that:
  • An interim MyBroadband report actually showed that MTN outperformed Vodacom, and
  • These MyBroadband reports have changed and are contradictory to the credible OOKLA reports which emanate from a global leader in internet performance testing.
Given the above, the claim should be withdrawn.

Relevant Clauses of the Code of Advertising Practice

In light of the complaint, the Directorate considered the following clauses of the Code to be relevant:
  • Section II, Clause 2 – Honesty
  • Section II, Clause 4.1 – Substantiation
  • Section II, Clause 4.2.1 – Misleading claims

Response

The advertising agency Ogilvy & Mather Johannesburg, on behalf of the respondent, rejected the argument that the ASA has established OOKLA as the only acceptable entity to substantiate network speed claims. It discussed the individual cases referred to in the letter of complaint and explained that most of these are irrelevant, as they date back to times when MyBroadband were incapable of testing smartphone performance, or when service providers were incorrectly comparing broadband in general to mobile broadband in particular. It submitted that the ASA Code is not prescriptive in terms of which entity should substantiate advertising claims, and places an onus on the advertiser to convince the ASA that the substantiation is valid, accurate, and applicable to the claims made.

In dealing with the merits, it relied on (and provided copies of) MyBroadband’s Evaluation Report, as well as a report from Catalyst Research & Strategy, a SAMRA accredited entity.

It refuted the notion that it was creating an expectation that its claims were based on OOKLA data, and added that both MyBroadband and OOKLA use equivalent methodologies, running on the same handsets in an over the top fashion and collect the same network performance metrics. While it acknowledges that OOKLA was launched several years ago and arguably has a larger base to survey, the MyBroadband version has already attracted more than 10 000 downloads and has run in excess of 100 000 tests during its first two months. This seems to be on par with OOKLA’s typical figures of 100 000 tests per month on South African networks, and it is expected that the two testing platforms will reach parity very soon. MyBroadband’s statistics are still valid and representative, its sample size and level of statistical significance justify the conclusions reached. This is confirmed by both MyBroadband’s documentation and the verification from Catalyst. As matters currently stand, the collective data from MyBroadband’s app supports the respondent’s claim.

Subsequent Correspondences

During the first week in July, the complainant submitted a graph it received from OOKLA. It explained that this graph is valid for the period 1 July 2017 – 30 September 2017 and clearly shows a difference of less than 1Mbps between MTN and Vodacom speeds achieved. It referred to OOKLA’s specific requirements that stipulate as follows:

"A tie occurs for any award if there is less than 1% deviation between the first and second place provider as determined by the Speed Score metric … This ensures that in any market we are determining clear winners based on what would be a noticeable difference in service to the end user”. 

Relying on the above, the complainant argued that the difference between it and the respondent falls within the parameters described, meaning that the parties are tied. This, in turn, means that the respondent is not able to claim superiority.

The respondent was afforded an opportunity to consider the new submissions and noted that the new OOKLA data received from the complainant reflects "4G LTE” technology only, whereas the claim at issue is made in respect of all mobile technologies, not just "4G LTE” technology. For this reason alone the new submissions are irrelevant, and should be ignored.

It attached copies of an OOKLA report for the "Date Range Q2 2017” which shows a "Speed Score” of 44.01 for Vodacom (with a "Download Trimean” of 47,77). MTN is reflected as having achieved a "Speed Score” of 39,05 (with a "Download Trimean” of 42,19). It added that the new data was prematurely submitted by the complainant, as OOKLA is still validating the data for the second quarter of 2017. It included an email from OOKLA dated 13 July 2017 which reads, inter alia, as follows:

"You are approved to make the ‘South Africa's Fastest Network’ claim based on our analysis of the Q2 Speed Score in South Africa for both Modern Devices and 4G LTE. Our recommendation is that you use our Modern Devices view for the claim due to the wider margin of victory. We also believe this is a better representation of the overall performance of a network since LTE coverage becomes a factor.
 
This claim can be used externally starting on July 17th, 2017 through the end of Q3”.

In addition, it argued that the 1Mbps difference concern raised by the complainant is an internal OOKLA rule and does not automatically apply to all other research. Irrespective of this, the claim in question is based on MyBroadband data, and not OOKLA data.

ASA Directorate Ruling

The ASA Directorate considered the relevant documentation submitted by the respective parties.

Clause 4.1 of Section II stipulates that advertisers must hold acceptable proof of all claims made, and that this proof should emanate from, or be evaluated by an independent and credible expert. In instances where survey data is relied on (as appears to be the case here), the Code stipulates, inter alia, that the survey "… shall emanate from an entity approved by, or acceptable to, the Southern African Market Research Association”, and that "… the accuracy of the claims based on the survey shall be confirmed by an entity approved by, or acceptable to, the Southern African Market Research Association”.

The complainant has made much of the fact that only OOKLA results should be accepted for network performance claims. The Directorate accepts that parties to network performance disputes have generally relied on OOKLA data, and that such data has been acceptable to the ASA for some time (refer to the Directorate ruling in the matter Vodacom "Baby Network” / Telkom SA / 22388 (28 August 2013) for example). However, this should not be misconstrued as suggesting that the ASA will only accept OOKLA data for network performance claims. While OOKLA appears to be a market leader in this regard, this does not automatically invalidate other methods of testing. 

In a recent ruling under the reference MTN / SA’s Fastest Mobile Network / Vodacom / 2017-6127F (22 June 2017), the Directorate was asked to determine whether or not MyBroadband data would be acceptable as substantiation for advertising claims or disputes. It held, inter alia, as follows:

"…The decision to accept or reject substantiation filed by parties must be based on the provisions of the Code as measured against the content and context of the relevant advertisement. There are, for example, instances where a publication did its own survey and an advertiser basis its claims on such a survey. It is for this reason that Clause 4.1.6 of Section II specifically requires advertisers relying on such survey evidence to ‘clearly state’ the source of such evidence in all advertising. 

The key differentiator is clarity, because consumers need to be in a position to make an informed decision based on the information contained in any advertisement. The Directorate can appreciate the respondent’s surprise at being confronted with a new set of research findings only nine days after a previous set emerged. However, this is not something that the Directorate holds any influence over, and it cannot disregard all future reliance on MyBroadband research by virtue of this fact alone”.

Nothing in the submissions before the Directorate materially change this view, and there is no reason to automatically disregard MyBroadband data simply because the parties have generally relied on OOKLA data in the past.

The information submitted by MyBroadband and Catalyst note, inter alia, that for the tests conducted via MyBroadband’s Mobile App between 19 April 2017 and 24 June 2017, Vodacom had an average download speed of 28,003Mbps (compared to MTN’s 25,093Mbps) and an average upload speed of 7,965Mbps (compared to MTN’s 8,472Mbps).

The more recent submissions by the respondent appear to suggest a marked difference between results for only "4G/LTE” devices as opposed to "Modern Devices” and appear to show that OOKLA has not completed its rigorous data validation process for the second quarter of the year. It also contains express verification from an OOKLA representative to the effect that the respondent is "… approved to make the ‘South Africa's Fastest Network’ claim based on our analysis of the Q2 Speed Score in South Africa for both Modern Devices and 4G LTE”.

The Directorate is mindful that the current claim is "South Africa’s best network” and not "South Africa’s Fastest Network”. However, the parties appear to agree that the overall expectation still relates to speed. This suggests that the respondent not only has a claim endorsement from MyBroadband, but also from OOKLA.

 
The complainant appears to hold evidence suggesting that its performance is on par with that of the respondent (albeit that the respondent argued that this parity relates only to "4G/LTE” performance). The Directorate, however, does not possess the technical know-how to compare the two sets of data and accurately and fairly discern which is more relevant and reliable. Furthermore, in Samsung LCD / Sony SA / 8000 (15 November 2006), the Directorate noted that supporting evidence submitted by a complainant does not automatically invalidate that relied on by a respondent. Given that the Code places the onus of proof at the foot of an advertiser making a claim, the Directorate will generally rely on a respondent’s evidence in determining whether a claim is valid. Should such evidence be accepted, a complainant with conflicting evidence may opt for arbitration in terms of Clause 16 of the Procedural Guide”.

This means that once the respondent has submitted satisfactory evidence for its claim (as appears to be the case here), the Directorate has no reason to reject the evidence. 

The report from Catalyst (an entity previously accepted by the ASA for the purpose of substantiating survey-type evidence) notes, inter alia, that:
  • "Statistically speaking, both samples (OOKLA at 100 000 and MyBroadband at 50 000) are extremely large and the margin of statistical significance at the 95% confidence level is marginal at 0,4 and 0,3 respectively.
  • In statistical terms, doubling a sample does not decrease the level of significance in half. Thus the larger sample that OOKLA has over MyBroadband; does not give it a meaningful edge over MyBroadband in terms of the margin of error – as shown in the point above.
  • Both samples are representative.
  • We can thus safely assume that the methodolgies [sic] employed by Ookla and MyBroadband are almost identical”.
This appears to show that the respondent’s claim of offering the best network (in the context of network speeds) appears to be adequately substantiated within the meaning of Clause 4.1 of Section II of the Code.

This brings the Directorate to the next concern raised by the complainant, namely that the respondent is creating a misleading impression that its superiority claim is based on OOKLA data, as opposed to MyBroadband data.

The complainant referred to several rulings where the point was made that OOKLA data is preferred, predominantly because no viable or local alternatives exist. This is clearly no longer the case, as discussed above. The complainant’s argument seems to be based on the fact that OOKLA had been the preferred expert entity until very recently, and that people are likely to assume that they still are. It also took issue with the "NETWORK GUARANTEE” logo being used, arguing that this reinforces the notion that this guarantee relates to superior performance, which is problematic given that the claimed superiority relates to MyBroadband tests, and not OOKLA tests.

The Directorate does not agree with this view, and is not convinced that an average consumer is concerned with or cognisant of which entity tested the network performance. There is also nothing before the Directorate to suggest that previous advertising executions made a point of informing consumers as to who did the testing. While it is accepted that the complainant, a commercial competitor of the respondent, is likely to have such industry-relevant knowledge and may have come to such a conclusion, the Directorate remains unconvinced that an average consumer would hold the same views.

While the Directorate acknowledges that the Facebook page makes no mention of the source of the superiority claim, and that the disclaimer in the television commercial is barely legible (refer to the provisions of Clause 4.2.6 of Section II for additional perspective), this does not detract from the fact that the respondent has submitted evidence (albeit from MyBroadband as opposed to OOKLA) in support of its claim. This means that, from the perspective of a hypothetical reasonable person, the claim to offer "SA’s Best Network” appears to be justified.

In light of this, the advertising is neither misleading, nor exploitative of the credulity of consumers. The advertising therefore does not appear to contravene the provisions of Clauses 2 or 4.2.1 of Section II of the Code.

The complaint is therefore dismissed.

Previous Ruling Next Ruling

© 2017 ASA. All rights reservedDeveloped and Maintained by JHNet Web Development