One Company Wants to Rank the Ad-Tech Industry. Another Calls Foul

It all started two weeks ago when Pixalate released its first Connected TV/OTT Seller Trust Index (CSTI), a ranking of the inventory quality of ad exchanges across the three biggest streaming platforms: Roku, Samsung and Amazon Fire TV.

SpotX was ranked second, fifth and 11th on Samsung, Roku and Amazon, respectively. Nick Frizzell, vp of inventory quality and planning at SpotX, then shared a rebuttal on why “companies like Pixalate don’t see the whole picture,” and essentially accused the measurement vendor of running a pay-for-play operation, where SpotX could climb back up the rankings if it entered into a commercial agreement with Pixalate.

“Ultimately, it is up to buyers to decide which criteria they use to evaluate the quality of SSPs. We support the idea of an industrywide evaluation to help in that assessment, but only if it is publicly clear which data is collected and how scores are calculated. We do not believe entering into a commercial relationship should be mandated, which influences the rankings,” he wrote.

Pixalate responded with its own blog post on Wednesday, explaining its methodology and pointing out “flawed assertations” from SpotX.

“First and foremost, let us be clear: Sellers are not required to be clients in order to achieve favorable rankings in Seller Trust Indexes,” according to the post.

Instead, Pixalate’s “opinions are based on our objective data sets, which consist predominantly of buy-side open auction programmatic traffic sources.”

A spokesperson added that Pixalate’s index is based on open-auction programmatic traffic.

Jay Seirmarco, svp of operations and legal affairs at Pixalate, told Adweek the company “should have been more transparent” about the data used to create the report, claiming that relying on buy-side information avoids any potential conflicts of interest.

“That just immediately undercuts the possibility of impropriety, or that someone could come aboard, be a client, and their sell-side data can impact the ranking. That’s just not know how these rankings work,” Seirmarco said.

When one black box meets another

This is Pixalate’s CTSI, but it shares other rankings—like of mobile in-app exchanges—that it uses to educate buyers on the state of the marketplace.

“There has to be some methodology to find quality. That’s what we’ve done here, using certain metrics that we feel are important,” said Amin Bandeali, co-founder and CTO of Pixalate.

For CTSI, Pixalate scored exchanges on household reach; channel share, or access to apps and inventory; invalid traffic (IVT), or how much fraud runs through the platform; server-side ad insertion (SSAI) transparency; and channel integrity, which evaluates traffic misrepresentation.

Each category is weighted differently. A spokesperson said quality-based metrics, like IVT, are weighted higher than quantity-based metrics, like household reach.

Pixalate points to its Media Ratings Council accreditation to validate its position of authority in OTT.

Meanwhile, SpotX points to its TAG certification to validate its position as a trusted and transparent exchange.

Classic dilemma

The two companies have essentially entangled themselves in a classic ad-tech dilemma: A black box takes an action, another company calls foul, and both point to independent auditors to support their stances in an effort to appeal to media buyers.

One of SpotX’s main points of contention is that it’s ranked below exchanges that don’t have direct relationships with the streaming platforms. For example, SpotX is Samsung’s only monetization partner, yet Verizon Media is ranked first on that platform.

There are many different ways of buying CTV inventory. While SpotX may be a direct seller of the inventory Samsung controls, there’s still other inventory that can be bought through different channels.

Pixalate also said that direct-sold inventory isn’t always of high quality.

“The simple fact is that there are numerous ways IVT can wreak havoc in a closed supply chain. Infrastructure fraud is one example, where an SSAI vendor double-fires impressions to manipulate the measurement process. Another example of such IVT is continuous play, even from ‘good’ OTT/CTV viewers,” the company said.

“In short, sometimes the picture is not as clear as it appears. And that’s where we believe our methodology—not focused on one particular metric or piece of conventional wisdom—provides a much-needed holistic view of what matters to industry stakeholders.”

Pixalate has published a separate blog post explaining its approach to measuring OTT ad fraud. A spokesperson added that it performs census measurement, meaning it renders IVT opinions on every impression instead of sampling.

Frustration with vendors

Other industry leaders have spoken up in support of SpotX and against what’s perceived to be a flawed ratings system.

“There will always be naysayers,” said Pixalate’s Bandeali.

Multiple sources have told Adweek they’ve encountered a similar pay-for-play issue with Pixalate. Generally, when a publisher’s inventory is flagged as fraudulent, or when an exchange wants to understand its low place in the rankings, a representative will approach Pixalate with questions.

When asked to provide more details on its methodologies, Pixalate would apparently instead offer to form a commercial agreement to then understand and ultimately fix the issue, according to the sources.

For example, SpotX ranked first in Pixalate’s 2016 Video Seller Trust Index but fell in subsequent rankings.

“We were surprised to see SpotX start to fall in the rankings, as access to premium inventory, transparent relationships and a fraud-free guarantee are core to our offering. Naturally, we reached out to Pixalate to learn more about what changed, what data they are gathering, and what their methodology is,” Frizzell wrote.

“This is when we quickly learned that these rankings lack any transparency. That, of course, could be changed if we were to engage in a commercial discussion to partner with Pixalate—they seemed willing to disclose some of their data sets and ‘help’ SpotX get back on top.”

Companies are often hesitant to freely share any kind of propriety information. Seirmarco said this issue is complicated for companies like Pixalate, since being fully transparent about methodologies could educate the very fraudsters they’re trying to weed out.

“What can we share that is helpful, but doesn’t result in gaming, doesn’t teach somebody, doesn’t induce somebody to take an action, to camouflage or to adapt? That’s a really tricky area,” said Seirmarco.