New report on ivory trade is statistically unsound, as the author herself notes

COMMENT: I have serious concerns about the new report published by the University of Portsmouth’s School of Law on the UK’s antique trade in ivory.

The Elephant in the Sale Room, as it is titled, is an exercise in futility. The real ‘Elephant in the Room’ here is the study’s vast shortcomings, rendering any solid conclusions at best misguided, at worst dangerous.

First, let’s take the statistics. There are two measures to consider here: margin of error and confidence in accurate results.

Statistically, to be 95% confident that the answers were an accurate reflection of the whole population – in this case the UK art and antiques market – while allowing for a margin of error of plus or minus 4% in the spread of answers – the standard for such studies – the sample size for a population of 20,000 should be just under 600, or 3% of the population.Ivory report copy

The sample size given here – 80 – is approximately 0.4% of the estimated population. Taken as a percentage of the Antiques Trade Gazette readership of 35,000, which I would see as a more accurate reading of the size of the market, that falls to 0.23% of the population, or just 7.5% of the minimum sample size needed to be confident of reasonably accurate results within a reasonable margin of error.

The sample size used in this case leaves a margin of error that allows you to drive two London buses through side by side.

Now add the fact that only around half of the sample actually answered a number of important questions and it gets worse.

For example, question 13 asked: How many of the following goods, either containing or made entirely from ivory, did you sell in 2015? This garnered a total of 39 replies, or 0.19% of the estimated population, rendering the response all but meaningless.

Questions 11, 12, 14, 15, 16, 17, 18, 19 and 21 met a similar level of response.

The researchers are struck by the fact that “none of the organisations that we researched had any specific advice on their websites regarding the laws and regulations on the sale of ivory”. The implication of this is that they are complacent or incompetent. However, at this point the report fails to acknowledge that the Government had removed its own advice from the internet because it was so confusing and misleading. If the Government can’t give accurate advice, how are the associations expected to?

The report does finally acknowledge the problem on page 42, where one of the 12 interviews supplementing the survey notes how “confusing” and “unhelpful” DEFRA’s website is on this.

Additional efforts, such as the 2016 CITES panel at the Art Business Conference, and Antiques Trade Gazette’s recent conference, which could have been added as a late footnote, are ignored entirely.

Perhaps most surprising and disturbing was the assumption made on page 25 of the report that the low response rate to the survey pointed to dishonesty among the trade, with dealers being “sometimes secretive regarding [their] commercial activities”, followed by a reference to Stuart Henry’s The Hidden Economy and “illegality” taking place in settings “which (on the surface) seem completely legal and this, in turn, makes participants disinclined to be open about their activities”.

This is staggering in its arrogance and complacency, blaming the failure of this poorly composed exercise on the “dodgy” trade, rather than looking to its own structure, methodology and execution for the true shortcomings.

How is anyone supposed to trust the authors as dispassionate and unbiased in this light?

At least, on the same page, the report goes on to admit: “with such a small sample it is difficult to make strong assumptions about the universe of the antiques trade”. Nevertheless, the report does just that, and unhelpfully too.

Turn to the next page, for instance, and immediately we are told: “The survey results show that some respondents failed to answer all of the survey questions [a huge understatement] suggesting that some questions were maybe too sensitive…”.

Page 32 makes the ‘astonishing’ discovery that auctioneers tend to sell more pieces than dealers, but this is hardly true of just ivory. If even a small-time auctioneer with only a monthly sale of 500 items and a 70% sell-through rate turns over 4200 lots a year, how many dealers could match that?

So does the report meet its three stated objectives?

  1. To evaluate types of ivory objects being sold in the UK, their source and the buyer’s demographic? (A: To a degree, no and no).
  2. To understand how traders appraise an item before sale to satisfy themselves whether or not it complies with the law (A: Partially, although until the conclusion on page 52, the report utterly ignores the crucial matter of the costs and time delay of carbon dating tests – recently estimated in parliament as averaging between £500 and £1000 per item).
  3. To evaluate the effect a total ban on the sale of ivory would have on the British antiques trade (A: Not even close, based on the sample size, response level and demonstrable lack of understanding of key considerations).

The report does make some sound recommendations – not least those to DEFRA – but none that has not already been mooted by the industry without having to resort to the time and expense of this exercise.

It at least acknowledges its own limitations under the first concluding recommendation: “The study highlighted the difficulties in obtaining information from the antiques trade about the nature of their practices regarding the sale of ivory. We would therefore recommend further research…”

Again the trade is blamed, whereas, in my view, the pointlessness of this study as executed is the real cause for complaint. How much did it cost? How could the money have been better spent?

Submit a Comment

Your email address will not be published. Required fields are marked *

You may use these HTML tags and attributes: <a href="" title=""> <abbr title=""> <acronym title=""> <b> <blockquote cite=""> <cite> <code> <del datetime=""> <em> <i> <q cite=""> <s> <strike> <strong>