Reading ASSURE Test Results
In the ASSURE Directory you can read the test results for each Diamond Verification Instrument. The performance of the instruments has been evaluated through many performance metrics; in the ASSURE Directory we have chosen to highlight three key ones.
You can view these three key performance metrics, along with the other performance metrics, and download the test reports issued by the independent third-party laboratory UL, in the ASSURE Directory.
Calculating Performance Metrics
The ASSURE Tests are performed by an independent laboratory technician that processes a minimum of 1,200 stones from the ASSURE Sample through each Diamond Verification Instrument. The laboratory technician processes the stones through the instrument in accordance with the procedures outlined in the Diamond Verification Instrument Standard. The stones are sorted into piles based on the categorisation of the instrument.
The instruments sort the stones in different categories. All instruments have a “natural diamond” category, and in addition, they either have specific “synthetic diamond” and “diamond simulant” categories, and/or a “refer” category. The ‘refer’ category means that the nature of the stone in undetermined and the stone is referred for further testing.
The test technician determines the number of stones correctly and incorrectly sorted in each category. Thereafter, the performance metrics are calculated.
For instruments that can only separate diamonds and synthetic diamonds (Operation Category 1) there are six performance metrics. For instruments that separate diamonds from synthetic diamond and diamond simulants there is a greater number of performance metrics (Operation Category 2 and 3)
Click here to see a visualisation of how the performance results have been calculated.
It is important to keep in mind that the performance results are relative, not absolute, indications of the performance of the instrument in daily operations. The ASSURE performance results simply show how well the instruments categorise the ASSURE Sample. The ASSURE Sample is highly contaminated and some of the synthetic diamonds included are not representative of what you can find on the market.
Novice or Expert Operator
For an instrument that requires manual interpretation of the test results or needs to be operated by an expert to obtain the optimal performance of the instruments, the instrument is tested twice, once by a novice and once by a manufacturer appointed expert operator.
Novice operator implies that the operator is not an expert on the instrument but has received an introductory level of training. The novice training may include familiarisation with the user manual, review of video tutorials, and in-person training as prescribed by the manufacturer. The results of a novice operator are meant to be similar to the results of one who has recently purchased and begun using the instrument.
The manufacturer appointed expert is either an operator from the manufacturer that performed the testing at UL’s test facilities, or an UL technician trained by the manufacturer to be deemed an expert on operating the instrument. The results of an expert operator are meant to be similar to the results that can be expected after an operator becomes very familiar with a specific instrument.
The resulting performance metrics are reported separately. In UL’s summary test reports, you can access both the novice and expert test results. In the summary tables, we highlight the test results from the “expert” if one was appointed, otherwise the results are of a novice.
Performance Metric Groups
When calculating the performance, we look at four areas: 1. the ratio of stones wrongly categorised (“false positive rate”), 2. the ratio of stones referred to further testing (“referral rate”), 3. the ratio of the stones correctly classified (“accuracy”), and 4. the speed of the instrument (“speed”).
In each of the four areas, the number of metrics is determined by how granular the instrument categorises the stones. The more categories the instrument have, the more performance metrics are required to evaluate its performance. Click here to read more about Operation Categories.
All instruments have been evaluated on three key performance metrics:
- Diamond False Positive Rate is the ratio of synthetic diamonds (and diamond simulants if applicable) erroneously classified as Diamond to the total number of synthetic diamonds (and diamond simulants). This is the fundamental measurement for the performance of the instrument. The optimal Diamond False Positive Rate is 0% where no synthetic diamonds (or diamond simulants) are classified as natural diamonds.
For example, if 2 out of the 200 synthetic diamonds were erroneously identified as natural diamonds, the rate would be 1%.
Keep in mind that a 1% Diamond False Positive Rate doesn’t mean you can’t rely on this instrument in your operations. This does not mean that 1% of all synthetic diamond would be classified as natural diamonds, as our ASSURE Sample is very challenging and not representative of what is available in the market place. Having a challenging sample allows us to properly differentiate between the instruments.
- Diamond Referral rate is the ratio of diamonds categorised as refer or referral to the total number of diamonds. A referral is a stone that could not be classified by the Diamond Verification Instrument and requires further testing by a separate instrument or gemmological laboratory to determine whether the stone is a diamond, synthetic diamond (or diamond simulant if applicable).
For example, if 100 out of the 1,000 diamonds were categorised as refer, the Diamond Referral Rate would be 10%. This means that 100 diamonds would have to be subjected to further testing. The Diamond Referral Rate indicates how effectively the instrument classifies diamonds. The optimal Diamond Referral Rate is 0% where no diamonds are referred for further testing. Further testing entails additional testing time and often incur an additional cost.
- Diamond Accuracy is ratio of diamonds correctly categorised as diamond to the total number of diamonds. The optimal diamond accuracy is 100% where all diamonds are correctly classified as diamond.
- Manufacturer Chosen Metric. Each instrument has unique capabilities and to allow for these differences, the manufacturer choose a fourth performance metric from the list of performance metrics that have been evaluated by the ASSURE Test Process.
Imagine that you have 1,000 diamond and 200 synthetic diamonds. The instrument sort the stones into two categories:
- Diamonds: 902 stones (900 diamonds and 2 synthetic diamonds)
- Refer: 298 stones (100 diamonds and 198 synthetic diamonds)
The Diamond False Positive Rate is 1%, that is 2 synthetic diamonds out of 200 synthetic diamonds were classified as diamond.
The Diamond Accuracy is 90%, that is 900 out of the 1,000 diamonds were accurately categorised as diamond.
The Diamond Referral Rate is 10%, that is 100 out of the 1,000 are undetermined and therefor categorised as refer.
Click here for the PDF version of this page.