An independent study by the Canadian patent search specialists Riahi Patents Inc. examined the efficiency of AI tools compared to benchmark data of non-AI tools. The study was conducted by several professional searchers with ten real inventions and it concludes that IPRally can strengthen the quality of a patent search significantly.

Methodology

The study was conducted in two parts. In both parts, ten different search cases designed by an experienced team were used. They cover eight different areas: mechanical, electrical, chemical, mobile application, software, medical devices, life science, and electronics. For each case, 7 to 10 essential technical features were identified. 

Multiple analysts with a similar amount of experience (7-9 years) with various search tools were asked to search for prior art for each case and to report the top 5 publications for each of them. All of the analysts were also introduced to the different AI platforms and trained to use them.

For IPRally, the analysts attended a 60-minute online product introduction and training session. 

In the first part of the study, the analysts used separate platforms, either a traditional platform or an AI platform, while in the second part, they used both a traditional platform and an AI platform for each case  to study their combined power. The time limit for each case was 4 hours. 

Upon return of the cases, they were evaluated based on the number of essential features identified, with the evaluation grid looking like this:

The score for each case was normalized, in this case:

Normalized score=(7+6+5+4+4)/(10*5)=0.52 

Results

In the study, IPRally was found to be the best performing AI platform both when used alone and in combination with a conventional tool.

IPRally alone outperforms a boolean search

The experienced team behind Riahi Patents concludes that when used alone IPRally provided slightly better results than the traditional approach: the average score for the standard tool was 0.575 (with a standard deviation (SD) of 0.115) whereas IPRally score was 0.644 (with SD = 0.106).

This means that in a 4-hour search case, even an experienced searcher can find better prior art, that is, documents having on average 12% more essential features, using IPRally than with a conventional search tool. 

Combining IPRally with conventional searching gives you superpowers

Based on Riahi Patents’ report, we calculated the percentual improvement in the normalized scores for each case, when IPRally is used together with a conventional tool compared with the conventional tool alone. The results are shown in the figure below:

This means that the analysts found on average 29% more features when using IPRally. In many cases, even significantly higher improvement is reported. 

It should also be noted that in none of the cases, the introduction of IPRally decreased the quality of the search, given the 4-hour time frame that the searcher had.

Discussion‍

IPRally contributed to the study by offering the tool for Riahi Patents use and arranging a one-hour training session for the searchers just before the study was conducted. This validates that, despite the unique graph-AI approach, IPRally is fast to adopt and even professional searches can benefit from it notably much.

We expect that the benefits of AI will be even bigger if the time used for the search is shorter and for people who are not doing searches every day, such as patent attorneys, patent engineers and R&D engineers, due to the complexity and slow learning curve of boolean searching.

In this study, the benefits of a combined traditional+AI search are indisputable and as our AI models improve, the shift will be towards AI only.

Acknowledgement

Riahi Patents invested 120 hours in research and 60 hours in reviews and data analysis. We at IPRally are grateful for their effort to compare our tool to other patent search platforms. A wider study, where the above results are extracted, is to be published soon by Riahi Patents Inc.