Published indicators
The quality indicators or key performance indicators (KPIs) on the quality dashboard relate to search (and associated written opinions) and to the examination procedure. The basis for the determined KPIs are, on the one hand, randomly selected patent applications that were evaluated by the Directorate Quality Audit with regard to possible quality deficiencies in 2023 and, on the other hand, results of a user survey for the years 2022/2023.
In more detail, the following KPIs are published for search:
- Written opinions in which the assessment of novelty or inventive step was incorrect: 6%
- Searches for which more relevant prior art documents could be found: 4%
- Users who rated the search products as good or very good: 80%
The KPIs under a) and b) are based on a random sample of 802 search products evaluated by the Directorate Quality Audit. The KPI under c) results from the aforementioned user survey.
The following KPIs are provided for the examination procedure:
- Grants where the assessment of novelty or inventive step was incorrect: 7.9%
- Grants with added subject matter issues: 5%
- Users who rate the examination products as good or very good: 78%
The KPIs under a) and b) are based on a random sample of 1050 grants evaluated by the Directorate Quality Audit. The KPI under c) results from the aforementioned user survey.
For the moment, the quality dashboard does not contain KPIs relating to opposition and appeal proceedings. However, this is planned for the future.
Further thoughts and possible benefit for applicants
The fact that the EPO reports transparently on the quality of its work and the associated deficits is highly appreciated. At first glance, the figures indicate a good or at least satisfactory level of quality. At least from the perspective of the EPO, which also publishes target values for the year 2024 on the quality dashboard. According to this, the portion of searches for which more relevant prior art documents could be found is just about on target (currently 4%, target: <4%). The same applies to the portion of grants with added subject matter issues (currently 5%, target: <5%). Moreover, the EPO would like to reduce the portion of written opinions on searches suffering from an incorrect assessment of novelty or inventive step from 6% to 5%. The biggest deviation from the target applies to grants for which the assessment of novelty or inventive step is incorrect. Here, the EPO would like to achieve a KPI of <5% by 2024, starting from the current 7.9%.
Nevertheless, having a closer look at the KPIs reveals that the portion of grants in which novelty or inventive step was assessed incorrectly is higher (!) than the portion of search products in which novelty or inventive step was assessed incorrectly. While 6% or about every 17th search product suffers from such an error, this portion rises to 7.9% or about every 13th grant. Assuming that the quality indicators were determined in a statistically through manner, this would mean that new errors concerning the assessment of novelty or inventive step tend to be added during the examination procedure instead of eliminating or correcting errors resulting from the search phase. This would be very unfortunate, especially since the examination procedure rests on the shoulders of a three-member examination division and provides for an exchange (at least in writing) with the applicant.
But what can an applicant learn from these quality indicators?
First of all, from a mathematical point of view, it is inadmissible to draw conclusions from the overall statistical analysis to an individual case. Nevertheless, every applicant has the understandable desire to obtain error-free search products and error-free examination results. Unfortunately, the applicant can only influence the practice of the EPO extremely indirectly, namely through application documents and submissions (with or without amended application documents) during the examination procedure. In other words, applicants must ensure that their patent applications are of high quality in order to (indirectly) avoid errors in search and substantive examination by the EPO. A high quality of the patent application will also serve the interests of the applicant in opposition and appeal proceedings in order to ultimately enforce his interests. In addition, applicants must ensure high-quality representation in the examination procedure, be it in house or external. This is the only way to identify potential errors resulting from the search phase and address them in a way that prompts the examining division to an appropriate correction. In view of the above KPIS, attention must also be paid to the prevention of additional errors occurring during the examination procedure.