Autonomous Vehicle (“AV”) Level 4 and Level 5 Patent Landscape Update Through End of Year 2021
Given the emergence of autonomous vehicles (AV) and their expected impact on the global economy over the next decade, for over a year Tech+IP Advisory (www.techip.cc) has been cataloging patents describing and claiming technologies relating to Level 4 and Level 5 (L4 / L5) autonomous driving capabilities according to the definition published by the Society of Automotive Engineers (here). The first two of these reports, published in March 2021 and July 2021, respectively, grouped identified patents into three distinct periods based on their priority dates:
- 2010 – 2014 (the “Pioneer Phase”);
- 2015 – 2019 (the “R&D Phase”); and
- 2020 – Present (the “Productization Gen 1 Phase”)
In addition, and unlike many (in fact, nearly all) such patent landscapes, Tech+IP sought to provide transparency to the process by publishing its search string queries and results. We hoped that the community at large would benefit from the transparency and that experts and practitioners in the field would be incentivized to provide comments and feedback.
Tech+IP is pleased to report that it received substantial feedback from a number of individuals, including corporate patent attorneys and others, and has worked hard to incorporate that feedback into its search and identification methodologies, resulting in this report that incorporates all patents issued worldwide through December 31, 2021. The feedback from the AV community indicated, among other things, that while the version 1 landscape did a good job of avoiding “false positives”, a number of relevant technologies were omitted. Our updated search strings (explained in detail later in this document) and the maturation of patent applications into issued patents over the last several quarters have resulted in an enhanced dataset of approximately 72,000 active patents (about 46,000 patent families), more than two times the version 1.0 results. Another improvement we made for this version of the report was to conduct a more extensive backward citation analysis of patents identified in the AV L4 / L5 Patent Landscape to better substantiate pioneering patents and their owners.
Again, we ask that interested parties contact us with feedback and constructive criticism. We truly believe an iterative process will yield the most complete and most useful AV L4 / L5 Patent Landscape.
V2. Methodology and Discussion of Enhanced Queries
The Landscape was created through a series of iterative searches using the Innography software platform owned by Clarivate. The searches are done by using technical keywords gathered from materials and market literature. Keywords linked to the presence of AV L4 / L5 technology were searched in the patents’ abstract, title, and claims and then combined with technology feature terms found in the full patent specifications.
The search method consists of three parts: 1) search of the abstract, title, and claim sections of the patent document; 2) search in the “body” or specification of the patent document, and 3) a process to remove false positives. The first two queries (1 and 2 above) are connected with the Boolean operator “AND”, indicating that results must be found in both. Keywords searched in the body of a patent help to refine and add relevance to the search results.
While the full set of changes made for v2 of the Tech+IP search terms is set forth later in this document, we thought it would be helpful to highlight and comment on some of the enhancements made:
• A key concept of our query is the presence in the patent of the phrase “autonomous vehicle” or its synonyms (“self-driving vehicle”, “driverless vehicle”, etc.). The latest update required the extraction of “duplicate” search strings and instead of e.g. “self-driv*” or “self driv*” we amended this to a one search string concept – “self” NEAR/3 “driv*”
• Our v1 query methodology did not include a search of the “body” (or specification) of the patent (only the abstract, claims, and title sections). In this version, we added a search to the specification as well, but connected these searches with a Boolean “AND”.
• As readers will note, and based on feedback received, we also incorporated additional subject matter (that while not limited to use in autonomous vehicles) are clearly important for them. For example, “lidar” and “radar” were added in queries of the patent abstract, claims, and title. In addition, we beefed up the use of the “NEAR” query concept to incorporate greater flexibility in word choice. For example, we now query (“vehicle*” NEAR/5 ((“predict*” NEAR/3 “object*”))), also for example, “collision avoidance” (from the earlier query set) is replaced with “collision*” NEAR/3 “avoid*”. In total, these enhancements added another 40,000 patents to the results and identified patents such as US patent 11,001,256 assigned to Zoox, entitled “Prediction and Avoidance for Vehicles” and US 20210397185 A1, assigned to Uatc that contains in its abstract: “methods for predicting object motion and controlling autonomous vehicles are provided.”
• Some of the other new keyword concepts in the version 2 query include: “time of flight”, “environment map”, “vehicle fleet”, “collision prediction”, “collision detection”, “object detection”, etc. Green coloring of words in the chart below indicates that such keywords were added for v2. At the same time, the Tech+IP team did a substantial amount of work identifying cases and keywords that yielded a large number of false positives and choosing improved keyword terms to eliminate many false positives while not increasing false negatives. Such improvements are shown in the chart below as well
One of the ways in which feedback from readers of earlier versions of the Landscape was used was to improve queries against known datasets of AV patents identified by their owners. This has been extremely useful in “training” the query set.
Finally, in this Landscape report we have “expanded” EP granted patents according to their active nationalized states (giving a more accurate count of patents per jurisdiction). For example, EP patent 2,625,079 (assigned to Waymo) has been nationalized in Germany, France, and Great Britain. This patent is expanded per mentioned countries and counted for each.
|
Keyword Search V2 (Q1 2022) |
(@(abstract,claims,title) ((("autonom*" OR =automated OR ("self" NEAR/3 "driv*") OR "driverless" OR ("self" NEAR/3 "park*") OR =automotive OR “autopilot”) NEAR/5 ("vehicle*" OR =truck OR =van OR =car OR =bus OR “automobil*”)) OR ((("autonom*" OR =automated OR "self" OR =automotive OR “autopilot”) NEAR/5 (=drive OR =driving OR =driven OR =mode OR "control*")) AND ("vehicle*" OR =truck OR =van OR =car OR =bus OR “automobil*”)) OR (("vehicle*" NEAR/5 ((( "collision*" OR "blind spot*") NEAR/3 ("avoid*" OR "detect*"))OR =fleet OR "neural network"OR =lidar OR ("light detect*" NEAR/3 "rang*") OR ( "predict*" NEAR/3 ("object*" OR "vehicle*" OR "pedestrian*" OR "person*" OR "position*" OR "behavior")) OR ("detect*" NEAR/3 ("object*" OR "vehicle*" OR "pedestrian*" OR "person*" OR "position*" OR "behavior" OR “landing strip”)) OR (("determin*" OR "identif*" OR "recogn*" OR "predefin*" OR "detect*" ) NEAR/3 (( "position*" NEAR/3 "vehicle*") OR "traffic*" OR "destinat*" OR “speed*” OR “travel*” )) OR ("detect*" NEAR/3 ("driver*" OR "passang*")) OR ("environment*" NEAR/3 "map*")))))AND (@body (=lidar OR ("light detect*" NEAR/3 "rang*") OR "radar" OR ("vehicle*" NEAR/5 "fleet")OR ("time" NEAR/3 "flight")OR =ToF OR ( "environment*" NEAR/5 "map*") OR “environment* data” OR ("brak*" NEAR/5 ( "automat*" OR "autonom*" OR “control*”)) OR ("convolut*" NEAR/5 "network*") OR ("learn*" NEAR/5 ("machine*" OR "deep"))OR ("virtual" NEAR/5 "object*") OR("artificial" NEAR/5 "intelligenc*") OR ( "predict*" NEAR/5 ("object*" OR "vehicle*" OR "pedestrian*" OR "person*" OR "position*" OR "behavior")) OR (“image” NEAR/3 (“recogn*” OR “detect*”)) OR “autonomy map*” OR (( "determin*" OR "identif*" OR "recogn*" OR "predefin*" OR "control*" OR "detect*" ) NEAR/5 (("position*" NEAR/3 "vehicle*")OR"traffic*" OR "destinat*")) OR ("navigat*" NEAR/5 ("rout*" OR "path" OR "destinat*")) OR “vehicle* to* vehicle* communicat*” OR "vehicle* to* everything* communicat*" OR "vehicle* to* infrastructure* communicat*" OR =V2V OR OR =V2X OR =V2I OR "cooperativ* adaptive cruise control*" OR ("detect*" NEAR/5 ("driver*" OR "passang*")) OR ((“vehicle*” OR “plan*” OR "augment*")NEAR/3 "traject*")OR (“control*” NEAR/5 (“lane*” OR “traject*” OR “path”))OR ((“object*”OR "collision*" OR “blind spot*”)NEAR/3 ("avoid*" OR “detect*” OR “predict*”)))))) |
|
Keyword Search V1 (Q3 2021) |
@(abstract,claims,title) (("autonomous" OR "self driv*" OR "self-driv*" OR "driverless" OR "automated" OR "fully automat*" OR "fully-automat*" OR "highly automat*" OR "highly-automat*" OR "autopilot" OR "self-park*" OR "self park*" OR "automat* move" OR "automat* moving" OR "automat* guid*" OR "autonomous operation mode") NEAR/5 ("drive" OR "driving" OR =car OR "vehicle" OR "automobil*" OR "bus" OR "van" OR "truck")) AND ("machine learning" OR "artificial intelligence*" OR "neural network*" OR "convolutional neural network*" OR "deep learn*" OR ("predict*" NEAR/5 ("object*" OR "vehicle" OR "pedestrian*" OR "position" OR "behavior")) OR "object detect*" OR "collision avoidance" OR "avoid* blind spot*" OR "image recognition" OR "autonomy map*" OR "trajectory plan*" OR "detect* landing strip*" OR ("determin*" NEAR/5 ("position*" OR "speed*" OR "travel*" OR "object" OR "vehicle")) OR "brak* control*" OR "environment* data" OR "plan* navigation route" OR "vehicle-to-vehicle communication" OR "vehicle to vehicle communication" OR =V2V OR "cooperativ* adaptive cruise control*" OR "vehicle to everything communication*" OR "vehicle-to-everything communication*" OR =v2x OR "vehicle-to-infrastructure communication" OR =V2I OR "vehicle to infrastructure communication") |
|
Removing False Positives
|
((@claims (("industrial" OR "warehouse*") NEAR/5 "vehicle*") OR ("medical" NEAR/5 "machine*") OR (“cloth*” NEAR/3 (“machin*” OR “vehicle”)) OR “clean* robot*” OR "marine" OR “unmanned ship” OR "locomotive*" OR ("emerg*" NEAR/3 "device") OR =elevator OR (“container” NEAR/3 “vehicle*”) "drone*" OR "aircraft*" OR "airplane*" OR "airship*" OR "aerospace*" OR "tractor*" "spacecraft" OR "driverless transport* system*" OR ("unmann* NEAR/3 “deliver*") OR ("automat* hinge") OR (("agricult*" OR "armour*") NEAR/5 "vehicle*") OR (("baggage*" OR "aerial" OR "underwater" OR "gatoreye" OR “unmanned aerial” OR "stowage" OR =rail OR "haulage*" OR “fly*”) NEAR/5 (=drive OR =driving OR =driven OR “vehicle*” OR“robot*”)))) OR (( @title (("industrial" OR "warehouse*") NEAR/5 "vehicle*") OR ("medical" NEAR/5 "machine*") OR (“cloth*” NEAR/3 (“machin*” OR “vehicle”)) OR “clean* robot*” OR "marine" OR “unmanned ship” OR "locomotive*" OR ("emerg*" NEAR/3 "device") OR =elevator OR (“container” NEAR/3 “vehicle*”) "drone*" OR "aircraft*" OR "airplane*" OR "airship*" OR "aerospace*" OR "tractor*" "spacecraft" OR "driverless transport* system*" OR ("unmann* NEAR/3 “deliver*") OR ("automat* hinge") OR (("agricult*" OR "armour*") NEAR/5 "vehicle*") OR (("baggage*" OR "aerial" OR "underwater" OR "gatoreye" OR “unmanned aerial” OR "stowage" OR =rail OR "haulage*" OR “fly*”) NEAR/5 (=drive OR =driving OR =driven OR “vehicle*” OR “robot*”)))) |
|
Removing False Positives V1 (Q3 2021) |
(@claims (“industrial”) OR ("warehouse*") OR ("medical" NEAR/5 "machine*") OR ("marine") OR ("unmanned ship") OR ("cleaner robot") OR ("clothes machine") OR ("emergency report* device") OR (=elevator) OR ("patient*” NEAR/5 "hospital") OR ("automat* baggage*") OR (("drone" OR "fly*" OR "aircraft" OR "underwater " OR "tractor" OR "container" OR "unmanned delivery" OR "automat* hinge" OR "gatoreye" OR "driverless transport*" OR “spacecraft” OR "drone*" OR “unmanned aerial” OR “stowage” OR “robot*” OR “rail” OR “haulage*” ) NEAR/5 ("drive" OR "driving" OR =car OR "vehicle" OR “automobil*” OR “bus” OR “van” OR “truck”))) OR (@title (“industrial”) OR ("warehouse*") OR ("medical" NEAR/5 "machine*") OR ("marine") OR ("unmanned ship") OR ("cleaner robot") OR ("clothes") OR ("emergency report* device") OR (=elevator) OR ("patient*" NEAR/5 "hospital") OR ("automat* baggage*") OR (("drone" OR "fly*" OR "aircraft" OR "underwater " OR "tractor" OR "container" OR "unmanned delivery" OR "automat* hinge" OR "gatoreye" OR "driverless transport system*" OR “spacecraft” OR "drone*" OR “unmanned aerial” OR “stowage” OR “robot*” OR “rail” OR “haulage*”) NEAR/5 ("drive" OR "driving" OR =car OR "vehicle" OR “automobil*” OR “bus” OR “van” OR “truck”))) |
Landscape Results
Global Patent Activity
As of December 31, 2021, the AV L4 / L5 Landscapes consist of approximately 72,000 patents worldwide (comprising approximately 46,000 patent families). Note that only “active patents” are counted for purposes of the Landscape. Abandoned, expired, or invalidated patents are excluded.
Fig. 1. below shows the Landscape patents grouped per assigned company, specifically, the top 20 companies. Collectively, this represents approximately 46% of all the Landscape patents.
In terms of geography, the top 20 list comprises a diverse grouping – companies headquartered in the United States and Europe represent 25% of the total patents (12% and 13% respectively); China companies represent 3% of the patents (2 companies overall) and the largest concentration is from Asia Pacific Countries (excluding China) at 18% of the top group.
Fig. 2. below shows a straight geographical breakdown without regard to assignee status. Universities and research institutes comprise 6% of the total, and over 3/4ths of such patents are assigned to China institutions, while only 4% of US patents are assigned to universities or research institutes.
Priority Data Analysis
The date a patent application was first filed (its “priority date”) is important because it tends to reflect early R&D (which can, based on a host of circumstances, point to importance in the marketplace). Fig. 3. below presents the Landscape organized by the priority date of the patent families into three distinct periods: Pioneer, R&D, and Productization Gen 1 Phase. Reflecting the broad set of technologies applicable to the Landscape and the increased investment in R&D dollars being spent in L4 / L5 autonomous solutions, the Landscape shows expected roughly exponential growth. Over the next year or two, it is expected that the growth will slow to a degree – though the charts will continue to be updated by Tech+IP to reflect filings from 2020 and 2021 which may not yet be published (by law this occurs not later than 18 months from the priority date).
Pioneer Phase (2010-2014)
The Pioneer phase of AV L4 / L5 technology comprises approximately 14,000 patents held by 1,600 different assignees. As of this update, six companies hold in the aggregate almost one-third (30%) of the total patents with Pioneer Phase priority dates: Toyota leads with 6%, Bosch, Waymo, and Porsche with 5% each, Denso and Ford with 4%. Overall, Japanese and other Asia Pacific x-China companies comprise 24% of all Pioneer Phase patents, followed by Europe (18%), the US (12%), and China (2%). Somewhat surprisingly, universities and institutes hold only 3% of the identified Pioneer Phase patents. Unsurprisingly, automotive OEMs and pure-play autonomous companies (such as Waymo and Cruise) are well represented.
R&D Phase (2015 – 2019)
As is often the case in broad-based R&D technology segments, this second phase of patenting reflects a period of rapidly increased patent activity – almost 4 times as many patents carry 2015 – 2019 priority dates as do Pioneer Phase patents (approx. 50,000 patents in total). At the same time, the patent activity shows a large number of new entrants working in the L4 / L5 autonomous space. 45,000 unique companies hold patents in this period, and 90% of the patents in the R&D phase are owned by “new entrants” (i.e., they did not have patents assigned to them carrying Pioneer Phase priority dates).
These new entrants include pure-play AV companies such Motional Ad, TuSimple, Stradvision, Deepmap, Nio, Plus.ai as well as other types of entities. A number of companies dramatically accelerated their holdings (and therefore their overall position in terms of the total number of AV patents) during this time period. For example, Ford increased their patent activity by nearly 5 times in the R&D Phase compared to the Pioneer Phase, and Baidu filings moved it from the 1463rd position to the 8th. Early leaders such as Waymo and Denso saw their percentages decrease (Waymo, for example, from 3rd to the 10th position on total holdings). It is in this kind of scenario that the impact of priority dates can become more important than total filings. The top 20 companies per number of patents in this phase are presented in Fig. 5.
Productization Gen 1 Phase (2020 – Present)
This phase – counting patents with priority dates from 2020 to the present -- will be subject to substantial change over the next year, as many filings reach their publication dates. As it currently stands, approximately 7,000 patents are owned by 1,900 different assignees (1,400 operating companies, 300 universities/institutes, 200 different inventors). 59% of the top 20 companies have patents filed in China while filing in the US at the present time (and perhaps due to non-US inventorship and patent prosecution strategies) represents only 15%.
Backward Citation Analysis
Rather than just cataloging identified patents, Tech+IP also conducted a comprehensive analysis of patents cited by identified patents (so-called “backward citation” analysis). This analysis (when adjusted to eliminate self-citation) helps to cast light on what might be some of the more important patents in the field (at least insofar as “community recognition” is reflected in citations and matters).
Tech+IP’s backward citation methodology is presented in Fig.7. In a nutshell, the 72,000 patents comprising this Landscape cite approximately 182,000 unique patents as potential prior art, of which 9,000 such patents are active patents that relate to the AV L4 / L5 technology space. In total, these 9,000 patents are cited 48,179 times by other AV L4 / L5 patents.
Table 1. shows holders of patents that are most often cited by other patentees. As an example, Waymo owns 417 patents that are cited in total 8,862 times by AV L4 / L5 patents assigned to other companies. The column citation ratio normalizes the raw data by dividing the number of citations by the number of patents cited, helping to indicate recognized and potentially important work. The best citation ratio of the top 20 companies (by the number of citations made by others) belongs to the State University System of Florida (143.0) followed by Maplebear Company (49.4), Waymo (21.3), Allstate Corp (19.5), and Here Holding (18.3). Moreover, Verizon Communications (ranked 23rd by a number of citations made by others and not listed in Table 1) had a significant citation ratio of 22.0 (with 17 patents cited 374 times by others) which indicates the importance of their AV portfolio.
Additionally, we analyzed 50 of the most cited patents in the AV space. Confirming its status as a true pioneer in AV, Waymo holds 19 of 50 the most cited patents, followed by Ford (4), Honda (3), and Intellectual Ventures, Cruise, and Zoox with 2 patents each.
The Top 5 the most cited patents with technological categorization are presented in Table 2.:
Final Thoughts
The rapidly growing and multi-faceted nature of the L4 / L5 Autonomous market is clearly reflected in the patents being published globally describing and claiming technologies with important uses in the space. This Tech+IP Landscape represents our effort to bring together the engineering world and the patent world by identifying and classifying patents according to the definitions established by the Society of Automotive Engineers.
We recognize that the process of making such a landscape is complex and highly dependent on the details of the search process. To make the problem amenable to computerization and avoid too many false positives or false negatives we have attempted to create a baseline strategy that will yield “good enough” (not perfect) results and be capable of iterative improvement. In an attempt to make continuous improvements and bring something useful to the patent community we have opted to be fully transparent in your search process. We encourage others to review, comment and challenge these findings. It was that process by a number of practitioners in the community that drove improvements from v1 of the search query to v2 presented today.
We fully anticipate (and hope for) an AV Landscape version 3 that is better and more useful. In the end, Tech+IP is doing this work because we believe that when information is more available and assumptions are transparent more deals get done, thereby improving the ecosystem for all participants – and that is the core business of Tech+IP Advisory.