There are no published patent applications indicating Luminar has a NN in silicon.
Their latest published application:
US2022309685A1 NEURAL NETWORK FOR OBJECT DETECTION AND TRACKING
claims a method of tracking multiple objects using a processor, ie, it is software running on a processor. Their processing hardware includes a segmentation module, a classification module, and a tracking module. each of which can include a NN:
View attachment 39891
[0076] As seen in FIG. 3, the vehicle includes N different sensors 102 , with N being any suitable integer (e.g., 1, 2, 3, 5, 10, 20, etc.). At least “Sensor 1” of the sensors 102 is configured to sense the environment of the autonomous vehicle by physically interacting with the environment in some way, such as transmitting and receiving lasers that reflect off of objects in the environment (e.g., if the sensor is a lidar device), transmitting and receiving acoustic signals that reflect off of objects in the environment (e.g., if the sensor is a radio detection and ranging (radar) device), simply receiving light waves generated or reflected from different areas of the environment (e.g., if the sensor is a camera), and so on. Depending on the embodiment, all of the sensors 102 may be configured to sense portions of the environment, or one or more of the sensors 102 may not physically interact with the external environment (e.g., if one of the sensors 102 is an inertial measurement unit (IMU)). The sensors 102 may all be of the same type, or may include a number of different sensor types (e.g., multiple lidar devices with different viewing perspectives, and/or a combination of lidar, camera, radar, and thermal imaging devices, etc.).
[0077] The data generated by the sensors 102 is input to a perception component 104 of the sensor control architecture 100 , and is processed by the perception component 104 to generate perception signals 106 descriptive of a current state of the vehicle's environment. It is understood that the term “current” may actually refer to a very short time prior to the generation of any given perception signals 106 , e.g., due to the short processing delay introduced by the perception component 104 and other factors. To generate the perception signals 106 , the perception component 104 may include a segmentation module 110 , a classification module 112 and a tracking module 114 .
[0078] The segmentation module 110 is generally configured to identify distinct objects within the environment, as represented by the sensor data (or a portion of the sensor data). Depending on the embodiment and/or scenario, the segmentation task may be performed separately for each of a number of different types of sensor data (e.g., the segmentation module 110 may include a number of modules operating in parallel), or may be performed jointly on a fusion of multiple types of sensor data. In some embodiments where lidar devices are used, the segmentation module 110 analyzes point cloud frames to identify subsets of points within each frame that correspond to probable physical objects in the environment. In other embodiments, the segmentation module 110 jointly analyzes lidar point cloud frames in conjunction with camera (and/or other) image frames to identify objects in the environment. Examples of lidar devices/systems and point clouds are discussed in further detail below. Other suitable techniques, and/or data from other suitable sensor types, may also be used to identify objects. As used herein, references to different or distinct “objects” may encompass physical things that are entirely disconnected (e.g., with two vehicles being two different “objects”), as well as physical things that are connected or partially connected (e.g., with a vehicle being a first “object” and the vehicle's hitched trailer being a second “object”).
[0079] The segmentation module 110 may use predetermined rules or algorithms to identify objects. For example, the segmentation module 110 may identify as distinct objects, within a point cloud, any clusters of points that meet certain criteria (e.g., having no more than a certain maximum distance between all points in the cluster, etc.). Alternatively, the segmentation module 110 may utilize a neural network that has been trained to identify distinct objects within the environment (e.g., using supervised learning with manually generated labels for different objects within test data point clouds, etc.), or another suitable type of machine learning based model. Example operation of the segmentation module 110 is discussed in more detail below in FIG. 5B, for an embodiment in which the perception component 104 processes point cloud data.
[0080] The classification module 112 is generally configured to determine classes (labels, categories, etc.) for different objects that have been identified by the segmentation module 110 . Like the segmentation module 110 , the classification module 112 may perform classification separately for different sets of the sensor data (e.g., the classification module 112 may include a number of modules operating in parallel), or may classify objects based on a fusion of data from multiple sensors, etc. Moreover, and also similar to the segmentation module 110 , the classification module 112 may execute predetermined rules or algorithms to classify objects, use a neural network that has been trained to classify identified objects within the environment (e.g., using supervised learning with manually generated labels for different point cloud representations of distinct objects, etc.), or use another suitable machine learning based model to classify objects. Example operation of the classification module 112 is discussed in more detail below in FIG. 5B, for an embodiment in which the perception component 104 processes point cloud data.
[0081] The tracking module 114 is generally configured to track distinct objects over time (e.g., across multiple lidar point cloud or camera image frames). The tracked objects are generally objects that have been identified by the segmentation module 110 , but may or may not be objects that were classified by the classification module 112 , depending on the embodiment and/or scenario. The segmentation module 110 may assign identifiers to identified objects, and the tracking module 114 may associate existing identifiers with specific objects where appropriate (e.g., for lidar data, by associating the same identifier with different clusters of points, at different locations, in successive point cloud frames). Like the segmentation module 110 and the classification module 112 , the tracking module 114 may perform separate object tracking based on different sets of the sensor data (e.g., the tracking module 114 may include a number of modules operating in parallel), or may track objects based on a fusion of data from multiple sensors. Moreover, and also similar to the segmentation module 110 and the classification module 112 , the tracking module 114 may execute predetermined rules or algorithms to track objects, may use a neural network that has been trained to track identified (and possibly classified) objects within the environment (e.g., using supervised learning with manually generated labels for different pairs or sets of point cloud frames, etc.), or another suitable machine learning model to track objects.
[0082] Because the blocks of FIG. 3 (and various other figures described herein) depict a software architecture rather than physical components, it is understood that, when any reference is made herein to a particular neural network or other software architecture component being “trained,” or to the role of any software architecture component (e.g., sensors 102 ) in conducting such training, the operations or procedures described may have occurred on a different computing system (e.g., using specialized development software). Thus, for example, neural networks of the segmentation module 110 , classification module 112 and/or tracking module 114 may have been trained on a different computer system before being implemented within any vehicle. Put differently, the components of the sensor control architecture 100 may be included in a “final” product within a particular vehicle, without that vehicle or its physical components (sensors 102 , etc.) necessarily having been used for any training processes.
I think it is improbable that Mercedes will revert to a software NN.
Now there are some interesting features of Luminar's laser projector for lidar, such as foveation which enables the laser pulses to be concentrated more densely on objects of interest, and this could well attract MB, but I doubt they would adopt software to process the reflected pulses.
Does Luminar even have a chip?
On or around March 17, 2023, various media outlets reported that Lidwave, an Israeli start-up, had accused Luminar of attempting to pass off a Lidwave chip as Luminar’s own technology after Luminar displayed an image of the processor at a recent investor conference, as well as in materials on its website. As a result, Lidwave threatened Luminar with legal action. Luminar subsequently removed the images in question from its investor presentation, website, and a YouTube video. On this news, Luminar’s stock price fell $0.68 per share, or 8%, to close at $7.80 per share on March 20, 2023, the next trading day.
Luminar Technologies, Inc.: Company Investigated by the Portnoy Law Firm
Investors can contact the law firm at no cost to learn more about recovering their losses LOS ANGELES, July 11, 2023 (GLOBE NEWSWIRE) -- The Portnoy Law...