Thesis project: Pallet perception through deep learning and sensor fusion

Develop methods using deep learning models to improve the current pallet perception system, with a focus on fusing model outputs with data from other sensors. The main focus will be on general pallet identification and localization. In your Master Thesis at Toyota, you will work on: A modern sensor system featuring: 2D RGB camera Time-of-Flight (ToF) 3D camera Correlation between pixels from the RGB camera and points in the 3D point cloud from the ToF camera NVIDIA chip in an embedded environment (GPU) for data processing Tasks include: Training and evaluation of deep learning (DL) models using different methodologies Algorithm development for data fusion Evaluation using real-world data and comparison with training results from a synthetic environment At Toyota, we work with innovative technologies to develop our autonomous trucks. The thesis project will investigate training methodologies, model architecture, and the selection of suitable model outputs that can be effectively combined with point cloud data to accurately localize a pallet in relation to the camera position. The training data will be generated in synthetic environments, which will be provided as part of the project. Therefore, the gap between synthetic and real-world data will also need to be considered when evaluating the results. The thesis project will be carried out in close collaboration with one of Toyota's teams specializing in machine learning development. SCOPE Master thesis 30 hp, 1-2 students Requirements For this master thesis, we are looking for students with following education or equivalent: 3D Computer vision Deep learning specialization / Machine Learning specialization Who is Toyota Material Handling? Toyota Material Handling is a global leader in material handling, and we are making significant investments to meet the needs of the future. At our site in Mjölby, 3,000 employees work across the entire material handling value chain, from development concepts to finished vehi