Putting knowledge back where neural networks can use it

Neural networks are becoming an ever more important tool for artificial intelligence applications. Mainstream use of neural networks now allow for one to simply plug raw data into black box networks to obtain solutions to a number of complex problems. Unfortunately, going directly from raw data to prediction discards a wealth of knowledge we have acquired on the problems at hand. Adding "human reasoning" in the deep learning framework is however a difficult process.
We therefore suggest simply adding knowledge-based inputs (e.g. maps) at key points in neural networks and providing guidance to help neural networks make use of this knowledge input. This approach puts the focus back squarely on finding useful knowledge for the neural networks, and circumvents the difficulties of constraining reasoning in neural networks.
To illustrate our point, we will discuss the use of HDMaps in Autonomous Driving tasks. HDmaps are a crucial input in autonomous driving to the point that most modern trajectory forecasting models are designed to make use of this additional knowledge input, and dedicated models have emerged to estimate the surrounding map from camera data. In particular, we will explain how properly leveraging imperfect existing maps as a neural network input is crucial to obtaining good estimates of the current surrounding map and considerably simplifies that problem.