Semantic Bird’s-Eye-View Map Prediction Using Horizontally-Aware Pyramid Occupancy Network

Abstract

Deep neural network has been used to predict the bird's-eye-view map from a frontal camera of an autonomous car. A state-of-the-art approach, namely pyramid occupancy network (PON), uses an encoder-decoder architecture to condense an image column into a context vector that describes the object occupancy along the radial direction. Our work, Horizontally-aware Pyramid Occupancy Network (H-PON), extends the PON model with a novel component that provides additional context information describing the relationships of the objects along the horizontal direction. This is done by also encoding the horizontal column of the image into an additional context vector using another encoder-decoder layer. This context vector is, then, expanded back providing improved features for semantic reasoning across the horizontal direction. We found that this simple extension significantly improves PON's semantic prediction performance in the nuScences dataset. Our experiment shows that the objects that are rarely seen and those that are further away from the center greatly benefit from this novel component.