In this paper, a fast and accurate multi-task approach for lane line detection was proposed. Specifically, road lanes are detected by a grid-based convolutional neural network instead of classifying lane lines directly on every pixel. The input images are split into grids, or sets of row-wise anchors for classification, leading to a significant drop in computational cost that is very valuable for embedded systems as autonomous cars. Despite the effectiveness of lane detection, the proposed algorithm still suffers from hurdles on the street such as cars or people. To this end, we proposed an additional segmentation-based scene parsing module to work in parallel with the lane line algorithm. The scene parsing acts as external guidance on where to make lane predictions for the lane line algorithm. The fusion of two methods was evaluated on a small driving scene dataset collected from a prototype autonomous car. The collected experimental results demonstrated the superiority of the proposed method compared to multi-thread approaches, in terms of both lane detection accuracy and computational cost. The proposed method was applied to a prototype golf-based autonomous car, achieving real-time accurate navigation results on campus road lanes.