Interpretation of fractures in raw outcrop maps is a tedious and time-consuming task. A few semi-automatic or automatic interpretation methods based on image processing are available; however, they are usually sensitive to the contrast of the image that, in turn, causes under or over-interpretation of fracture geometry. A successful interpretation of fractures from a raw outcrop image requires two stages: (1) conversion of a multi-bit per pixel raw outcrop image to a binary map that preserves fracture geometry and connectivity, and (2) replacement of the binary fracture images with line segments or polylines. These two stages are fracture recognition and fracture detection, respectively. We apply the U-net architecture to recognize fractures in a raw outcrop map. When 200 training epochs are applied to our images, the training accuracy reaches 0.94, while the mean square error decreases to 0.02. The implementation of U-net yields good results for fracture recognition. We propose a pixel-based fracture detection algorithm. The algorithm can automatically interpret the fractures in the recognized binary map as line segments or polylines. By combining fracture recognition and detection, we can interpret automatically fractures in a complex raw outcrop map.