ML technology has enabled a revolutionary leap in many digital economies generating growth in activity and business mainly for the ITC sector. Part of the growth is generated through sharing of IP, knowledge, tools and datasets. We want to adopt this approach for the digital construction sector. ConTag will provide visual and 3D training dataset and pre-trained deep neural networks (DNN) to benchmark against. We expect this shared and open datasets to kick-start further ML developments in both academia and industry. It is intended as a seed point for collaborative research. The application area we are targeting is visual tagging of assets in the construction sector from digital imagery. For ML and visual tagging to be useful in the construction sector categories need to much more specific and fit for purpose in the real world construction scenario.