Understanding the link between community diversity and ecosystem function is a fundamental aspect of ecology. Systematic losses in biodiversity are widely acknowledged but the impact this may exert on ecosystem functioning remains ambiguous. There is growing evidence of a positive relationship between species richness and ecosystem productivity for terrestrial macro-organisms, but similar links for marine micro-organisms, which help drive global climate, are unclear. Community manipulation experiments show both positive and negative relationships for microbes. These previous studies rely, however, on artificial communities and any links between the full diversity of active bacterial communities in the environment, their phylogenetic relatedness and ecosystem function remain hitherto unexplored. Here, we test the hypothesis that productivity is associated with diversity in the metabolically active fraction of microbial communities. We show in natural assemblages of active bacteria that communities containing more distantly related members were associated with higher bacterial production. The positive phylogenetic diversity–productivity relationship was independent of community diversity calculated as the Shannon index. From our long-term (7-year) survey of surface marine bacterial communities, we also found that similarly, productive communities had greater phylogenetic similarity to each other, further suggesting that the traits of active bacteria are an important predictor of ecosystem productivity. Our findings demonstrate that the evolutionary history of the active fraction of a microbial community is critical for understanding their role in ecosystem functioning.