Nitrogen and phosphorus loading of rural areas in Japan has recently exceeded environmental tolerance levels, thereby causing deterioration of aquatic environments. Rural areas need suitable low-energy, low-cost purification technologies. Vegetation-based water treatment uses the purification functions inherent to natural ecosystems and is therefore likely to be adaptable to rural areas. Here, we review the water purification processes associated with vegetation-based water treatment systems. We then describe three purification technologies that are being developed for practical water pollution treatment in rural districts. The first is a free-water-surface-flow constructed wetland planted with indigenous wild rice (Zizania latifolia) in the Koibuchi College biotope zone, which receives secondary-treated wastewater from the college dormitories. The second is a hybrid-type subsurface-flow constructed wetland system for purifying milking-parlor wastewater in cold regions. The third is a biogeofilter channel in which terrestrial plants (including most crops) can be grown by adjusting the filter material height to suit the plants’ moisture tolerance. We discuss the advantages and disadvantages of the three treatment technologies in terms of their introduction at various sites, and we consider yudone, a traditional vegetation-based purification system once used by Kyushu-region farmers.