There are at least 23,000 private properties sitting empty in Wales, with some councils bringing less than 1% back into use.
In 2012-13 there were at least 19,612 empty homes, rising to 23,303 in 2016-17. A charity called it "a blight".
Of these, 962 (4.9%) were brought back into use in 2012-13, rising to 1,347 (5.8%) in the last financial year.
The Welsh Government said it was working with councils to bring homes back into use.
There were an estimated 1.4 million dwellings in Wales as of 31 March 2016, according the Welsh Government.
Housing charity Shelter Cymru said: "Empty homes are a blight on the community and a total waste of resource. They attract vandalism and vermin."
There is also a huge disparity between councils about the percentage of homes put back into use.
While Torfaen brought a third of its empty properties back into use in 2016-17, 108 out of 325, Cardiff only turned around 0.8% - 10 out of 1,318.
In a bid to tackle the issue, local authorities have powers to charge premiums of up to 100% on top of council tax on long-term empty homes and second homes.
Wrexham, Flintshire and Powys charge an extra 50%, Anglesey and Ceredigion charge 25% and Denbighshire is considering its own plans.
A Shelter Cymru spokeswoman said: "We welcome the fact that some local authorities have increased the council tax for empty homes but we're also interested in exploring the role of social landlords in refurbishing and managing empty homes in Wales."
Since 2006, authorities have had the power to seize, renovate and lease any property empty for longer than six months under Empty Dwelling Management Orders (EDMO).
These have been used just three times in Wales in the past 11 years and are seen as a last resort.
The Welsh Local Government Association said: "Local authorities have a wide range of tools and powers which can be used to encourage the owners of empty residential properties to bring them back into use.
"These include 'soft' powers of influence and encouragement as well as grants, loans and enforcement powers."
A spokesman said: "These initiatives, operated by local authorities and together amounting to over £50m, provide home owners with the finance required to renovate the properties for rent or sale."