The government says up to 300,000 families in England could benefit from plans to scrap council tax for so-called granny flats.
Some exemptions already exist for over 65s living in annexes but Communities Secretary Eric Pickles wants to broaden those out to cover any family member.
He said the move could ease the pressure on the supply of homes.
But Labour said the move appeared to be a government attempt to "deflect attention from their housing crisis".
Currently, annexes are usually treated as distinct dwellings - and charged council tax - if they have a separate kitchen, bathroom and place to sleep. They do not need to have their own front door.
Local authorities typically charge more than £1,000 a year for full rates of council tax.
But under changes introduced in 1997, an annexe is exempt if the occupier is over 65, "mentally impaired", or "substantially and permanently disabled".
Mr Pickles told the Daily Telegraph: "We are keen to remove tax and other regulatory obstacles to families having a live-in annexe for immediate relations.
"It seems fundamentally unfair that hardworking families who want to extend their homes to allow their relatives to live in are hit twice by the taxman."
Ministers are also considering changing planning laws to make it easier to convert garages and other outbuildings into living spaces for relatives.
Officials are concerned that at present too many conversions are refused planning approval by local councils.
The changes are likely to require legislation, but government sources were unable to say how quickly they could be brought in.
BBC political correspondent Gary O'Donoghue said the "substantial loss" in revenue for local authorities was a "thorny issue" and Whitehall sources would not commit central government to making up the difference.
But he added: "This is likely to be a popular idea, aimed at demonstrating the government's claim that it's on the side of hard-working families."
For Labour, shadow communities secretary Hilary Benn said it was unclear who would benefit from the moves.
"This seems to be nothing more than an attempt to deflect attention from their housing crisis.
"What we need is to get building and get the economy moving again. That's why Labour is proposing to build 25,000 new affordable homes and a temporary cut to the rate of VAT. "
Earlier this month, MPs said it should be easier for people to build their own homes in England to help the housing crisis.
A Commons communities select committee report suggested local authorities must get a greater say on housing projects, while pension funds and other investment funds should be encouraged to invest in building new homes.
The report said 230,000 households were forming each year, but in 2011 only 110,000 new homes were built.