The country as we know it today was actually formed quite recently, when south Yemen and north Yemen - which believed the country should be run in different ways - joined together in 1990, after many years of fighting.
However, since then the fighting has not stopped, which has made life for those living in the country very difficult.
Find out more about why there is still a conflict in Yemen with Newsround's guide.
What has happened since Yemen was formed?
Despite joining together in 1990, the north and south of the country still disagreed with each other.
Fighting between the government and anti-government fighters called the Houthi (also known as 'rebels') continued.
The situation reached a peak in 2011 when protests led to the president at the time, Ali Abdallah Saleh, resigning and his deputy, Abdrabbuh Mansour Hadi, taking over.
However, the fighting still didn't stop, as Mr Hadi struggled to keep everyone in the country happy.
In 2014, the country descended into civil war, as the rebels took over the capital city of Yemen called Sanaa.
This civil war is still going on today.
What is the situation at the moment?
When the Houthi took over Sanaa, President Hadi fled to the country next door, called Saudi Arabia, which still supports him.
He set up a temporary capital in a different city in Yemen called Aden. He has still not been able to return to Sanaa.
At the moment, the country is locked in a difficult situation.
The Houthi rebels don't accept that Hadi's government makes their rules. They believe in a new ruling group, with a leader called Mohammed Ali al-Houthi.
However, much of the rest of the world - including the UK and the US - do not accept this. They both support the government in Saudi Arabia.
Earlier in 2016, the United Nations - which is an organisation which works to bring peace - helped to make the two sides talk to each other to try to sort out their problems.
But after three months, the talking stopped and the fighting continued.
Another thing making the situation in Yemen more difficult is that it has become a place for extreme groups, like Al-Qaeda and the group that calls itself Islamic State, to base themselves and grow stronger.
It has also always been one of the poorest Arab countries, which has made it difficult to deal with the situation.
What's being done to help people living there?
The fighting has had a devastating impact on normal people trying to live their lives in Yemen.
Charities are working to try to help them.
A UK organisation called the Disasters Emergency Committee (DEC), which represents 13 aid charities, launched a big appeal asking for people to donate money to help make sure that people in Yemen have what they need to live.
At the end of 2016, the British Red Cross, a charity represented by the DEC, said: "The ongoing conflict in Yemen has devastated millions of people's lives."
"More than half the population don't have enough food. Almost a quarter face starvation. Families are living with no water or electricity."
Restricted deliveries of food and fuel, and roads and buildings being destroyed, has led to as many as 21 million people not having the basics essentials they need to live.
With airports closed and borders blocked, many people are unable to leave the country, despite the problems.
Thousands of people have also been killed and injured in the fighting.