Frank Hsieh, a former head of Taiwan's opposition Democratic Progressive Party (DPP), is on a private five-day visit to China.
Mr Hsieh, the party's 2008 presidential candidate, is the highest-level opposition politician to visit China.
His trip is viewed as an icebreaker for both sides, although he is not scheduled to meet top officials.
His party's losses in recent races have been blamed partly on the lack of a clear strategy on engaging Beijing.
Mr Hsieh's trip has been met with mixed reactions within his own party and has not been mentioned in Chinese state media.
Reports said the lawmaker was greeted warmly by local officials when he arrived in the coastal city of Xiamen in Fujian province.
He will visit Dongshan island, the home of his ancestors, and the Olympic stadium in Beijing.
Mr Hsieh will meet mostly academics in what has been repeatedly characterised as a personal trip - similar to how previous trips by lower-level politicians have been cast.
"The purpose of the trip is to build mutual trust," he told reporters prior to leaving Taiwan.
While some Taiwanese believe increasing understanding between the two sides could only be good, others fear it could eventually lead to a change in DPP policy and the end of Taiwan's independence movement, says the BBC's Cindy Sui in Taipei.
The party has been locked in debate over whether to revise its stance towards relations with China.
"The DPP should face the reality of China's rise... if the DPP keeps refusing to change its position and let the Kuomintang and the Chinese communist parties work hand in hand, I'm afraid the DPP will never be able to get back into power," Mr Hsieh said before he left.
In January incumbent President Ma Ying-jeou won a second term in office, campaigning aggressively on his record of improving ties with China.
While some analysts believe Mr Hsieh's trip is a positive step towards influencing the DPP's policy on China, others say it will only serve to add to tension in the opposition ranks.
The DPP has historically supported Taiwan's independence and advocated a Taiwanese identity that is separate from mainland China.
It has also rejected the "one China" principle as the basis for relations between the two sides, our correspondent adds.
It remains to be seen how the two sides will get around the issue of Taiwan's sovereignty, the main obstacle to building relations and source of hostility between the two sides in the past, she adds.