The Scottish pair lost a champions tie-break to go down 6-4 3-6 10-2 to top seeds Leander Paes and Radek Stepanek at the Ariake Colosseum.
Andy Murray earlier set up to a singles semi-final clash with Milos Raonic.
The Murrays had come from a set down to beat Eric Butorac and Jarkko Nieminen 5-7 6-3 10-4 in the previous round.
Paes and Stepanek play with Danielle Bracciali and Frantisek Cermak in the semi-finals after defeating Juan Monaco and local favourite Kei Nishikori.
In the other half of the draw, Tomas Berdych and Nenad Zimonjic will face Jeremy Chardy and Marcelo Melo or Alexander Peya and Bruno Soares.