Google's bids for a pool of wireless patents were based on mathematical constants, say sources.
The portfolio of 6,000 patents was auctioned to realise some value from the assets of bankrupt telecoms firm Nortel.
During the sale, Google's bids were based on pi, other constants and the distance between the Earth and the Sun.
Google lost the auction as a consortium including Apple and Microsoft made the winning bid of $4.5bn (£2.8bn).
"Google was bidding with numbers that were not even numbers," a source involved in the auction told the Reuters news agency.
The sale of the patent portfolio started as a five-way scrap between two separate consortia and individual firms including Google and Intel.
Initial estimates suggested the portfolio would attract around $2bn (£1.24bn) but the four days of intense bidding saw the total rise sharply.
During its bids, Google picked numbers including Brun's constant and Meissel-Mertens constant that were said to have "puzzled" others involved in the auction. When bids from rivals hit $3bn, Google reportedly bid pi, $3.14159bn, to up the ante.
"Either they were supremely confident or they were bored," Reuters' source said.
It is not clear what inspired Google to draw on obscure mathematics for its bids. However, Google co-founder Sergey Brin is widely acknowledged to be a maths prodigy and the bids may reveal his influence.
As the bids got bigger some firms dropped out and others became partners to pool their resources. From going it alone, Apple joined a consortium that included Microsoft, Research In Motion and Sony.
"It did become clear to us very quickly that this was something that a bunch of big companies with humongous balance sheets had decided was strategic for them," said John Amster, Chief Executive of RPX that led one consortium. The RPX-led group dropped out as the price climbed.
Ultimately the portfolio was being fought over by two groups: Google and Intel on one side and the Microsoft/Apple-led consortium on the other.
Google's failure to secure control of the patents could cost it dear in the future, warned intellectual property analyst Florian Mueller.
"I would have thought they would seize this once in a lifetime opportunity to become a new wireless patent player," Mr Mueller told the BBC. "It's not going to have, any time soon, a comparable opportunity to acquire such a diversity of relevant patents in a single purchase."
Currently Google had about 700 patents in its mobile portfolio, he said, many of which relate to using handsets to serve its core competences such as search.
By contrast, he said, the Nortel patents relate to future technologies that will make mobile networks faster and handsets more powerful.
Controlling that, he pointed out, would be very useful as a bargaining chip with rivals. Owning the patents could also ease the burden on firms making Android devices as they would have fewer licence fees to pay.
Use of Android technology from Google is free provided handset makers pipe traffic back to the search giant so it can make money with adverts.
However, the numbers of companies asking for cash to use the non-Google developed technologies found in Android phones was rising, he said.
For instance, Microsoft has announced licensing deals with many Android phone makers including General Dynamics and HTC.
With the control of the patents passing to a consortium that includes firms that are Google's bitter rivals in the mobile phone world, licence fees could increase.
"It's reducing the claim that Android is free to an absurdity," said Mr Mueller.
Google has not issued a formal statement on the auction outcome but has reportedly called it "disappointing".