The Supreme Court in Jerusalem has rejected a request by 21 people to be registered as Israeli nationals rather than Jews or Arabs.
Israel's population register does not allow Israeli nationality; petitioners say this is undemocratic and exposes minorities to discrimination.
Most of the activists are described in the population registry as Jewish.
In its 26-page ruling, the court explained it did not have the authority to determine ethnicity or statehood.
In an editorial, the Haaretz newspaper described the move as a "defeat for Israeliness."
"The state and its legislators must relate to all its citizens as Israeli citizens and eliminate the nationality line item in the Population Registry," the paper said.
The court said that any recognition of Israeli nationality would have "weighty implications'' on the state of Israel and endanger Israel's founding principle: To be a Jewish state for the Jewish people, the Associated Press news agency said.
The main petitioner was Uzzi Ornan, 90, a linguistics professor at the Technion-Israel Institute of Technology in Haifa. He runs a small group which campaigns for the recognition of Israeli nationality.
"With an Israeli identity, we can be secure in our democracy, secure in equality between all citizens,'' Mr Ornan said.
Jewish religious holidays are also national holidays in Israel.
About 50% of Israel's Jewish population define themselves first and foremost as Jewish, but 41% of Israelis identify themselves as Israeli, according to figures obtained from the Israel Democracy Institute, a think-tank.
Anita Shapira, a professor emeritus of Jewish history at Tel Aviv University, said she feared that if nationalism developed into an Israeli one, the Jewish essence would be lost.
"The attempt to claim that there is a Jewish nationality in the state of Israel that is separate from the Jewish religion is something very revolutionary," she said.