The clash took place during a special operation against suspected militants in a village just outside Kazakhstan`s largest city and former capital of Almaty, where prosecutors said they had been planning attacks.
The spokesman for general prosecutors Nurdaulet Suindikov said the militants hid in a house in the village of Boraldai but then they were surrounded by the security forces as residents were evacuated for their own safety.
"Two members of the special Arystan battalion of the national security committee were killed," he said in the capital Astana.
"After refusing to give themselves up and providing armed resistance, five members of the terrorist group were killed, including their leader."
He said that the group had been behind the murder of two police officers on November 8 and were planning new "violent acts" in Almaty. There were no civilian casualties in the clash.
Suindikov named the dead leader of the group as Khasen Agzhan, who was born in 1977 and was wanted by the security forces on charges of "creating and leading a terrorist group."
He said that grenades, two Kalashnikov machine guns and a Makarov pistol had been found in their safe house.
The spokesman said that the security forces in Almaty were on high alert but said there was no reason for panic.
"The security forces for the Almaty city and region have moved onto a strengthened service regime but the situation is under full control.
"There are no threats to the population or cause for worry," he added.
The clash came three weeks after seven people were killed in the southern Kazakh city of Taraz when a suspected Islamist went on a shooting rampage and then blew himself up.
An Islamist group called Jund al-Khilafah (Soldiers of the Caliphate) had claimed previous more minor unrest and Kazakh prosecutors said in November its members wanted to "unleash jihad on the territory of Kazakhstan".
Such unrest has until recently been highly unusual in majority Muslim but secular Kazakhstan, which under strongman leader Nursultan Nazarbayev has earned a reputation as by far the most stable country in Central Asia.
Around 70 percent of Kazakhstan`s 16.5 million people are Muslims and Kazakh authorities have repeatedly expressed concern about Islamic extremism sweeping in from other Central Asian states and Afghanistan.
Lauded by his supporters as a Central Asian equivalent of great national leaders like Turkey`s Mustafa Kemal Ataturk or India`s Mahatma Gandhi, Nazarbayev, 71, has dominated Kazakhstan for over two decades.
He has long boasted of a proud record of maintaining harmony in the multi-ethnic state and has attracted considerable investment from foreign companies impressed by the nation`s stability.