A suicide attacker detonated a vehicle and a fierce gun battle followed. Officials say the attack is now over and the area has been cleared.
A Taliban spokesman said the group carried out the attack.
It comes a week after it said it was launching its "spring offensive", warning of large-scale attacks.
Why are the Taliban resurgent in Afghanistan?
Tuesday's bombing happened during the morning rush hour in Pul-e-Mahmud, a busy neighbourhood where homes, mosques, schools and businesses nestle close to the Ministry of Defence, other ministries and military compounds.
Soldiers and security officers are reported to be among the casualties, but the majority are civilians.
The blast shattered windows up to 1.6km (one mile) away. The Associated Press quoted a police officer as saying it was one of the most powerful explosions he had ever heard, and he could not see or hear anything for 20 minutes after.
Image copyright Reuters
It appears the initial blast cleared the way for Taliban fighters to enter the area - a commonly used tactic.
"One of the suicide attackers blew up an explosives-laden truck in a public parking lot next to a government building," Kabul police chief Abdul Rahman Rahimi told AFP news agency.
"The second attacker engaged security forces in a gun battle before being gunned down," he added.
Hours after the assault was declared over, another blast rocked Kabul. Police told a BBC reporter in Kabul it was caused by an improvised explosive device (IED).
"It was as huge as in the morning," said Afghan MP Elay Ershad. "So everyone [is] scared of this situation."
Casualties are high because the assault happened in a densely packed neighbourhood during rush hour.
Usually the Taliban does not say it is behind such attacks, which cause large numbers of civilian deaths. But today they did because the target was high-profile and it appears that, for them, hitting the target was worth the civilians killed and wounded.
What we hear less of are similar planned attacks that are foiled by the Afghan security forces, and there are plenty. But preventing such attacks altogether is next to impossible.
The country has been at war for three decades, explosives are easily available and bomb-making skills are common.
The Taliban's "spring offensive" is part of a propaganda war. The group fights all year round and so do the Afghan security forces. It is the story of the past 37 years.
In a statement on its Pashto-language website, the Taliban said several militants successfully breached the defences of the National Directorate of Security, Afghanistan's main spy agency which protects high-ranking government officials
It is unclear precisely how many attackers were involved and the Taliban have been known to exaggerate details.
The presidential palace - only a few hundred metres from the blast - condemned the attack "in the strongest possible terms".
"Such cowardly terrorist attacks will not weaken the will and determination of Afghan security forces to fight against terrorism."
A tweet from President Ghani's office's official account suggested the attack "clearly shows the enemy's defeat in face-to-face battle with ANSDF" - the Afghan National Security and Defense Forces.
But correspondents say that in fact the Taliban has gained in strength since the bulk of British, American and other Nato forces left in 2014.
It has also been buoyed by a flood of foreign fighters joining its ranks, and now controls sizeable parts of Afghanistan.
Taliban spokesman At least 28 people have been killed and 329 injured in a huge explosion in the centre of the Afghan capital Kabul, police and officials say.
