The bomber entered the uncompleted Redeemed Christian Church of God at 9:55 am in the Jigawa area on the outskirts of the city in Yobe state and detonated his explosives, killing the worshippers, including a woman and her two children as well as the pastor, they said.
"Four worshippers died instantly while the fifth victim gave up shortly after she was taken to hospital," a police officer involved in the evacuation told AFP.
"The victims included a woman and her two children, the pastor and another worshipper," added the officer who spoke on condition of anonymity.
Witness Garba Manu told AFP the bomber arrived at the church on a motorised rickshaw disguised as a worshipper and detonated his explosives as soon as he entered the church.
"I saw when the bomber alighting from a motorised rickshaw and entering the church and as soon as he entered a loud explosion ripped through the church which is under construction," he said.
"I saw him walked into the church and he didn't raise any suspicion. Five worshippers, including three females and two males were in the church when the bomber entered and they were all killed," he added.
"The dismembered body of the bomber was also recovered," said Manu, who lives in the area.
There was no immediate claim of responsibility but Potiskum, the commercial capital of Yobe state, has been hit repeatedly by Boko Haram Islamists in their six-year insurgency, including by suicide bombers.
The insurgents have also launched a series of attacks on mosques, villages and markets in neighbouring Borno state in the past few days, claiming more than 200 lives.
Boko Haram are keen to prove they are not a spent force despite a four-nation military offensive that has run them out of captured towns and villages since February this year.
The use of improvised explosive devices and suicide bombings has increased since May 29 when President Muhammadu Buhari took office vowing to crush the rebellion.
According to an AFP tally, at least 450 people have lost their lives since that date.