But none of them are locked in as judges for the upcoming TV show, and Cowell told celebrity TV show Access Hollywood he was in no hurry to make a decision.
Singer Jessica Simpson was also reported this week to be keen for a spot on the panel of the singing contest. But sources close to the show told People magazine that a final decision may take weeks.
Cowell told Access Hollywood he had spoken to Carey, who is pregnant with twins.
But he added; "We`ve spoken to lots of people. I`ve spoken to Paula ... we`ve taken it down to a smaller group of people and she (Abdul) is in that panel. But you now, it`s more than me who makes the decision."
Despite their often tempestuous relationship on "American Idol", Cowell has made no secret of his admiration and off-set friendship with the volatile Abdul.
Cowell said last month that he would be one of the "X Factor" judges.
The U.S. version of the "The X Factor" is due to debut on Fox in the fall of 2011, offering an unprecedented $5 million prize for the winner, along with a record contract.
But unlike "Idol", the "X Factor" judges will mentor the contestants closely, helping them with song choice, production, coaching and even style during the TV contest.
"It`s good to kind of delay the process ... You genuinely have got to trust they they`re going to be able to do the job you`re paying them to do," Cowell said of the new panel.
Cowell said that singers Katy Perry and Elton John -- whose names were floated in the media earlier this year --are not on the short list. (*)
Editor: Kunto Wibisono
Copyright © ANTARA 2011
