In a hearing at Oxford Crown Court on Friday, Walsh admitted 18 counts of making indecent images and one of possessing 143 indecent pictures downloaded from the internet in March this year.
The 51-year-old was caught when a passenger on a train looked over his shoulder and saw him reading an article. When he looked more closely he realised it was what he described as a depraved "fantasy story". The passenger reported the incident to police.
Walsh has been put on the Sex Offenders' Register ahead of being sentenced in January.
Paul Farrer, representing Walsh, who now lives in Stoke on Trent, said the offences had come at a "difficult time" when he was considering his future as a priest at St John the Evangelist Church in South Bar, Banbury.
Mr Farrer said: "This behaviour was wholly alien to his normal character.
"All the images were viewed and downloaded during a single 40-minute period. They were simply viewed, downloaded and stored on his machine."
Rachel Drake, prosecuting, said Walsh had not paid to visit any child pornography websites.
"It is clear from the examination of the computer that it had been used to view websites which were free," she said.
She added the examination had shown he had visited four of five different websites.
The images recovered from Walsh's computer covered levels one, two and four of a one to five scale used to denote the explicit nature of pictures.
There is an ongoing discussion between the prosecution and defence over how many images there were at each level. The matter is expected to be resolved at the sentencing hearing on January 17.
Walsh was granted conditional bail by Judge Julian Hall. He said: "I don't give you any promises as to what your sentence will be."
Following the hearing the Archbishop of Birmingham, the Most Rev Vincent Nichols, said: "This court case is the conclusion of a sequence of sad events which stretch back to March 2004.
"Many people will be distressed at this outcome. I count myself among them. Walsh has given this diocese fine service. Since March he has stepped down from all active ministry as a priest.
"On a number of occasions since then Walsh has expressed his sorrow and regret at the damage he has done to the Church and to himself.
"Walsh will be much in my prayers, as will all those who feel let down by these events and are now deeply upset."