Three young girls saw a mobile phone appear over the top of their changing room cubicle on two occasions at Bicester Leisure Centre.
The alleged voyeurism happened in the swimming pool cubicles at around 7.15pm on September 10, and at about 5.50pm on September 24.
There is currently no description of the offender but police are appealing for anyone with information to get in touch.
Investigating officer PC Lee Dickinson, based at Bicester police station said: “These incidents are very concerning and we are in the process of carrying out a full investigation into it.
“We are appealing for anyone who may have witnessed this incident to come forward and help us with enquiries.
“We would like to hear from anyone with any information.”
Anyone with information is asked to call 101 and quote 43180278561.
Reports can be made online at https://www.thamesvalley.police.uk/tua/tell-us-about/cor/tell-us-about-existing-case-report/ or call Crimestoppers anonymously on 0800 555 111.