So back to finding out what this class weblogic.xml.query.xdbc.Context was and how we could solve the memory leak.
A quick search in Eclipse showed this class gets loaded from com.bea.core.xquery_18.104.22.168.jar and the Eclipse calling trace showed a call to org.apache.xmlbeans.impl.store.XqrlImpl$CompiledQuery.
We knew we are using apache xmlbeans in our application and checked our Weblogic start classpath to see this jar had been added to the classpath since we had some runtime issues without it.
But we did not want to use the weblogic version of xmlbeans, so only tried with the Apache one instead.
So the changes made were to remove the above jars from start classpath and add saxon jars instead. We also edited weblogic-application.xml to specify the package from Apache
We then repeated our performance tests and monitored the heap dumps to verify the problem had gone away.
There was no instance of the weblogic.xml.query.xdbc.Context class at all which caused earlier heap block, so this indicates our fix is applied correctly