Through work and over the years, I've worked with a number of sporting bodies (FA, PGA, British Gymnastics, LTA, England Lacrosse, England Hockey, plus many more) and a number of them are clients of mine and the company i work for still.
I've therefore spoken to a number of senior people conversationally about all sorts of issues. To pick up on a couple of points briefly, most come back in the main to culture, but in no particular order....
The post 16 drop off in sport is massive in this country, particularly for women, infact for women it starts with puberty. Girls that do dance, gymnastics, athletics get sucked in to wanting to be popular, socialising and 'boys'. The major NGB's are always fighting against this but it seems like they are never winning. The government should do more because the long term effect on health, both mental and physical is well documented. Instead it seems like even standard PE is being reduced in general comprehensives.
Facilities is a problem as pointed out, Lacrosse now use Hockey pitch dimensions (I'm not sure if this is true at the top level) so that facilities can be multi purpose. The bodies try to address issues where they can but its tough and money isn't always forthcoming.
Mark mentioned 'middle classes' I remember having a conversation about this a number of years ago and it was brought to my attention that most sport is, and always has been middle class. Football is the big exception. Historically it was the middle class that had the time, and even now it is quite often the middle class that get the opportunity, not necessarily because it is not available to all, but it is the parents who value it and the parents who put emphasis in to it for their children.
Better educated kids tend to value it more as adults too, go to any evening netball league. You'll find most females still participating in their 20's are university educated, its not coincidence.