You know, I had the exact same feeling up until about 3 years ago. I moved down south with my now wife to her home town, Horsham. The petrol is crippling for home games never mind away games so I gave up my AST and just kept my HST. I still went to the occasional away game (although that has slowly stopped as well).
Her old man and brother are both Chelsea fans but due to work commitments and costs it is rare they get up to the Bridge so to get their fix of live football they went to watch the local non-league side, Horsham, who at that time were in the Isthmian League Ryman Premier Division (one below Conference South).
One weekend, I went along and to be honest I didn't get hooked right away; It was ok, there was a bit of an atmosphere, you could drink on the terrace but the football was pretty dire. It just didn't feel the same as the Albion.
Anyway, I started to go more and more, started to recognise the same old faces, meeting up early for drinks and train journeys, started to know the songs and, before I knew it, I was going to every game that Albion were away.
I've been to about 30 non-league grounds with Horsham to go with a range of ex-league clubs in the conference during my time travelling with the Albion and 80+ league grounds, due to the ridiculous amount of cup competitions and last season we were relegated to the league below promising a whole range of new grounds next season, too.
I still have my Albion season ticket and I don't go to enough home games to get a Horsham season ticket but at £10 a match ticket (most grounds less than that and the prices due to go down to about £5 at most grounds next season in the league below), it gives you that fix you need as a live football fan for about £100 less per game.
And, as we all know, once you get that bug of going it is already too late. You've got the bug and you start to care, whether you like it or not...