Like you, I never fancied TT week cos of the crowds. I've been over for the Manx GP three times. Magic. Love it - much quieter than TT week so you can get served in the pubs, lot of interesting, quirky enthusiasts & machinery (lot of classic stuff) & there's plenty of room on the course - you can put in laps pretty much as you wish without masses of other traffic, you can be out on the Mountain with hardly anyone else about...but you still feel the history of the place: Mountain mile, Kate's cottage, Quarry bends, Laurel Bank etc.etc.etc...
Top ManxGP racers are very quick, some unbelievably smooth, some loose & crazy, & they're racing over the full mountain course.
Still have no desire to subject myself to the TT, but I really miss the Manx - the atmosphere is like nothing else I've ever experienced - & will go again one day. Like Big J, I don't believe there's anything in motorcycling to compare with being on a bike, on the IOM during racing weeks.
KB