I always thought major reason for the decline was all the other alternatives coming up back then (msn messenger and the likes) and more recently things that aren't really an alternative but steal time anyway (facebook and the likes) ?
I agree here, my friends weren't affected by ops (we were all ops in our own channels) - they moved to MSN and gave up on IRC. Then MSN was dropped and they moved to facebook/google chat, and now it's almost like IRC was by using group chats in whatsapp, but without meeting any new people.