This has been an issue since release and can't find any solution for it. Here's a video to see how it looks like http://www.youtube.com/watch?v=do_llujhc1Y If anyone has managed to fix it please spread the knowledge
Topic on Talk:Mass Effect
Also, schematics here and here. Discussion thread here.
EDIT: putting aside the majestic fix by Methanhydrat, I'm still banging my head over what's actually wrong in the game code. And with ME3Explorer's incredible advances, I could dig up way better inside packages (even though it's still not perfect: for example I can not find any usage whatsoever of the m_fCamStickScalar property of BioCameraBehavior.. but if you set it to 0 the in-game camera stops to work)
And it's really a mess, as good old Dybuk reported while developing the controller mod. Not that having different BioCameraBehaviors on top of BioPlayerController on top of BioPlayerInput on top of PlayerInput (on top of Input?) was bound to be easy already, but spaghetti code with convoluted ifs really doesn't help.
Anyway, speaking of the latter class, I have confirmed bEnableMouseSmoothing is indeed at least doing something. It's not enough though of course.
Therefore, devoid of ideas, I shot in the dark and tried to compare the scripts with ME2 code.. And while there really isn't anything similar to the PreProcessInput function where ME2 employs the definitive bUseMouseDampening (or any other similarity whatsoever between BIOC_Base and SFXGame), the Engine file was basically identical as far as I could care.
Except the PlayerInput function in ME1 is missing a mysterious multiplication by the Outer.CustomTimeDilation object. Thus I'm left wondering if the culprit for mouse acceleration couldn't be this triviality, or what else is hiding in the other haystack.
EDIT2: lol