I am guessing you are using Direct3d to render, since I noticed that the settings menu acts pretty wonk in it with regards to anti-aliasing. If you see something like "Nonmaskable 3" under your settings, that's why it isn't working; FSAA needs to be set to an integer value in ogre.cfg (FeralHeart wants to set it to "Nonmaskable/Level #" for some odd reason...). You can edit ogre.cfg and manually change it to FSAA=8. Also, some values don't work, even if displayed by the FeralHeart settings menu. in OpenGL, levels up to 16 are displayed, but when set to 16, AA is not enabled for me. Also, I am guessing AA needs to be kept to a power of two so stick to (0, 2, 4, 8, et cetra...)
You should be able to see the difference immediately on the sky's horizon of the start screen when fullscreen (it shouldn't look sharp enough to cut a pizza any longer.)