It looks like you're new here. If you want to get involved, click one of these buttons!
And you're apparently trying to use the .htaccess of vanilla after one was used by wordpress.
Check step 7.http://vanillaforums.org/docs/installation#moreinfo
Senior Developer at Vanilla Forums [GitHub, Twitter]