Improper use of schema markup will in fact directly affect your rankings.
It says as much in the notice you got from Google.
To improperly use schema is not a good idea. This is especially true of "Review" schema.
My recommendation is to fix the current issues to eliminate any possibilities of a negative effect on your rankings.