Typically, if you get a penalty and your rankings drop, you assume when the penalty is lifted your rankings return.
Often that is not the case, especially when it comes to link penalties.
A Google Webmaster Help thread has two statements from John Mueller from Google which explains why that may be the case.
First let me quote his post:
Now let’s pull out two clauses:
(1) The primary manual action that is affecting your site is that these unnatural links are being ignored. Yes, when links that were once counted in your ranking are no longer counted and never will be counted in the future, that will have an impact on your rankings for a long time.
(2) Keep in mind that the manual action here might not be the strongest element affecting your site’s performance, we use over 200 factors… Now this statement is more interesting. John is saying that a manual action might not be the reason his rankings are not doing well in Google.
John is saying that even if he didn’t have a manual action his rankings still probably wouldn’t be all that great.
I love the transparency in this post.
Forum discussion at Google Webmaster Help.