The more you have an impact on the copyright owner's potential market value of their work, the less likely the court will consider your use fair. Context matters; after all, reviews can have a large market impact but still receive Fair Use protections. Likewise, the court may find that your work has a significant market impact even if you don't directly affect the original work. For example, even if an author isn't actively pursuing making his book into a movie, the court would find that an unauthorized movie has a market impact on the author. This is because it deprives him of his ability to later create or license a movie.
Click here to learn if you are protected by Fair Use