Horizontal pleiotropy occurs when the variant has an effect on disease outside of its effect on the exposure in Mendelian randomization (MR). Violation of the 'no horizontal pleiotropy' assumption can cause severe bias in MR. We developed the Mendelian randomization pleiotropy residual sum and outlier (MR-PRESSO) test to identify horizontal pleiotropic outliers in multi-instrument summary-level MR testing. We showed using simulations that the MR-PRESSO test is best suited when horizontal pleiotropy occurs in <50% of instruments. Next we applied the MR-PRESSO test, along with several other MR tests, to complex traits and diseases and found that horizontal pleiotropy (i) was detectable in over 48% of significant causal relationships in MR; (ii) introduced distortions in the causal estimates in MR that ranged on average from -131% to 201%; (iii) induced false-positive causal relationships in up to 10% of relationships; and (iv) could be corrected in some but not all instances.