Cooperative spectrum sensing is key to the success of cognitive radio networks. Recently, fully distributed cooperative spectrum sensing has been proposed for its high performance benefits particularly in cognitive radio ad hoc networks. However, the cooperative and fully distributed natures of such protocol make it highly vulnerable to malicious attacks, and make the defense very difficult. In this paper, we analyze the vulnerabilities of distributed sensing architecture based on a representative distributed consensus-based spectrum sensing algorithm. We find that such distributed algorithm is particularly vulnerable to a novel form of attack called covert adaptive data injection attack. The vulnerabilities are even magnified under multiple colluding attackers. We further propose effective protection mechanisms, which include a robust distributed outlier detection scheme with adaptive local threshold to thwart the covert adaptive data injection attack, and a hash-based computation verification approach to cope with collusion attacks. Through simulation and analysis, we demonstrate the destructive power of the attacks, and validate the efficacy and efficiency of our proposed protection mechanisms.