A game-theoretic analysis to defend against remote operating system fingerprinting

Remote Operating System (OS) Fingerprinting is a precursory step for launching attacks on the Internet. As a precaution against potential attacks, a remote machine can take a proactive counter-strategy to deceive fingerprinters. This is done by normalizing or mystifying the distinguishing behaviors in the packets. However, the unified modification causes a significant performance degradation to benign clients. Using a game-theoretic approach, we propose a selective and dynamic mechanism for counter-fingerprinting. We analyze the interaction between a fingerprinter and a target by modeling the problem as a signaling game and name it Defense-Game. Here, the defender observes the attacker's actions and takes moves accordingly. We derive the Nash equilibrium strategy profiles based on the information gain analysis and propose a defense mechanism named Strategic-Defender. Our game-theoretic approach appropriately distinguishes a fingerprinter from a benign client and mystifies packets to confuse the fingerprinter, while minimizing the side effects on benign clients. The performance analysis shows that the defense mechanism can reduce the probability of success of the fingerprinter significantly, without deteriorating the overall performance of other clients. Later, we restructure the game as a screening game where the attacker plays as the follower so that it can recognize the type of the target and act accordingly. We name the game Attack-Game and solve it for the equilibrium strategies. We evaluate the performance of Strategic-Defender when the attacker takes actions according to the optimal results of Attack-Game. The evaluation results show that the attacker's payoff may increase a bit, however, it is still too low to succeed.