OpenSSL BN_mod_exp_recp

Hunt Hypothesis

The OpenSSL BN_mod_exp_recp function may indicate the use of weak cryptographic practices that could lead to vulnerabilities in key exchange operations. SOC teams should proactively hunt for this behavior in Azure Sentinel to identify potential misconfigurations or outdated cryptographic implementations that could be exploited by adversaries.

YARA Rule

rule OpenSSL_BN_mod_exp_recp
{	meta:
		author = "Maxx"
		description = "OpenSSL BN_mod_exp_recp"
	strings:
		$c0 = { B8 C8 02 00 00 E8 ?? ?? ?? ?? 8B 84 24 D4 02 00 00 55 56 33 F6 50 89 74 24 1C 89 74 24 18 E8 ?? ?? ?? ?? 8B E8 83 C4 04 3B EE 89 6C 24 0C 75 1B 8B 8C 24 D4 02 00 00 6A 01 51 E8 ?? ?? ?? ?? 83 C4 08 5E 5D 81 C4 C8 02 00 00 C3 53 57 8B BC 24 EC 02 00 00 57 E8 ?? ?? ?? ?? 57 E8 ?? ?? ?? ?? 8B D8 83 C4 08 3B DE 0F 84 E7 02 00 00 8D 54 24 24 52 E8 ?? ?? ?? ?? 8B B4 24 EC 02 00 00 83 C4 04 8B 46 0C 85 C0 74 32 56 53 E8 ?? ?? ?? ?? 83 C4 08 85 C0 0F 84 BA 02 00 00 57 8D 44 24 28 53 }
	condition:
		$c0
}

Deployment Notes

This YARA rule can be deployed in the following contexts:

• Microsoft Defender for Endpoint — Custom indicators / advanced hunting
• Email Gateway — Attachment scanning
• File Share Monitoring — Periodic scanning of shared drives
• YARA CLI — Manual threat hunting on endpoints

This rule contains 1 string patterns in its detection logic.

• Source Rule

False Positive Guidance

• Scenario: Regular TLS handshake during normal HTTPS traffic
  Filter/Exclusion: http.client.method == "GET" and http.server.hostname != "localhost"

• Scenario: Scheduled system certificate renewal using certbot
  Filter/Exclusion: process.name == "certbot" or process.args contains "renew"

• Scenario: Admin task using openssl to generate a new key pair
  Filter/Exclusion: process.name == "openssl" and process.args contains "genpkey"

• Scenario: Automated backup job using rsync with SSH encryption
  Filter/Exclusion: process.name == "rsync" and process.args contains "ssh"

• Scenario: Internal tool using OpenSSL for internal encryption operations
  Filter/Exclusion: process.name == "internal_tool" or process.args contains "internal_encryption"