How to Deter Advanced Botnets with Intrusion Detection Systems

The most basic botnets are easy to control because they employ basic social engineering methods. They may trick a user to click on an email or web link, install a plugin, watch a porn video, download a file, or click on an online ad. Even those exploiting vulnerabilities on operating and antimalware software are sometimes easy to detect. The basic ones are easy to detect with simple signature matching.

However, advanced botnets use advanced techniques to prevent detection and blocking. It is important to understand what techniques a bot is using to launch effective anti-bot protection methods. Advanced bots and botnets require advanced intrusion detection systems.

What Features Do Advanced Bots Have?

Rarely will a bot that has malicious intentions use a single technique. Only that advanced bots have their owners spending more time to work and improve on these bots. These bots hence protect themselves from being easily sniffed out. They may use obfuscation, fast-flux, virtual machine detection, rootkits, and random generation of domain names to achieve this.

1. Randomly generating domain names 

These bots try to hide by randomly generating new domain names. The owner will register one domain but the bot will query several of them. It may also increase the number of domains generated per bot per given time. This makes it even harder for security vendors to pre-register domain names and figure out the command control server.

Different bots employ different string generators with some basing the generation on popular social media topics. Others construct words randomly and combine them with suffixes. Those that keep changing domains to hide are known as domain fast-flux bots. Some bots also rotate their websites across multiple IPs. These prevent detection by changing their DNS records. They are called IP fast-flux.

2. Virtual Machine Detection to Hide

In some cases, bot detection methods involve placing a virtual machine in a network and then monitoring any incoming traffic. They will know it is obvious that the incoming traffic is from malware since that server does not send out any traffic.

However, some advanced types of bots and botnets can detect if the infected machine is a virtual machine or not. They can try to identify by trying remote timing attacks, using listed services, or checking if the registry entries are generated from virtual machines. This makes them very hard to detect.

3. Rootkits

Botnets that employ rootkits hide from antivirus and malware detection tools. They make use of the windows system kernel files to hide their binary files from detection. Hacker defender is a good example of these types of bots. These types of bots are hard to detect using normal anti-virus systems.

4. Fast-Flux

These types of botnets are common in phishing and spamming campaigns. They are hard to detect and control because they hide the servers responsible for the malware updated copies. They do so by using fast-flux. In this case, they make it hard to block the IP of the server by changing the DNS to IP mapping of their download location. It also renders other IP-based infection detection useless. Some versions of this type use double fast flux where they change the A and NS records. It becomes even more difficult to detect them.

5. Obfuscation 

Malware using this method makes it hard to detect with normal signature matching. They can use JavaScript obfuscation if they are web-based or packing and encryption of their binaries. Javascript hides the attack vector while the packing and encryption generate a new hash that differs from the previous one of the same malware.

Defending Yourself From Advanced Botnets: Intrusion Detection Methods

1. Reverse Engineering Techniques

Advanced intrusion detection methods use reverse engineering methods on botnets that try to hide detection. It is useful for bots employing randomly generating domain names, fast-flux, and other methods that rely on changing IPs and domain names. Detection methods do this to identify how the malware attacks systems. It can analyze the .exe file sections, disassemble the malware, and other aspects such as the IP and domains used to identify the domain mainly used command control server.

2. Use of honeypot machines

Advanced botnet detection methods include using a honeypot, which is a machine placed in a network and exposed to hackers and botnets. The honeypot does not send any traffic out and therefore any incoming traffic is suspected as malicious. The intention of placing the honeypot in a network is to monitor these attacks. A honeynet comprises many machines of this kind. They can be either virtual machines or vulnerable machines in the network. These machines can also be set to look for malicious servers. They can use attack links used by malware owners. Once the virtual machine visits these links, it can detect which machines have been affected and their settings.

Honeypots can be configured to emulate vulnerabilities. However, these are easy to detect by malware. They are most useful for automated worm-like bots. They can then collect self-replicating malware. These honeypots can also consist of simulated networks of real and virtual hosts. This makes it difficult to be detected by malware with the capability to detect virtual machines.

3. Spamming bot detection

Command and control servers are now easy to detect for spamming bots and these have reduced significantly in number. The number has also reduced because it is easy to detect the domain providers. The DNS blacklisting technique is one way of deterring these types of botnets. A graph of nodes linked to the known botnet is created. It is possible to detect any new peers infected with the type of botnet if any botnet of the type queries the service providers or the quarrying node correlates to those on the known list.

4. Network-based bot detection 

These network detection systems employ network-level data to identify sniffing intrusion detection tools. They can also use network flow monitors.

5. Behavior analysis bot detection

 The detection systems, in this case, use behavior clusters to identify botnets before employing anti bot protection. They simply track botnets and use clustering to classify any new suspects.

6. Agents

The use of static and mobile detection systems allows security companies to deploy software that can actively lookout for data about vulnerability. Static agents are placed on static positions while mobile agents move to different hosts and servers looking for vulnerability data instead of waiting for data and relying initially on alerts from malware and botnets.