A What?
A directory traversal attack is a type of brute force attack by which a hacker uses to get both an understanding of your web site structure and potential access to files which have not been protected from public access.
Why?
It can also be used to steal all the content from a web site by both analysing files called from web pages as well as those detected during the scan. Its not uncommon for unskilled users to place files on a server for easy access by others (friends, business colleagues etc) then forget about them, this then provides public access to unscrupulous people who might find the file(s) useful.
The other risk is that previously known exploitable code might be left on the server, especially if default installation files are left intact on the server with default user account and password details.
How is it done and What is the risk?
A hacker might scan a range of IP addresses to locate web servers that do not yet have a domain name associated with them, they will also use Google to find sites that have content they are interested in and then mine the site for potential data files. One common tool used is "DirBuster", it has a very extensive dictionary of keywords that it scans for, the tool uses forged headers to mask its appearance and then it systematically navigates the whole web site. If the tool crafts URL's that are invalid the web server will return status codes to indicate the type of error, the tool interprets these returned status code to make its mining effort more efficient.
Returned HTTP Status Codes
HTTP status codes are returned by the web server in response to URL requests. If a legitimate user stores a URL's in their browser history that is no longer valid or moved, the web server will return a numeric code response indicating whether the web site content exists or not and whether we may have typed the URL improperly.
If the content is present and valid then the web server will return a status code of 200, because that indicates that everything went properly, the attacker now knows the URL path is valid and can rapidly interrogate the path for additional files and directories.
Here is a summary of the most important HTTP status codes that web browsers use and tools like DirBuster utilize to find directories and files in web sites.
- 100 Continue – Codes in the 100 range indicate that, for some reason, the client request has not been completed and the client should continue.
- 200 Successful – Codes in the 200 range generally mean the request was successful.
- 300 Multiple Choices – Codes in the 300 range can mean many things, but generally they mean that the request was not completed.
- 400 Bad Request – The codes in the 400 range generally signal a bad request. The most common is the 404 (not found) and 403 (forbidden).
One down side to this attack is the load on the site increases causing delays and disruptions to legitimate traffic. The other is a sudden increase in logging of the web server activity and this does have the potential to exhaust disk space on the web server. The other downside is the wasted bandwith which you the client are paying for in your hosting plan.
Prevention
The most effective way to prevent this attack is to implement an intrusion detection system that recognises the success/fail traffic pattern and either blocks/drops or rate limits the client IP. Most attackers are lazy and don't alter the headers the tools use, so these can be identified and blocking at the web server level can also be implemented.
A sample .htaccess file that blocks common tools is shown below (Thanks to thehackerspot.com for crafting this configuration file):
RewriteEngine On
<IfModule mod_rewrite.c> R
ewriteCond %{HTTP_USER_AGENT} ^w3af.sourceforge.net [NC,OR]
RewriteCond %{HTTP_USER_AGENT} dirbuster [NC,OR]
RewriteCond %{HTTP_USER_AGENT} nikto [NC,OR]
RewriteCond %{HTTP_USER_AGENT} SF [OR]
RewriteCond %{HTTP_USER_AGENT} sqlmap [NC,OR]
RewriteCond %{HTTP_USER_AGENT} fimap [NC,OR]
RewriteCond %{HTTP_USER_AGENT} nessus [NC,OR]
RewriteCond %{HTTP_USER_AGENT} whatweb [NC,OR]
RewriteCond %{HTTP_USER_AGENT} Openvas [NC,OR]
RewriteCond %{HTTP_USER_AGENT} jbrofuzz [NC,OR]
RewriteCond %{HTTP_USER_AGENT} libwhisker [NC,OR]
RewriteCond %{HTTP_USER_AGENT} webshag [NC,OR]
RewriteCond %{HTTP:Acunetix-Product} ^WVS
RewriteRule ^.* https://127.0.0.1/ [R=301,L]
</IfModule>
Ultimately, the best way to reduce the access and damage to a web site is to:
- Implement authentication mechanism like CAPTCHA forms to reduce the effectiveness of automated tools.
- Secure all content not meant for public access.
- Keep access control lists up to date (.htaccess for example). For a comprehensive list see https://perishablepress.com/ultimate-htaccess-blacklist/
- Rebuild a clean image of your site regularly.
- Have automated cleanup tools remove old log files.
- Automatically remove empty directories and temporary files used in the running of your web site.
- Have automated reporting tools so you can see the status of your running site.
- For commercial web sites, using a tool like SaltStack to build and deploy your site means that you should be able to build a new image of your site at any time.
- Build a new clean OS image that's fully patched and up to date with the minimal software installed to reduce other potential attacks.
- Consider Country (GeoIP) blocking, if your market is a particular country then limit traffic to your site to that country.
For the more advanced user/administrator, having the ability to run the web site from RAM disk will both speed it up and in the event of an attack a reboot puts a clean image up for fast access with no/minimal disk IO, this reduces the impact on your server or cloud infrastructure when these tools are used against you.
How dangerous are these tools?
The tool "DirBuster" itself is not dangerous, it leaves the door open to other exploits. As these tools evolve, the risks will change.
