Security Ripcord

While reviewing some Apache log files the other day I started to wonder if somebody had already come up with a way to detect common attack characteristics by the information they contain.  Although searching for entries containing “SELECT,” “xp_cmdshell,” and other attack terms can turn up useful information it is difficult to cover all of the attack types and the different ways they can be represented within the log entries.  Some Googling brought me to a project started by Romain Gaucher called apache-scalp which is hosted on Google Code.

Scalp! is a log analyzer for the Apache web server that aims to look for security problems. The main idea is to look through huge log files and extract the possible attacks that have been sent through HTTP/GET (By default, Apache does not log the HTTP/POST variable).

This tool uses an event filter file created by and for the PHPIDS project.  PHPIDS was created parse web traffic and alert on anomalous activity as it occurs instead of having to parse the log files.  To help in those instances that PHPIDS is not deployed, Scalp applies the same principle to the collected information.  Utilizing the PHPIDS signature file, default_filter.xml, Scalp will generate an alert file in the user’s choice of text, HTML, and XML formats.  This output, however, is alert data only and can be very cumbersome to review.  When a lot of information is involved human review is not a very effective way to determine trends and other specifics about the data.

<?xml version=”1.0″ encoding=”utf-8″?>
<!–
File created by Scalp! by Romain Gaucher - http://code.google.com/p/apache-scalp
Apache log attack analysis tool based on PHP-IDS filters
–>
<scalp file=”apache_log” time=”Sat-27-Dec-2008″>
<attack type=”xss” name=”Cross-Site Scripting”>
<impact value=”5″>
<item>
<reason><![CDATA[Detects JavaScript with(), ternary operators and XML predicate attacks]]></reason>
<regexp><![CDATA[(?:with\([^)]*\)\))|(?:\.\s*source\W)|(?:\?[^:]+:[^;]+;)]]></regexp>
<line><![CDATA[xxx.28.xxx.249 - - [26/Aug/2008:00:00:13 -0700] “GET /d.AuthenticateUser1?p_page=http://webx.companyX.com/publish/01/na/en.html&p_HTTP_USER_AGENT=Microsoft%20Internet%20Explorer-4.0%20(compatible;%20MSIE%206.0;%20Windows%20NT%205.1;%20SV1;%20.NET%20CLR%201.1.4322) HTTP/1.0″ 200 1547
]]></line>
</item>
<item>
<reason><![CDATA[Detects JavaScript with(), ternary operators and XML predicate attacks]]></reason>
<regexp><![CDATA[(?:with\([^)]*\)\))|(?:\.\s*source\W)|(?:\?[^:]+:[^;]+;)]]></regexp>
<line><![CDATA[xxx.16.xxx.158 - - [26/Aug/2008:00:00:19 -0700] “GET /d.AuthenticateUser1?p_page=http://webx.companyX.com/publish/01/na/en.html&p_HTTP_USER_AGENT=Microsoft%20Internet%20Explorer-4.0%20(compatible;%20MSIE%206.0;%20Windows%20NT%205.1;%20SV1;%20.NET%20CLR%201.1.4322) HTTP/1.1″ 200 1570
]]></line>
</item>
<item>
<reason><![CDATA[Detects JavaScript with(), ternary operators and XML predicate attacks]]></reason>
<regexp><![CDATA[(?:with\([^)]*\)\))|(?:\.\s*source\W)|(?:\?[^:]+:[^;]+;)]]></regexp>
<line><![CDATA[xxx.84.xxx.90 - - [26/Aug/2008:00:00:41 -0700] “GET /d.AuthenticateUser1?p_page=http://webx.companyX.com/publish/01/vie/en.html&p_HTTP_USER_AGENT=Microsoft%20Internet%20Explorer-4.0%20(compatible;%20MSIE%206.0;%20Windows%20NT%205.1;%20SV1;%20.NET%20CLR%201.1.4322) HTTP/1.1″ 200 1568
]]></line>
</item>
<item>
<reason><![CDATA[Detects JavaScript with(), ternary operators and XML predicate attacks]]></reason>
<regexp><![CDATA[(?:with\([^)]*\)\))|(?:\.\s*source\W)|(?:\?[^:]+:[^;]+;)]]></regexp>
<line><![CDATA[xxx.160.xxx.243 - - [26/Aug/2008:00:00:57 -0700] “GET /d.AuthenticateUser1?p_page=http://webx.companyX.com/publish/01/par/en.html&p_HTTP_USER_AGENT=Microsoft%20Internet%20Explorer-4.0%20(compatible;%20MSIE%206.0;%20Windows%20NT%205.1;%20SV1;%20.NET%20CLR%201.1.4322) HTTP/1.1″ 200 1573

To help identify trends and other interesting information associated with alerts generated by Scalp I have put together a external parser to generate a readable report.  The Scalp External XML Reporter (SEXR) takes the information within a Scalp XML file and produces several outputs to either standard out or a text file.

The first output is the full parse of the generated alerts.  Although this output can still contain a lot of information, it is much easier to read and understand than the original Scalp output.  (The following output, as well as the rest in this post, have been snipped for brevity and their format slightly altered getting the text into this post.)

sexr.py: Conducting full scan of 1 files
scalp: {’file’: ‘apache_log’, ‘time’: ‘Sat-27-Dec-2008′}
attack: {’type’: ‘xss’, ‘name’: ‘Cross-Site Scripting’}
impact: {’value’: ‘5′}
item
reason
- Detects JavaScript with(), ternary operators and XML predicate attacks
regexp
- (?:with\([^)]*\)\))|(?:\.\s*source\W)|(?:\?[^:]+:[^;]+;)
line
- xxx.28.xxx.249 - - [26/Aug/2008:00:00:13 -0700] “GET /d.AuthenticateUser1?p_page=http://webx.companyX.com                    /publish/01/na/en.html&p_HTTP_USER_AGENT=Microsoft%20Internet%20Explorer-                        4.0%20(compatible;%20MSIE%206.0;%20Windows%20NT%205.1;%20SV1;%20.NET%20CLR%201.1.4322) HTTP/1.0″ 200 1547

item
reason
- Detects JavaScript with(), ternary operators and XML predicate attacks
regexp
- (?:with\([^)]*\)\))|(?:\.\s*source\W)|(?:\?[^:]+:[^;]+;)
line
- xxx.16.xxx.158 - - [26/Aug/2008:00:00:19 -0700] “GET /d.AuthenticateUser1?p_page=http://webx.companyX.com/publish/01/na/en.html&p_HTTP_USER_AGENT=Microsoft%20Internet%20Explorer-4.0%20(compatible;%20MSIE%206.0;%20Windows%20NT%205.1;%20SV1;%20.NET%20CLR%201.1.4322) HTTP/1.1″ 200 1570

item
reason
- Detects JavaScript with(), ternary operators and XML predicate attacks
regexp
- (?:with\([^)]*\)\))|(?:\.\s*source\W)|(?:\?[^:]+:[^;]+;)
line
- xxx.84.xxx.90 - - [26/Aug/2008:00:00:41 -0700] “GET /d.AuthenticateUser1?p_page=http://webx.companyX.com/publish/01/vie/en.html&p_HTTP_USER_AGENT=Microsoft%20Internet%20Explorer-4.0%20(compatible;%20MSIE%206.0;%20Windows%20NT%205.1;%20SV1;%20.NET%20CLR%201.1.4322) HTTP/1.1″ 200 1568

Although helpful and informative it might be a little more helpful to know more specific information about the alerts.  SEXR’s count scan option is designed to remove some of the more extraneous information by providing the number of alerts detected and the names of the alerts with which they are associated.

sexr.py: Conducting count scan of 1 files
scalp: {’file’: ‘apache_log’, ‘time’: ‘Sat-27-Dec-2008′}
attack: {’type’: ‘xss’, ‘name’: ‘Cross-Site Scripting’}
Impact 5 Items: 299
- ‘Detects JavaScript with(), ternary operators and XML predicate attacks’: 248
- ‘Detects self-executing JavaScript functions’: 51
Impact 4 Items: 655
- ‘Detects common XSS concatenation patterns 1/2′: 655
Impact 3 Items: 80
- ‘Detects common comment types’: 80
attack: {’type’: ‘lfi’, ‘name’: ‘Local File Inclusion’}
Impact 5 Items: 199
- ‘Detects specific directory and path traversal’: 199
attack: {’type’: ‘rfe’, ‘name’: ‘Remote File Execution’}
Impact 5 Items: 383
-’Detects url injections and RFE attempts’: 383
sexr.py: Done

After determining the types of attacks that were detected from the Apache log files it might be interesting to know where these attacks originated.  SEXR’s source IP scan option provides the source IP address for all of the alerts detected and a count of how many times these source IP addresses were associated with the attack.

sexr.py: Conducting IP scan of 1 files
scalp: {’file’: ‘apache_log’, ‘time’: ‘Sat-27-Dec-2008′}
attack: {’type’: ‘xss’, ‘name’: ‘Cross-Site Scripting’}
Impact 5 Items: 299
- Total Source IP Addresses: 209
- xxx.176.xxx.42: 1
- xxx.177.xxx.68: 1
- xxx.129.xxx.3: 1
- xxx.89.xxx.250: 1
- xxx.179.xxx.235: 1
- xxx.253.xxx.222: 3

Although Scalp does all of the heavy lifting I believe that SEXR can play an important role in evaluating the results that Scalp produces.  I hope that SEXR will help make the information provided by Scalp more usable for system, network, application administrators and security professionals.  To this end Romain has allowed me to update a few portions of Scalp to make its XML output more informative while also adding SEXR and the Scalp DTD file to the source code available online with the Scalp project.

If you have any comments, recommendations, or updates for the code, please let me know.

Go forth and do good things,

Don C. Weber

Reviewing my daily blog hits I came across another interesting search that directed somebody here.  “what a firewall and ids can tell you about an incident” Although I sarcastically commented on my Interesting Search Keywords page “That you need to turn on more logging” I have to admit that I meant it.  Firewalls and Intrusion Detection Systems will only provide the analyst (be it a professional incident responder or a system/network administrator) with a limited amount of information.  The real meat of an intrusion is going to come from the coalition of data from disparate logging resources.

• Firewall logs, depending on how logging is configured, is going to tell you about the connection attempts in and out of the network.  Most likely you are going to see the traffic that failed.  If the firewall has proxy capabilities you will be provided with specifics about the proxied traffic, also most likely the stuff that failed.  Depending on where the firewall is located within your infrastructure will determine how much noise will be present within the logs.  Internet facing firewalls will have a lot of noise and will therefore probably have failed logging toned down.  Internal firewalls, however, should not see much failed traffic.  So anything that shows up could protentially be interesting.  Comparing these logs could also be affective in determining the origin of specific attacks, were they intitiated internally or externally.
• IDS logs could be helpful or noisy as well.  These really depend on how well the security or network administrator has maintained the IDS sensors and where they have been placed.  Having every signature and capability turned on will lead to quite a bit of false positives to sort through.  But too much tweaking could leave administrators oblivious to reconnaissance efforts.  IDSes are great for determining if attackers are using known attack vectors to gain a beachhead within an environment.  But, like anti-virus engines, they are only as good as their signatures no matter how well the community and commercial companies stay on top of it.  I’m willing to bet it is going to be a long time before we see signatures for Clickjacking (BTW, Jeremiah Grossman recommends this article).
• These points aside, having the logs from both of these devices provides you with the best information of all.  What is the normal operational state of communications within the environment? There are no if, ands, or buts about it.  If you do not know what can be considered normal there is no way to asertain what is abnormal.  Having good logs from these devices will enable you to pin-point a specific time period that reconnaissance or even an attack occurred.  This could help you narrow your window of other investigation items be it system logs, file assess or write times, or registry modifications.

I said it on Twitter the other day, ” Logs are interesting, logs are fun, logs should be done by EVERYONE…..get to logging!!!” and I meant it.  It is very hard to investigate an intrusion or incident if there are no logs to review.  Bringing together the firewall logs, IDS logs, application logs, operating system logs, and anything else there is provides the foundation from which to build the response.  Without them you will just be staring at each other wondering what to do next.  And I’ll tell you what to do if you don’t have any logs.  Contain the incident, investigate as much as you can, return to a good operational state, disclose if you have to disclose, and TURN ON LOGGING.

Go forth and do good things,

Don C. Weber

I have to concede to Chris on several points of his latest post. I do so because:

1. He definitely has more experience, than I, deploying a variety controls in a variety of environments of varying size.
2. He definitely has more experience, than I, speaking to the capabilities of these controls and providing comprehensive and understandable analogies and examples.
3. He definitely has more time, than I, to correlate and integrate, free and expensive, disparate and concise literature and case studies to fuel his analogies and examples and employ them in a variety of circumstances.

After all, it is what he does for a living. And he is very good at it. That is why he is listed in my blogroll and the majority of the blogroll’s associated with my daily information security firehose. Hell, it is why he can list articles in many hard and soft copy information security publications.

Me, on the other hand, I am a security professional wielding my experiences and knowledge to the best of my ability to provide my employers and customers with the same level of service Chris provides despite my limitations due to time in service. I use my experiences with technology, interactions, and introspection to form my conclusions and present them as the very best solution for the situation. I will personally guarantee the deployment of every one of my recommendations and the provide mitigation suggestions when it is, as we know it will be, circumvented, exploited, outdated, outclassed, obsolesced, ineffective, unmanageable, flappable, overly expensive, or just plain wrong. In other words, I am confident and I am willing to make mistakes because I can fix them and the majority of the time I will not make them again.

I truly think that this whole blogging interaction started because of my attempt to be flamboyant about the topic to draw attention to it. Unfortunately, as most gussied up topics do, the central point of the discussion was lost for a while. Luckily, in his last post, Chris brought it back around. Let me try to talk about my point in very plain English.

I have a problem with vendors who are developing products that provide many security controls on one system (not UTMs, I’m talking firewalls performing a combination of spam detection, anti-virus, protocol analysis, data evaluation, and whatever else the vendor thinks will sell) and selling it as the ultimate solution for the perimeter of a company’s infrastructure. I have a problem with these solutions because the technologies they are combining on one system are not simple applications. They are robust technologies with a lot of complexity and I am afraid that the vendors will not take the interoperability of these technologies into considerations before they push them to market. I would much rather recommend to my employers and customers that we limit the utilization of such technologies to select portions of the internal network where they can provide the most value with the least concern. I feel much better placing tried and true simple, relatively speaking, controls at the locations associated with high risk. I don’t have full proof examples. I don’t have case studies to back up my hypothesis. I have my feelings and opinions. And, actually, since I am not dealing with Fortune 500 CEOs, CTOs, CISOs, and patent producing PHD weilding end users, I don’t really need it. In the realm of the small, limited budget, network, my feelings and opinions have been, to this point, sufficient.

Next, I don’t think I have a problem with purchasing a UTM to provide a combination of spam detection, anti-virus, protocol analysis, data evaluation, and whatever else the vendor thinks will sell because I believe that UTM developers have taken the complex nature of these technologies into consideration. I was hoping that somebody I know would respond by telling my, and their, readers whether or not UTM solutions are better than the “all-in-one” firewall solution advertised in the DarkReading article, and why. If I had to guess, because of my aforementioned lack of UTM experience, I would think that UTMs separate the responsibilities in much the same manner as role-based control.

Can anybody answer this question for me? It is all I really wanted out of the whole conversation.

So, Chris, I lay my King down so that we may reset the board and start the next conversation fresh. I think you are correct when you say that I need to provide more clarifying evidence during my conversations. I will take it to heart as much as I can in my day to day security related duties. I’ll even attempt to do so in my blogging. But, as my blog is more for personal edification, education, and venting I have a feeling that a few misguided and ill-informed opinions will slip in from time to time.

Go forth and do good things,

Don C. Weber

UTM, security, complexity, Security Ripcord, Chris Hoff

All of this started when I decided to react to the Kelly Jackson Higgins’ DarkReading Firewalls Ready for Evolutionary Shift article. Fast on its heals came a follow-up in reaction to a comment about UTMs. In the follow-up, since I have little experience with UTMs, I though I would call on a few people to validate or correct my assumptions as to how UTMs operate. Chris Hoff, however, took this as a call to arms and decided to teach me a thing or two about blogging while dissecting my carefully crafted viewpoint. Apparently, empirical demonstration is the wave of the future to be practiced by all who have a point to convey. In keeping with Chris’ effort I will attempt to comply on the limited research budget allocated by the senior management of Cutaway Security.

First, we need to start with the question. Chris wants to know how I came to the conclusion that:

single-sourced solution adds “complexity” and leads to “increased risk?”

Let’s start with the definitions from Webster.com.

complexity -
1 : something complex
2 : the quality or state of being complex

That won’t do.

complex -
1 a: composed of two or more parts : composite b (1)of a word : having a
bound form as one or more of its immediate constituents <unmanly is a
complex word> (2)of a sentence : consisting of a main clause and one or
more subordinate clauses
2: hard to separate, analyze, or solve
3: of, concerned with, being, or containing complex numbers <a complex
root> <complex analysis>

That’s better. My favorite is “2: hard to separate, analyze, or solve.” It is my favorite because it is the resultant situation when you combine things, especially complex things like…….oh, say…..firewalls, proxies, protocol analyzers, anti-spam engines, intrusion detection/protection capabilities, and SSL decryptors. In my mind, we are starting with things that are already complex and now we are combining them and thereby making them even more complex. It would be one thing if they merely worked side by side, but by configuring these technologies to interrelate we are not adding to the complexity we are multiplying the complexity. Certainly we already have some of these technologies interrelating, such as firewalls and proxies, and we have been doing this for a while. In fact, it has been manageable. But tack on a third, forth, fifth complex technology and I do not think that even the most intuitive management interface is going to be sufficient.

“Knock, knock….Don…..empirical, remember?” Oh, sorry, Chris.

To prove all of my points I thought about throwing up a firewall and installing several of these technologies and providing all of you with a break down of the complex installation and configuration procedures. I had also planned on researching each of the technologies for exploitable vulnerabilities and presenting the combined exploitable vulnerabilities as a complex series of doors to the internal data goldmine of any organization. But then I added the total funds allocated by the senior management of Cutaway Security and the project funding supplied by Chris and realized that I had already exceeded the budget by merely thinking about it. Oh, well. I guess some related vulnerabilities posted to SecurityFocus will have to suffice.

• Snort/Sourcefire DCE/RPC Packet Reassembly Stack Buffer Overflow Vulnerability
• Squid Proxy NTLM Authentication Buffer Overflow Vulnerability
• SpamAssassin Vpopmail and Paranoid Switches Remote Command Execution Vulnerability
• PCRE Regular Expression Library Multiple Integer and Buffer Overflow Vulnerabilities

Wait a cotton picking minute! All of these vulnerabilities only give the attacker privileges at the level that the application is running. No administrator or vendor would run these as root. And there are NEVER any privilege escalation vulnerabilities that would……opps…..hang on a sec….

• Linux Kernel Ptrace Local Privilege Escalation Vulnerability
• Linux Kernel CIFS Local Privilege Escalation Vulnerability

So, if you take the number of technologies implemented on a device, and you add them together you get a number higher than what you started with. And, if you count the number of risks associated with those technologies and you add them together (although I think you should multiply) you get a higher number of risks than you started with. All in a technology that is suppose to REDUCE RISK.

Although this might not have been the direction that Chris wanted me to go, this will have to suffice. By combining these technologies we are definitely increasing our risk in a way that the reduction in cost does not compensate. I guess the quickest example of this would have been to remind Chris that we ask our system administrators to limit the number of applications they place on a system. We do so to reduce the complexity of that resource, separate the vulnerabilities, and limit the amount of risk to the data traversing these applications. How can we, in good concience, request and even require them to follow these rules and then blatantly ignore them ourselves?

UPDATE:

In true blogging form (meaning I found it after I finished writing the original part of this post) I see that Mike Rothman has also weighed in on my opinion about complexity and risk. This post, in combination with Chris’, makes me realize that neither of the read the UTM Remix post very carefully. In the post I state that:

I wanted to cover this because UTM is actually a different animal then what I was originally addressing. Although I do not have any experience with Unified Threat Management, as a blogger I don’t feel ashamed jumping into it. I am sure that Chris Hoff, Rich Mogull, Lori MacVittie, Andy Willingham, or Alan Shimel will correct me if I am misguided.

Although Chris and Mike have labeled me a UTM hater, I have actually distanced UTM from my arguement as I believe they are different. I was hoping that the individuals listed could weigh in and clarify for me.

So, once again, I’ll state that I do not believe, albeit there is a lack of direct experience, that UTM fails to take the issue of increased risk due to complexity into consideration. Rather I believe, but need clarification on, that UTMs separate the functionality so that the data being analyzed is done so in a location that does not affect the overall operation of the system. Meaning, that if one of the technologies is exploited the attacker will not gain control of the entire system or will s/he be able to escalate their privileges in a manner that would adversely affect the rest of the system and the environment in which it has been deployed.

However, I do believe, and once again need clarification, that UTMs are expensive and difficult to deploy properly. Tarek has made a good comment about this to the UTM remix post. He points out that if the company has the money to spend on it, and he has seen instances where they do, that multiple UTMs from different vendors will help reduce some of the risk introduced by complexity. As it appears that he has worked in these environments more that I have, I will have to take his advice into consideration in the future.

Hopefully, all of this dispels Mike and Chris’ fear that I am spreading dis-information and FUD. And, yes, we are all still friends. But they do own me a beer.

Go forth and do good things,

Don C. Weber

Tarek, firewall, IDS, proxy, snort, squid, spamassassin, vulnerability, risk, Security Ripcord, Chris Hoff

After reading LonerVamp’s take on the application aware firewall, I started to wonder why people constantly want to consolidate their controls. This is not a new debate and DarkReading’s article Firewalls Ready for Evolutionary Shift is not ground breaking as the integration of firewalls and other security technologies has been bouncing around for years. Indeed, here we see Marcus J. Ranum talking about it on “Date: Fri, 29 Mar 2002 12:00:29 -0500″:

I suspect you are referring to “intrusion prevention” - which is a hot new marketing term but basically everything that’s being billed
as “intrusion prevention” is just firewalling + antivirus with a bit of fresh paint on it.

I’m willing to bet he has changed his tune a little bit since then but the evolution of firewalls with additional integrated controls has been going on since 2002 at least.

Of course I can see why people desire to integrate the technologies.

• It is more cost effective to have two or more technologies on one piece of hardware.
• You only have to manage one box.
• The controls can augment each other more effectively and efficiently (according to the advertising on the box).
• Firewalls usually represent a choke point to external and potentially hostile environments.
• Vendors can market it as the Silver Bullet (no relation to Gary McGraw’s podcast) of controls.
• “The next-generation firewall will have greater blocking and visibility into types of protocols,” says Greg Young, research vice president for Gartner.
• etc

Well, I have a problem with all of this. Why are we making our controls more complex? Complexity leads to vulnerabilities. Vulnerabilities lead to exploits. Exploits lead to compromises. Compromises lead to loss.

Certainly, everything has vulnerabilities. But that is my problem with placing multiple controls on one system. Fine, if my firewall has a vulnerability then it can be bypassed and my organization is screwed until we can respond. But I would prefer that my firewall was not bypassed because of a vulnerability in another control like a protocol analyzer or an intrusion detection system. Oh wait, these will be newer technologies with better software development practices so there should not be any additional vulnerabilities that allow for exploitation of the system or bypass of the controls……RIGHT!!!!

Don’t get me wrong. I am all for developing new technologies that will allow organizations to analyze their traffic so that they get a better picture of what is traversing and exiting their networks. I just think they will be more effective if they are deployed so that they augment each other’s control measures instead of threatening them by increasing the risk through complexity. Controls should reduce risk, not increase it.

So, when considering how to protect your data please do not cut corners. Evaluate your data distribution and dissemination, consider your architecture, determine which controls will increase efficiency while increasing security, and then deploy those controls so that they augment each other effectively.

Go forth and do good things,
Cutaway

LonerVamp, firewalls, IDS, controls, security, DarkReading, Security Ripcord

I am not sure why yet but Sourcefire and Checkpoint have decided to not move forward with the acquisition effort. I noticed it this morning on the TaoSecurity Blog. I’m sure we’ll be hearing alot more about this in the near future.

Cutaway