Security Ripcord


Cutaway Joins InGuardian, Inc.

June 1st, 2010 cutaway Posted in InGuardians, Security | 3 Comments » 1,095 views

When I left the United States Marine Corps and started college I knew two things.  1. I wanted my career to be in Computer Security and 2. I wanted to work for a group of professionals who operate at the same level of the Force Reconnaissance unit I had the pleasure of serving with for six and a half years.  I chose computer security because I figured it would be a growth industry with a strong job market for years to come (I was correct).  My vision of a Force Recon like security organization was born out of the need to surround myself with individuals who are self-driven, educators, drawn from various circumstances, volunteers, and flat-out just ready to do whatever it takes to do a good and professional job.

I achieved my first goal right out of college and basically due to sheer luck.  The company I was hired with decided to expand a security team and I was in the right place at the right time.  My second goal was a little harder to fulfill.  It has taken me eight years, five different jobs, collaboration with many outstanding security and IT professionals, countless hours of extra work, and the will to try and turn bad situations into mutually beneficial experiences for all involved.  It has been a long row to hoe but I have finally achieve the second goal.  At the beginning of this month I became a new member of InGuardians, Inc.

For the past two weeks I have been transitioning from the world of Incident Response back into the world of penetration testing, research and development, and security architecture.  I have been exposed to the world of Smart Grids, hardware hacking, and software evaluation.  To say the very least I have been thrust from the frying pan and into the fire.  The world does not stop spinning because of personal transition and I am experiencing this first hand right now.  But, it is exciting and educational.  My eyes have already opened to a new realm of possibilities and I intend to use that knowledge to ensure our team is a pivotal part of security developments and implementation in the years to come.

I would like to thank all of the member of InGuardians, Inc for their confidence in me and bringing me on-board.  I would also like to thank everybody out there who has helped me get to this point.  There are too many of you to do this individually, but I think you can all agree that I can show my thanks my continuing with my open education stance and providing you with an insight as to what I am experiencing and how I think it affects individuals as well as the industry.

Go forth and do good things,

Don C. Weber


Help support my training and travel to security conferences. Get your SANS Training and GIAC Certifications through the Security Ripcord.


SANS Security 508

April 10th, 2010 cutaway Posted in Education, Incident Response, Security, forensics | 3 Comments » 2,409 views

I recently attended SANS Security 508 at SANS 2010-Orlando.  When I told Harlan Carvey that I was going to attend this training he was concerned that I would not be exposed to anything I had not already exposed myself to through work and personal effort.  When I arrived on-site I got the same feeling from Rob Lee although his concerned seemed to be more centered around the value added by the course to more experienced incident response professionals.  Well, although their concerns were valid, I have to say that attending this class was a very valuable experience from the networking I accomplished, to the new (to me) concepts about how file systems work, to the concerns about how some applications leverage that information to produce system artifacts.

I am not going to delve into too much about the topics covered in the class.  It is outlined for you on SANS’ website and, well, Rob and his crew worked very hard on pulling all of the concepts together.  For that you should attended the course or purchase the course material if you would like a deeper understanding.  However, there are a bunch of priceless illustrations that help the students understand some of the complex topics that can be confusing when first exposed to the information.  The ones that popped out to me the most were the images covering the Forensic Investigation Methodology, Date/Time correlations, and the Filesystem/Sleuthkit Review.  Each of them, at a glance provides excellent clarification.

So, to alleviate the concerns of Harlan and Rob, I learned a lot by attending the course.  I guess I am just one of those people.  I will admit that I was not as challenged as when I attended some of the other SANS trainings I have attended.  Certainly it was a fire hose for most of the attendees.    I just figure that this means that for the past two years I have been approaching incident response properly.  Rob’s class most validated the processes I have formed surrounding my acquisition and initial analysis techniques.    A lot of this came to me via Harlan’s training both professionally, individually, and via reading his blog and books.  Some of these concepts were also developed via the experiences of Chris Pogue.  His Sniper Forensics talk is a direct representation of many of the concepts I employ.  (It was good to finally meet him after two years.)

Since I am not going to go too deep into the concepts covered by the class (although they should shape some of the future content here) I will provide you with some of the notable quotes that came from Rob.

“…training the new breed of incident responder.”

Absolutely, SEC508 provides a sound foundation.  It exposes incident responders to the basics of the field.  Starting with a sound foundation is what is necessary.   (Tangent Alert!)  It also takes incident response and digital forensics out of the court room and back into the data center.  Which is important because the data center changes much faster than the court room.  By letting the court room lead our incident response processes we are limiting our capabilities to adapt to new threats and attack methodologies.  Let the court room keep up with us.

“…EMTs do not worry about adjusting evidence …”

Another statement enforcing the point I just made.  Of course, what should be noted is that EMTs approach an incident with a specific methodology.  They have a plan and they execute it.  When necessary, they deviate from that plan.  But familiarization and continuous training around the basics of that plan make it second nature to them.  This means that their actions can be accounted for and justified when evidence is necessary.

“Evidence integrity goes to the weight of what the evidence can be used for….”

Basically, be more concerned about the actions you have taken to gather information.  Once again, following your plan, knowing the basics, and documenting deviations.  Just because there is or is not a hash does not mean that, if necessary, the information will not be admissible during a court case.  But court cases should not be your major concern.  Consistent and repeatable process should be your concern.  This is necessary in case there is a need to repeat the data analysis in a court room, for a Board of Directors, or for a team of auditors.

“Tools do not have to be validated.  The output, what was found, is more important than the tool that was used to interpret the data.”

This is one of the first concepts that Harlan explained to me when I started working with him.  Different tools display information better than other tools (which is why we have a wide variety of them).  But just because a tools presents the data in a certain way, or has been doing so for X number of years, does not mean it is doing so correctly.  Other methods may be necessary to validate tool output.  This concept holds true for a perl/python script that was written last night by a kid in Poughkeepsie, NY or a long standing data analysis tool such as EnCase or FTK.

The forensic industry “is not a fad.  Organizations are spinning up internal teams to handle incident response and investigations.”

This is nothing new but it is a great validation.  Rob is exposed to a wide range of people from many different operational backgrounds.  This statement is also supported by the explosion of process and tool development in the digital forensic and incident response field.

I will end with a personal favorite of mine.  The following quote validates a realization I recently came to while cleaning up after an incident response.  If you have a weak heart, and hold onto old concepts  dearly, you may want to skip the following quote.  (I am paraphrasing because I just realized I didn’t write it down.)

“How many passes does it take to destroy data so that  forensic analysis tools cannot recover it?  One, yes you are correct.”

Yes, you read that right.  Only one pass is necessary.  Wow, that will save a lot of time not to mention a lot of energy related to processor intensive multiple writes using random data.  I am not going to track down all of the links that support this statement.  Basically, once information has been overwritten it cannot be accessed by the tools we typcially deploy.  Even advanced tools can only guess at the former state of a bit.  The cool thing is that since there are multiple layers to the file systems, there is a chance that a tool or process did not correctly overwrite the information.  This is a key concept covered by SEC508.  And as incident responders we also realize that just because data was destroyed in one location that it is not stored in some other location.  Which is why our processes include involving an organization’s network, workstation, server, and application administrators as well as management.  These people will understand where residual data resides within the organization.

So, to wrap this up, I highly recommend SEC508 to new and experienced incident response and digital forensic professionals.  You are going to learn something you did not know.  You are going to make contacts that will be invaluable in the future.  And, if you obtain the GIAC certification, you are going to have a valuable certification in a growing and increasingly important field that is having global impact.

Go forth and do good things,

Don C. Weber


Help support my training and travel to security conferences. Get your SANS Training and GIAC Certifications through the Security Ripcord.


ITB Issue 0×1 – Call For Collaboration

February 7th, 2010 cutaway Posted in Incident Response, Into The Boxes, forensics | No Comments » 2,023 views

The success of Into The Boxes Issue 0×0 was only possible because of the collaboration provided by members of the Digital Forensics and Incident Response community.  In order for this publication to continue we need more people to step up and provide their input.  As you can see from the first issue we are looking for input that can be implemented by people in the DF/IR fields.  This input can be in the form of detailed articles or quick tips.  All input will be given serious consideration.  The ITB editors will provide authors with recommendations to strengthen their write-ups to ensure the best value to the community and help the authors develop as DF/IR professionals and writers.

Please help ITB by providing your submissions or letting us know about your intent to submit via the ITB Call Box.  We are also looking for article recommendations which we will place in the ITB Research Box so that others have good ideas as to what will help the DF/IR Community.  Obviously, if you would like to contribute but do not know what to write about, check out the ITB Research Box for recommendations.

Go forth and do good things,

Don C. Weber


Help support my training and travel to security conferences. Get your SANS Training and GIAC Certifications through the Security Ripcord.


APT-style Manual Compromise Viewed Via Timeline Analysis

February 5th, 2010 cutaway Posted in Incident Response, Security, forensics | No Comments » 2,145 views

Most of the time the initial infection vector associated with APT-style attacks incorporate the client-side exploitation of vulnerabilities in any number of software.  Actually, when dealing with APT-style events I prefer “initial compromise vector” (ICV) as APT backdoors should not be considered or even referred to as malware because it provides an incorrect understanding to the whole incident response.  User and system activity coupled with the fact that many organizations do not detect a compromise for several months means that the ICV can be elusive.  It is much more likely that the actions following the initial compromise are going to persist over time.

The following timeline information was pulled from a system.  A complete timeline was generated using system artifacts parsed using System Combo Timeline and several TLN-based Enscripts.  Each line is in the order display within the timeline.  no lines have been removed.  This was a fairly lucky situation as many times unrelated activity needs to be purged from the timeline to see this type of detail and understand specific activities.  Specific comments will follow the lines I want to highlight.

2009 11 04 08:59:42|SysEvent.Evt – EVT|COMP-SYS|S-1-5-18|Service Control Manager/7035;Info;LiveUpdate,start
2009 11 04 08:59:42|SysEvent.Evt – EVT|COMP-SYS|N/A|Service Control Manager/7036;Info;LiveUpdate,running
2009 11 04 09:00:06|AppEvent.Evt – EVT|COMP-SYS|N/A|Symantec AntiVirus/16;Info;Virus definitions are current.
2009 11 04 09:00:12|SysEvent.Evt – EVT|COMP-SYS|N/A|Service Control Manager/7036;Info;LiveUpdate,stopped

Very interesting that many of these events seem to occur during some type of AV updating or scanning activity.  Although really just an interesting side note, this does show that AV can only prevent what it is programmed to understand.  The following events do not fall into that category.

2009 11 04 09:19:09|EnCase File TLN|COMP-SYS|COMP-SYS\Disk Image\C\Documents and Settings\All Users\Application Data\Microsoft\Crypto\RSA\S-1-5-18|Change Time (ctime)
2009 11 04 09:19:09|EnCase File TLN|COMP-SYS|COMP-SYS\Disk Image\C\Documents and Settings\All Users\Application Data\Microsoft\Crypto\RSA\S-1-5-18|Modified Time (mtime)
2009 11 04 09:19:09|EnCase File TLN|COMP-SYS|COMP-SYS\Disk Image\C\Documents and Settings\All Users\Application Data\Microsoft\Crypto\RSA\S-1-5-18\d42cc0c3858a58db2db37658219e6400_dbcab107-5342-4908-9e95-0aba2546f18b|Accessed Time (atime)
2009 11 04 09:19:09|EnCase File TLN|COMP-SYS|COMP-SYS\Disk Image\C\Documents and Settings\All Users\Application Data\Microsoft\Crypto\RSA\S-1-5-18\d42cc0c3858a58db2db37658219e6400_dbcab107-5342-4908-9e95-0aba2546f18b|Change Time (ctime)
2009 11 04 09:19:09|EnCase File TLN|COMP-SYS|COMP-SYS\Disk Image\C\Documents and Settings\All Users\Application Data\Microsoft\Crypto\RSA\S-1-5-18\d42cc0c3858a58db2db37658219e6400_dbcab107-5342-4908-9e95-0aba2546f18b|Created Time (crtime)

These files appear to be associated with Microsoft Cryptographic Service Providers.  I am still researching what this actually implies.  Please post a comment if you can provide some insight.

2009 11 04 09:21:00|SysEvent.Evt – EVT|COMP-SYS|S-1-5-18|Service Control Manager/7035;Info;UpsGSx,start
2009 11 04 09:21:00|SysEvent.Evt – EVT|COMP-SYS|N/A|Service Control Manager/7036;Info;UpsGSx,running
2009 11 04 09:21:00|SysEvent.Evt – EVT|COMP-SYS|S-1-5-18|Service Control Manager/7035;Info;UpsGSx,stop

This service is a malicious backdoor that has been related to Hydraq. Notice that new services do get logged to Windows Event Logs but only if your systems are configured to log.  But why did this service start?

2009 11 04 09:21:00|EnCase File TLN|COMP-SYS|COMP-SYS\Disk Image\C\WINDOWS\Tasks|Change Time (ctime)
2009 11 04 09:21:00|EnCase File TLN|COMP-SYS|COMP-SYS\Disk Image\C\WINDOWS\Tasks|Modified Time (mtime)

Ah, a modification to the Scheduled Tasks folder.  This could be an indication that a job was scheduled to run.

2009 11 04 09:21:00|Registry Hive: software|COMP-SYS|0|$$$PROTO.HIV/Microsoft/SchedulingAgent
2009 11 04 09:21:00|Registry Hive: system|COMP-SYS|0|$$$PROTO.HIV/ControlSet001/Enum/Root/LEGACY_UPSGSX/0000
2009 11 04 09:21:00|Registry Hive: system|COMP-SYS|0|$$$PROTO.HIV/ControlSet002/Enum/Root/LEGACY_UPSGSX/0000

More activity associated with the UPSxxx backdoor service starting.

2009 11 04 09:21:00|Task Log|COMP-SYS|”At1.job” (a.exe)    Started 11/4/2009 5:21:00 PM|0
2009 11 04 09:21:00|Task Log|COMP-SYS|”At1.job” (a.exe)    Finished 11/4/2009 5:21:00 PM    Result: The task completed with an exit code of (0).|0

Well, now, what do we have here?  A scheduled task running a strange executable.  And, at the exact same time that the UPSxxx backdoor service started.  Nobody said that timeline information will be presented down to the millisecond.  Obviously this action occurred before some of the other events.  Had the conversion from local time to UTC been off this data might have been lost and not associated with these events.

2009 11 04 09:31:58|SysEvent.Evt – EVT|COMP-SYS|N/A|Tcpip/4226;Warn;

Once, Harlan Carvey recommended that I get a subscription to EventID.net.  Here is where that subscription pays off.  The description for Event ID 4226 reads “TCP/IP has reached the security limit imposed on the number of concurrent TCP connect attempts.”  Well, now, it seems that there is a large number of outgoing connection attempts.  Additional comments from other users of EventID.net provide indications that this type of activity can be linked to malware.  And to think, this was alerted in the Windows Event Logs no less.  Did I mention that these should be turned on?  Starting to seem very useful right about now.

2009 11 04 09:35:30|EnCase File TLN|COMP-SYS|COMP-SYS\Disk Image\C\WINDOWS\system32\snmpapi.dll|Accessed Time (atime)
2009 11 04 09:35:30|EnCase File TLN|COMP-SYS|COMP-SYS\Disk Image\C\WINDOWS\system32\inetmib1.dll|Accessed Time (atime)
2009 11 04 09:35:30|EnCase File TLN|COMP-SYS|COMP-SYS\Disk Image\C\WINDOWS\system32\netstat.exe|Accessed Time (atime)
2009 11 04 09:36:53|EnCase File TLN|COMP-SYS|COMP-SYS\Disk Image\C\WINDOWS\system32\wbem\wbemperf.dll|Accessed Time (atime)
2009 11 04 09:36:53|EnCase File TLN|COMP-SYS|COMP-SYS\Disk Image\C\WINDOWS\system32\systeminfo.exe|Accessed Time (atime)
2009 11 04 09:36:55|EnCase File TLN|COMP-SYS|COMP-SYS\Disk Image\C\WINDOWS\system32\wbem\stdprov.dll|Accessed Time (atime)
2009 11 04 09:41:17|EnCase File TLN|COMP-SYS|COMP-SYS\Disk Image\C\WINDOWS\system32\tasklist.exe|Accessed Time (atime)
2009 11 04 09:43:27|EnCase File TLN|COMP-SYS|COMP-SYS\Disk Image\C\WINDOWS\system32\nbtstat.exe|Accessed Time (atime)
2009 11 04 09:46:58|EnCase File TLN|COMP-SYS|COMP-SYS\Disk Image\C\RECYCLER|Change Time (ctime)
2009 11 04 09:46:58|EnCase File TLN|COMP-SYS|COMP-SYS\Disk Image\C\RECYCLER|Modified Time (mtime)

Okay, here is the hot and juicy stuff.  Wow, look at those Access times.  Access times on executables are indicative of the execution of these programs.  “Netstat”, “Systeminfo”, “tasklist”, and “nbtstat”.  All one after another? Very interesting.  The accessed DLLs also show us that other programs might have been run around this time.  SNMP?  Simple Network Management Protocol – a little network management anyone? WBEM? Web-Based Enterprise Management.  Definitely more reconnaissance behavior.

I would also like to point out that this activity appears to have occurred approximately ten minutes after the backdoor was placed on this system using the scheduled task.  So, following logic, this system was compromised from another system within the network.  The attacker completed the working on that first system (possibly similar reconnaissance efforts) and then connected to this system via the backdoor.  Why the backdoor?  Well, logging is turned on.  There is no network logon event.  This could very well mean that the user’s actual credentials have not been compromised, YET.  Or, this could mean the attacker did not want to generate a logon event, or it could mean that he required some functionality provided by the backdoor.  We are starting to read into this a little too much at this point.  But our initial thoughts, compromised from another system on the internal network and performing reconnaissance steps, are still sound.

2009 11 04 10:12:46|EnCase File TLN|COMP-SYS|COMP-SYS\Disk Image\C\System Volume Information\_restore{01E266C2-86F5-40B2-9145-B4252FFF29C3}\RP34\A0034337.properties|Accessed Time (atime)
2009 11 04 10:12:46|EnCase File TLN|COMP-SYS|COMP-SYS\Disk Image\C\System Volume Information\_restore{01E266C2-86F5-40B2-9145-B4252FFF29C3}\RP34\A0034337.properties|Change Time (ctime)
2009 11 04 10:12:46|EnCase File TLN|COMP-SYS|COMP-SYS\Disk Image\C\System Volume Information\_restore{01E266C2-86F5-40B2-9145-B4252FFF29C3}\RP34\A0034337.properties|Modified Time (mtime)
2009 11 04 10:13:10|EnCase File TLN|COMP-SYS|COMP-SYS\Disk Image\C\Documents and Settings\Administrator\Local Settings\Temp\vmware-user\vmware-3856-mks-user-476.log|Accessed Time (atime)
2009 11 04 10:13:10|EnCase File TLN|COMP-SYS|COMP-SYS\Disk Image\C\Documents and Settings\Administrator\Local Settings\Temp\vmware-user\vmware-3856-mks-user-476.log|Change Time (ctime)
2009 11 04 10:13:35|EnCase File TLN|COMP-SYS|COMP-SYS\Disk Image\D\System Volume Information\_restore{01E266C2-86F5-40B2-9145-B4252FFF29C3}\RP33|Created Time (crtime)
2009 11 04 10:13:35|EnCase File TLN|COMP-SYS|COMP-SYS\Disk Image\D\System Volume Information\_restore{01E266C2-86F5-40B2-9145-B4252FFF29C3}\RP33\change.log.1|Created Time (crtime)
2009 11 04 10:13:37|EnCase File TLN|COMP-SYS|COMP-SYS\Disk Image\D\System Volume Information\_restore{01E266C2-86F5-40B2-9145-B4252FFF29C3}\RP34\A0034288.dll|Change Time (ctime)
2009 11 04 10:13:37|EnCase File TLN|COMP-SYS|COMP-SYS\Disk Image\D\System Volume Information\_restore{01E266C2-86F5-40B2-9145-B4252FFF29C3}\RP34\A0034288.dll|Created Time (crtime)
2009 11 04 10:13:37|EnCase File TLN|COMP-SYS|COMP-SYS\Disk Image\D\System Volume Information\_restore{01E266C2-86F5-40B2-9145-B4252FFF29C3}\RP34\A0034287.dll|Change Time (ctime)

Ah, the System Restore Point.  This simple occurrence could be very helpful indeed.  It is quite possible that the Upsxxx backdoor was captured and stored in this System Restore Point.  This is very useful information especially if the malware has rolled over to a new version.  If your security team is looking for a different version of the backdoor, systems compromised with the version captured in this restore point may still be beaconing or allowing access to your environment.  Time to dive into these System Restore Points and see what they yield.

Hopefully you have found this helpful.

Go forth and do good things,

Don C. Weber

Web-Based Enterprise ManagementWeb-Based Enterprise Management

Help support my training and travel to security conferences. Get your SANS Training and GIAC Certifications through the Security Ripcord.


Hydraq Details Revealed Via Timeline Analysis

February 5th, 2010 cutaway Posted in Incident Response, Security, forensics | No Comments » 2,157 views

The other day I was handed a system that was known to be compromised with Hydraq.  The goals were to determine when, how, and what happened after the compromise.  Locating the malicious process during memory analysis was easy with so many known system artifacts.  Not really very useful although it did determine that the rasmon.dll program did contain a domain name unique to the rest of the intelligence from the Internet.

I follow a very specific initial process when appoaching compromises on Microsoft Windows systems.  Keyword searches, timeline generation, and registry analysis top my list unless inidcators point me in a different direction.  Keyword searches on unallocated space and the systems pagefile did not provide any positive information related to the goals.   So I moved onto identifying the backdoor’s service to have a little more information to use during timeline analysis.  Easy enough with a quick review of the Service Registry keys.

System\ControlSet001\Services\RaSuPoo – Date Modified: 1/12/2010 11:05:40 AM
Description – ** NONE PROVIDED **
DisplayName – ** NONE PROVIDED **
ImagePath – %SystemRoot%\System32\svchost.exe -k netsvcs
Parameters – Date Modified: 1/12/2010 11:05:40 AM
ServiceDll – c:\windows\system32\rasmon.dll
Security – Date Modified: 1/12/2010 11:05:40 AM

Right off the bat I noticed a few things that were interesting.  Neither the “Description” nor the “DisplayName” key values were present.  Although they are listed here it is only to note that they were missing completely.  Now these things might be unique to this system but they are definitely indicators that can be used to identify other compromised systems with in the environment, possibly even if the backdoor is rolled over to a different version by the attackers.

Something else that is important to note is the matching “Last Write” for the primary registry key and the subkeys “Parameters” and “Security”.  When these times do not match I rely on the times provided by the “Security” key to determine the likely date and time that the system was compromised.  As these times matched for this service I was initially inclined to utilize these times as indications of the initial compromise.

Additional review of the services using intelligence from the Internet located other malicious services.  The following Registry information represents one of these malicious services.

System\ControlSet001\Services\AppMgmt – Date Modified: 1/12/2010 11:05:40 AM
Description – Provides software installation services such as Assign, Publish, and Remove.
DisplayName – Application Management
ImagePath – %SystemRoot%\System32\svchost.exe -k netsvcs
Parameters – Date Modified: 10/14/2009 01:25:37 AM
ServiceDll – C:\Documents and Settings\Administrator\AppMgmt.dll
ServiceDllUnloadOnStop – 00000001
Security – Date Modified: 4/4/2005 12:24:16 PM

There are several interesting things to note about this process.  First of all is the location of the AppMgmt.dll.  This location should flag this service as malicious during a manual review of running services as well as when this service is running in memory.  The “Last Write” time of the “Parameter” key is a good indication of the date and time the system was compromised by this backdoor.  After additional review this seems to be the earliest indication of compromise.

Continuing with timeline analysis produced an interesting finding from the event logs.

2010 01 12 11:05:40|SysEvent.Evt – EVT|INF-SYS|S-1-5-18|Service Control Manager/7035;Info;RaShniI,stop

As you can see, the backdoor service with a different random key name than the current registry setting has stopped.  Now, this could be typical behavior associated with the functionality of the backdoor or it could be indications of another malware rollover to a new version.  However, now we have additional indicators for the timeline analysis.  A little fancy searching produced sixteen other instances of randomly named services that stopped and produced an Event Log entry.  Timeline analysis also produced several instances of systems restarting as represented by the “Task Scheduler Service” starting.

2010 01 11 11:40:09|Task Log|INF-SYS|”Task Scheduler Service”   Started at 1/11/2010 9:40:09 AM|0

Continued review provided an even more interesting finding.  Instances of a LEGACY registry key for each “stopped” service associated with the backdoor.

2009 10 29 05:48:46|Registry Hive: system|INF-SYS|0|$$$PROTO.HIV/ControlSet001/Enum/Root/LEGACY_RAS0HWK/0000

Additional investigation of the System Registry Hive produced the following information.

System\ControlSet001\Enum\Root\LEGACY_RAS0HWK – Date Modified: 10/27/2009 5:18:46 AM
0000 – Date Modified: 10/27/2009 5:18:46 AM
DeviceDesc – RaS0HWk
Service – RaS0HWk

Then, after trimming the fat from timeline entries the following details emerged.  I have trimmed these a bit for brevity.

2009 10 27 05:18:59|Task Log|INF-SYS|”Task Scheduler Service”   Started at 10/27/2009 13:18:59 PM|0
2009 10 27 05:18:46|Registry Hive: system|INF-SYS|0|$$$PROTO.HIV/ControlSet001/Enum/Root/LEGACY_RAS0HWK/0000
2009 10 27 05:19:17|SysEvent.Evt – EVT|INF-SYS|S-1-5-18|Service Control Manager/7035;Info;RaS0HWk,stop
2009 10 30 09:10:55|Task Log|INF-SYS|”Task Scheduler Service”   Started at 10/30/2009 17:10:55 PM|0
2009 10 30 09:10:43|Registry Hive: system|INF-SYS|0|$$$PROTO.HIV/ControlSet001/Enum/Root/LEGACY_RAS1DBH/0000
2009 10 30 09:11:19|SysEvent.Evt – EVT|INF-SYS|S-1-5-18|Service Control Manager/7035;Info;RaS1DbH,stop
[snip]
2010 01 11 01:10:49|Task Log|INF-SYS|”Task Scheduler Service”   Started at 1/11/2010 9:10:09 AM|0
2010 01 11 01:10:44|Registry Hive: system|INF-SYS|0|$$$PROTO.HIV/ControlSet001/Enum/Root/LEGACY_RASX4NS/0000
2010 01 11 01:10:01|SysEvent.Evt – EVT|INF-SYS|S-1-5-18|Service Control Manager/7035;Info;RaSX4nS,stop
2010 01 12 01:55:02|Registry Hive: system|INF-SYS|0|$$$PROTO.HIV/ControlSet001/Enum/Root/LEGACY_RASHNII/0000
2010 01 12 01:55:18|SysEvent.Evt – EVT|INF-SYS|S-1-5-18|Service Control Manager/7035;Info;RaShniI,stop

Now, this backdoor behavior is significant because I have not found any reference to this restart behavior in any of the Hydraq malware analysis reports.  It also means that the Last Write times for the “RaSuPoo” service registry key is not indicative of the time that the AppMgmt rolled over to the new backdoor.  Whether the restart behavior was intentional or not this behavior has successfully obfuscated some critical system information that may not have been apparent had it not been for timeline analysis.

Additional analysis of timeline data using the LEGACY data also revealed another significant finding.  Malware reports related to Hydraq provide mention of a backdoor service with the name “Upsxxx” where [xxx] are random.  Timeline review revealed a LEGACY key for this service.

System\ControlSet001\Enum\Root\LEGACY_UPSYVM – Date Modified: 10/27/2009 4:07:09 AM
0000 – Date Modified: 10/27/2009 4:07:09 AM
DeviceDesc – UpsyVm
Service – UpsyVm

The significance of this registry key is not the fact that it appears to have been created before the earliest RaSxxxx service keys.  Rather it is the fact that there are no Registry entries pertaining to the initial Upsyvm service keys.  This means that the original service keys were deleted once this backdoor was rolled over to the new backdoor.  Another significant finding revealed during timeline analysis.

Now, none of these indicators provide details about the “smoking gun” – manual activity – that has taken place on this system.  However, understanding all of these activities provides a complete understanding of the series of events that could lead up to manual malicious activity on other systems.  Details of system artifacts and backdoor activity times will help identify other compromised systems and other activities.  This information also provides new methods that are being used to obfuscate data and hide critical details about events following a compromise.  Hopefully the information provided here helps you overcome some of these techniques.

Go forth and do good things,

Don C. Weber


Help support my training and travel to security conferences. Get your SANS Training and GIAC Certifications through the Security Ripcord.


PreFetch EnScript and SysComboTLN Update

February 1st, 2010 cutaway Posted in Incident Response, Security | No Comments » 1,807 views

System Combo Timeline has been updated.  If you use syscombotln you will want to get this new version as there is an important bug fix.  I have also updated regtln.pl and evtparse.pl to handle double-byte and non-printable characters better.  This helps when analyzing log and registry files from Windows systems with various language packages.  Of course I do so my just stripping out the unwanted characters from the data entries.  You do not loose this information as you always have the original files.  But if you want/need to review the original entries you will need to review the original files.  These two tools are also available individually on the Scripts and Tools page.

PrefetchFolderAnalysis2.EnScript is a new addition to the Scripts and Tools page.  This EnScript was originally developed by Kelcey Tietjen.  I have updated it so that it outputs in TLN format and writes a file to the Export directory configured in EnCase.  It uses the last run time to place the entry in the timeline at the last time the file was executed.  If you integrate the results of this EnScript with the system’s File Directory listing you will have this included with MAC times.  Special attention, however, should be paid to the data entry.  The data will contain the original path of the executed command and the number of times that it was executed.  Extremely helpful during analysis.

If you would like to integrate the output from PrefetchFolderAnalysis2.EnScript into your timelines generated from syscombotln just place the resulting file into the “output” directory with any other TLN formated files you have created.  Running the syscombotln script will concatenate them all together.

Go forth and do good things,

Don C. Weber


Help support my training and travel to security conferences. Get your SANS Training and GIAC Certifications through the Security Ripcord.


Using Logs To Reduce Response Gap

January 29th, 2010 cutaway Posted in Firewalls, Incident Response, Security, forensics | 1 Comment » 2,774 views

One of the keys to incident response is to reduce the gap between compromise and when an organization starts taking action.  There are a number of tricks to identify compromised hosts.  Most organizations retain all types of logs for any number of reasons.  Unfortunately, auditing and never really using logs for anything except for records retention can cause organizations to treat them as merely objects to move around and not necessarily utilize for any action.  Even organizations who perform monitoring and effective records retention have a tendency to store and forget about older logs.

Access to old logs becomes painfully necessary during an incident response.  Without a well thought out and tested access plan, review of older logs can be a tedious and time consuming effort.  This can be further complicated by resources such as server load caused by searching and network connectivity caused by transfer.  I have experienced a few of these problems recently and I thought I would share them so that you can use them to prepare to reduce the response gap within your organization.

Firewall logs get large quickly.   The size of the organization is usually going to determine the length of time logs will/can be retained.  How long does it take you to search all of your firewalls back six months?  This becomes very important when firewall management and log storage have been outsourced to another organization.  What are your service level agreements for request response times?  Have you tested this response time?

Firewall log review will help you identifying internal IP addresses that connection to external IP addresses.  Now what do you do?  Well, if your environment is made up of static IP addresses this works just fine.  What happens if your organization utilizes DHCP for connectivity?  How long do you retain logging information for your DHCP leases?  How fast does it take to correlate firewall search results with the DHCP leases?  If you cannot answer these questions then additional testing will be necessary.

Monitoring for specific IP addresses is one thing, but often the information you have may be related to specific domains.  This means that DNS request logs become very important.  All of the same review questions apply.  Concerns about outsourced DNS apply.  Testing applies.

Time for a little math.  Let’s say you have an IP address of a known malicious server on the Internet.  If it takes you two days to search your firewall logs for connections and then it takes you an additional two days to correlate that information with DHCP logs, your incident response gap is four days.  That is four days before you can even begin applying your incident response process.  Four days before you can start requesting legal permission to adhere to law and regulations where the system resides.  Does it take two days for legal approval?  Now you are up to six days.  Do you have data analysis personnel that can respond to all of your locations?  Add two more days to mail systems and data analysis doesn’t begin for eight days.

Hopefully this helps.  Please test your computer incident response plan.  When you do, think outside of the box.  Ask additional questions.  Request information and time how long it takes.  Conduct lessons learned followed up with specific and managed goals.

Go forth and do good things,

Don C. Weber


Help support my training and travel to security conferences. Get your SANS Training and GIAC Certifications through the Security Ripcord.


Syscombotln and Tools Update

January 5th, 2010 cutaway Posted in Incident Response, Malware, Security, forensics | 3 Comments » 2,815 views

System Combo Timeline:

The syscombotln tool has been updated to fix several bugs and time/date issues.  I have also decided to stop being lazy and updated all of the internal modules and external scripts/tools associated with this tool to properly handle the TLN format as Harlan outlined. This includes the TLN.EnScript which is NOT included in the syscombotln tool.

New functionality includes parsing the Windows XP setupapi.log file.  I have included this functionality due to a little analysis trick pointed out to me by Jason Luttgens and Jon Gross of Mandiant.  Basically any time a Windows Help (chm) file is executed the information is logged in the user’s HTML Help (hh.dat) file.  This information can be used to specify some Initial Infection Vector information.  This information, in turn, may be augmented by driver-based information which, in Windows XP, is logged in the setupapi.log file.  I cannot provide specifics at this time, but I can tell you that you will know suspicious entries when you see them.  (If anybody has specific examples, please provide them in the comments.)  Although I have not had time to parse the hh.dat file, I have had time to parse the setupapi.log file.  The syscombotln module for this file is very basic but it should handle all files well (please let me know if you experience cases where it does not).  An added benefit of parsing this log file is that external USB storage device installation information will also be added to your timelines.  And if there are anti-forensic efforts recommending deleting this log, you know we want to review it’s information and add it to our timelines.

IronGeek Resource:

Just a quick note.  When researching the information in the last post I ran across this great resource by IronGeek.  Once again he has posted some amazing content.  Take a look at his “Forensically interesting spots in the Windows 7, Vista and XP file system and registry” resource.  You might have known about this, but it is new to me, so just in case you missed it as well.

Scripts and Tools:

I have decided to start uploading my scripts and tools when I generate an update.  To this end I have created the Scripts and Tools page which includes some Window Registry tools (including some older RegRipper plugins) and a few Enscripts.  Check this page often for updates and new scripts/tools.  Leave comments with comments, updates, requests.  To help with consistency, I have also started using Subversion to help me track development of all my projects.  Basically because I have been brow-beating (unsuccessfully) Harlan to do the same with his tools.  I have started keeping all my projects on an external USB drive (which I backup often).  To keep each project separate I use the following steps.

  1. Copy folder to Projects directory.
  2. Type “svnadmin create /media/<usb drive>/Dev/Projects/Repo/<project name>”
  3. Type “svn import <project name> file:///media/<usb drive>/Dev/Projects/Repo/<project name> -m “Initial Import”
  4. Move original project directory “mv <project name> <project name>_bk”
  5. Check out repository “svn checkout file:///media/<usb drive>/Dev/Projects/Repo/<project name> <project name>”
  6. Double check files are there and work with it a little while.
  7. Delete _bk

This works across Linux systems and should work on Windows systems using something like tortoisesvn.  Hopefully you find that useful for your script and tool development.


Help support my training and travel to security conferences. Get your SANS Training and GIAC Certifications through the Security Ripcord.


System Combo Timeline Released

November 30th, 2009 cutaway Posted in Incident Response, Security | 3 Comments » 2,251 views

Tired of doing a lot of work by hand I have started a project to quickly generate a timeline file from system artifacts recovered from systems running a Windows operating system.  The goal is to quickly generate information that can be used to determine actionable intelligence during an incident response.  System Combo Timeline is a set of scripts that will internally generate or use external tools (thank you, Harlan) to generate TLN-based timeline files from specific system artifacts.  These files are combined into one TLN-based file with can be reviewed with the hope of understanding some of the actions that occurred on the system.  Information taken from this TLN combo file can be used to direct the rest of the analysis or even provide information to help guide future incident response activities.

Hopefully you find this tool useful. There is still a lot of work to do to pull in additional capabilities and make the tool more flexible.  As always feedback and requests for features are always welcome.  Leave a comment and let everyone know what you think about the tool.

Go forth and do good things,

Don C. Weber


Help support my training and travel to security conferences. Get your SANS Training and GIAC Certifications through the Security Ripcord.


Bodyfile and Timeline EnScripts

November 29th, 2009 cutaway Posted in Incident Response, Security | No Comments » 1,881 views

Being able to utilize multiple tools is key for any digital forensic and incident response analyst.  However, moving back and forth between different operating systems or starting and stopping memory intensive tools can have an impact on quickly exporting critical information from a system.  The FLS utility provided in the Sleuth Kit tools produces an excellent bodyfile and is, in my mind, the default standard for generating timeline information.  However, if you are working in EnCase, it would be nice to have a method to quickly output a bodyfile text file that can be quickly integrated with other timeline information for a more in-depth and detailed analysis.

After a bit of trial and error working with EnScripts I decided to contact Lance Mueller to get over a few EnScript hurdles.  Lance quickly updated my initial code and helped me understand how to gather file information and output the information to a text file.   After a few more tweaks I now have a working BodyFile EnScript.  The only real difference between the bodyfile generated by this EnScript and the FLS bodyfile is that the EnScript file times are provided in a human-readable format whereas FLS outputs in Epoch time and needs to be converted prior to evaluation.

Although this was very useful when generating timelines from multiple sources, I often found myself modifying the bodyfile to match the Timeline (TLN) format outlined by Harlan Carvey from his blog post:  Timline Analysis, pt III.

Time - MS systems use 64-bit FILETIME objects in many cases; however, for the purposes of normalization, 32-bit Unix epoch times will work just fine

Source - fixed-length field for the source of the data (i.e., file system, Registry, EVT/EVTX file, AV or application log file, etc.) and may require a key or legend. For graphical representation, each source can be associated with a color.

Host - The host system, defined by IP or MAC address, NetBIOS or DNS name, etc. (may also require a key or legend)

User - User, defined by user name, SID, email address, IM screenname, etc. (may also require a key or legend)

Description - The description of what happened; this is where context comes in…

I have to admit that I don’t always follow this format to the letter for one reason or another.  I often use the last two fields as open fields for specific information about the artifact that I am documenting.  Where I can I provide “User” information, but remembering to be flexible allows me to include valuable information, although, it would also be just as easy to tack it on the end as well.

After a few times of converting the EnScript bodyfile to TLN format I decided it was just better to have an EnScript that output into TLN format.  Thus I now have the TLNFile EnScript which can easily be integrated with other TLN formatted files to create excellent timelines.

Go forth and do good things,

Don C. Weber


Help support my training and travel to security conferences. Get your SANS Training and GIAC Certifications through the Security Ripcord.