Monday, December 31, 2007

Sguil Status

One of you wrote recently to ask about the status of the open source Network Security Monitoring suite called Sguil. You noticed the last release of Sguil (0.6.1) occurred in February 2006. I can assure you Sguil is not dead. In fact, just last week I wrote an article for a new BSD magazine about installing the sensor and server components of Sguil 0.7.0 (from CVS on FreeBSD 7.0.

To keep up with development read the sguil-devel mailing list and visit #snort-gui on irc.freenode.net.

I expect to see Sguil 0.7.0 released before 13 February 2008 to avoid hitting the two year mark.

Last Book Reviews of 2007 Posted

Amazon.com just published my five star review of Ajax Security by Billy Hoffman and Bryan Sullivan. From the review:

Ajax Security was the last book I read and reviewed in 2007. However, it was the best book I read all year. The book is absolutely compelling and every security professional and Web developer should read it. It's really as simple as that.

I am not a Web developer. I was not very familiar with Ajax (beyond its buzzword status and a vague notion of functionality) when I started reading Ajax Security. I attended the authors' Black Hat 2007 talk and was thoroughly impressed and disturbed by the security implications they presented. I expected Ajax Security to be a good book, but one can never be sure if talented hackers and presenters can transfer their skills to the written word. Ajax Security gets the job done.


Ajax Security is my Best Book Bejtlich Read in 2007 award winner. Amazon.com will soon publish my four star review of Geekonomics by David Rice. From the review:

I really, really liked Geekonomics, and I think all security and even technology professionals should read it. Why not give the book five stars then? The reasons are twofold: 1) the book fails to adequately differentiate between safety and security; and 2) the chapter on open source demonstrates fundamental misconceptions that unfortunately detract from the author's message. If you are kind enough to keep the
thoughts in this review in mind when reading Geekonomics, you will find the book to be thoughtful and exceptionally helpful.

It is important to remember that Geekonomics is almost exclusively a vulnerability-centric book. Remember that the "risk equation" is usually stated as "risk = vulnerability X threat X impact". While it is silly to assign numbers to these factors, you can see that decreasing vulnerability while keeping threat and impact constant results in decreased risk. This is the author's thesis. Rice believes the governing issue in software security is the need to reduce vulnerability.

The problem with this approach is that life is vulnerability. It is simply too difficult to eliminate enough vulnerability in order to reduce risk in the real world. Most real world security is accomplished by reducing threats. In other words, the average citizen does not reduce the risk of being murdered by wearing an electrified, mechanized armor suit, thereby mitigating the vulnerability of his soft flesh and breakable neck. Instead, he relies on the country's legal system and police force to deter, investigate, apprehend, prosecute, and incarcerate threats.
Finally, Amazon.com published my three star review of The Book of Pf by Peter N.M. Hansteen. From the review:

I was excited to see a new book on Pf on the market. Three years ago I read and reviewed Building Firewalls with OpenBSD and PF (BFWOAP) by Jacek Artymiak and gave it five stars. I hoped The Book of Pf (TBOP) would acknowledge the best ideas in BFWOAP and expand into Pf developments of the last three years. TBOP is strong when it addresses how to install or use Pf on operating systems other than OpenBSD. Elsewhere, the book is too weak to merit more than three stars.

Hopefully by the time you read this all of the links will be working and the reviews will be posted.

Best Book Bejtlich Read in 2007

Last year I posted my first year-end ranking of books I had read and reviewed in 2006, titled Favorite Books I Read and Reviewed in 2006. I decided to continue the tradition this year by posting my 2007 rankings, and awarding Best Book Bejtlich Read in 2007 (B3R07).

2007 was not my most productive year in terms of reading and reviewing books. I read 17 in 2000, 42 in 2001, 24 in 2002, 33 in 2003, 33 in 2004, 26 in 2005, and 52 in 2006. This year I read and reviewed 25 books, several during the last week. My ratings can be summarized as follows:

  • 5 stars: 9 books

  • 4 stars: 11 books

  • 3 stars: 4 books

  • 2 stars: 1 book

  • 1 star: 0 books


The competition for the B3R07 award was intense. Keep in mind these are all five star books.

And, the winner of the Best Book Bejtlich Read in 2007 award is... 1. Ajax Security by Billy Hoffman and Bryan Sullivan (Addison-Wesley). Ajax Security was the last book I read and reviewed in 2007. However, it was the best book I read all year. The book is absolutely compelling and every security professional and Web developer should read it. It's really as simple as that.

If you'd like to read a very thorough and technically perceptive review of the book, I recommend this post by Dre: Ajax Security opens up a whole new can of worms.

Let me conclude by saying the competition for the top slot was very tight. I really loved all top five books, and the bottom four were excellent too. There are even some good four star books, but a book must rate five stars in order to be considered here.

Congratulations to No Starch for placing 4 books in my five star list. Addison-Wesley was the runner-up with 2 books, but the publisher also produced the B3R07 award winner.

Happy reading in 2008!

Thursday, December 27, 2007

Long Live Emerging Threats

If you haven't noticed, availability of Bleeding Threats has been lousy recently. If you read Matt Jonkman's recent post you'll notice the arrival of Emerging Threats. I am currently getting my copy of the Bleeding ruleset there; I am no longer using Bleeding Threats.

Friday, December 21, 2007

Snort Report 11 Posted

My 11th Snort Report on Snort Limitations has been posted. From the start of the article:

In the first Snort Report I mentioned a few things value-added resellers should keep in mind when deploying Snort:

1. Snort is not a "badness-ometer."
2. Snort is not "lightweight."
3. Snort is not just a "packet grepper."

In this edition of the Snort Report, I expand beyond those ideas, preparing you to use Snort by explaining how to think properly about its use. Instead of demonstrating technical capabilities, we'll consider what you can do with a network inspection and control system like Snort.


The editors titled this piece "Snort Limitations" -- I didn't.

Thursday, December 20, 2007

Predictions for 2008

For the last five years I've resisted the urge to write year-end predictions (thanks Anton). However, I'm seeing indications of the following, so maybe this is more about highlighting trends than taking wild guesses.

Here are my five predictions for 2008.

  1. Expect greater government involvement in assessing the security of private sector networks. I base this item on what's happening in the UK following their latest data breach. The article Data watchdog seeks dawn-raid powers states the following:

    The Information Commissioner’s Office (ICO), which polices the security of the nation’s data, is to be given the power to raid Government departments suspected of breaching protection laws.

    The move, announced today by Gordon Brown, comes in response to the loss by HM Revenue & Customs (HMRC) of personal details of some 25 million Britons. The Prime Minister said the ICO would be given extra powers to carry out “spot checks” of government departments.

    However, it is unclear whether the new powers will extend to companies - something that Richard Thomas, the Information Commissioner, is pressing for.

    "Alarm bells must ring in every boardroom," Mr Thomas said today.

    He added: "For some time I have been pressing the government to give my Office the power to audit and inspect organisations that process people’s personal information without first having to get their consent."

    Mr Thomas also repeated a call for the law to be "changed to make security breaches of this magnitude a criminal offence."
    (emphasis added)

    Security raids would be an amazing event. I think it would significantly alter the way security is managed by every major company.

  2. Expect greater military involvement in defending private sector networks. I base this item on reporting by the Baltimore Sun, no longer posted on their site but repeated elsewhere:

    In a major shift, the National Security Agency (NSA) is drawing up plans for a new domestic assignment: helping protect government and private communications networks from cyberattacks and infiltration by terrorists and hackers, according to current and former intelligence officials.

    From electricity grids to subways to nuclear power plants, the United States depends more than ever on Internet-based control systems that could be manipulated remotely in a terrorist attack, security specialists told The Baltimore Sun.

    The plan calls for the NSA to work with the Department of Homeland Security (DHS) and other federal agencies to monitor such networks to prevent unauthorized intrusion, according to those with knowledge of what is known internally as the "Cyber Initiative." Details of the project are highly classified.

    Director of National Intelligence Mike McConnell, a former NSA chief, is coordinating the initiative. It will be run by the DHS, which has primary responsibility for protecting domestic infrastructure, including the Internet, current and former officials said.

    At the outset, up to 2,000 people -- from the Department, the NSA and other agencies -- could be assigned to the initiative, said a senior intelligence official who spoke to The Baltimore Sun on condition of anonymity.


    I know nothing about this outside of what I just posted, and the story House panel chief demands details of cybersecurity plan discussing activities of the US House Committee on Homeland Security.

  3. Expect increased awareness of external threats and less emphasis on insider threats. Maybe this is just wishful thinking, but the recent attention on botnets, malware professionalization, organized criminal cyber enterprises, and the like seems to be helping direct some attention away from inside threats. This may be premature for 2008, but I expect to see more coverage of outsiders again.

  4. Expect greater attention paid to incident response and network forensics, and less on prevention. This could also be wishful thinking, but I am seeing a lot of movement in the commercial space involving effective incident response processes and tools. I've been speaking to several vendors while I build my IR and forensics lab for work and 2008 will see some very cool capabilities arrive, particularly in live response and remote forensic assessments. Several vendors will aggressively ship network forensic systems in 2008 with increased tie-ins to other existing products, like SIMs, firewalls, IPS, and the like.

  5. Expect talk of an "IPv6 gap," especially with respect to China. Leading up to the start of the Olympic Games in China in 2008, I am sure we will here a lot about IPv6. I mentioned this last year. Talk of an "IPv6 gap" will build upon a perceived "space gap" as China pursues its vision to put men on the moon by 2020. You will hear people say we need IPv6 because it is "inherently secure" or something similar. The China hacking stories of a few months ago embedded themselves in the IT consciousness, and that will be a continuing theme. I'm not sure if any of this will result in IPv6 being effectively deployed in 2008, 2009, or even 2010 in the US.


A year from now I'll see how these trends played out in 2008 and report back.

Two Book Reviews Posted

Amazon.com just published my five star review of Absolute FreeBSD, 2nd Ed by Michael Lucas. From the review:

Almost five years ago I reviewed Absolute BSD, Michael Lucas' first book on FreeBSD. I gave that book five stars, back when several other BSD books provided competition. On the eve of 2008, I am happy to say that Michael Lucas is probably the best system administration author I've read. I am amazed that he can communicate top-notch content with a sense of humor, while not offending the reader or sounding stupid. When was the last time you could physically feel yourself getting smarter while reading a book? If you are a beginning to average FreeBSD user, Absolute FreeBSD 2nd Ed (AF2E) will deliver that sensation in spades. Even more advanced users will find plenty to enjoy.

Amazon.com also just published my five star review of Linux Firewalls by Mike Rash. From the review:

Disclaimer: I wrote the foreword for this book, so obviously I am biased. However, I am not financially compensated for this book's success.

In the foreword I note that Linux Firewalls is a "great book." As a FreeBSD user, Linux Firewalls is good enough to make me consider using Linux in certain circumstances! Mike's book is exceptionally clear, organized, concise, and actionable. You should be able to read it and implement everything you find by following his examples. You will not only learn tools and techniques, but you will be able to appreciate Mike's keen defensive insights.


Are you seeing a trend here? In October I reviewed Security Data Visualization from No Starch and my Amazon.com Wish List has several other No Starch titles on it. Nice work No Starch!

Make Cleaning Awesome

Over three years ago I blogged about my Dyson vacuum cleaner. 99.9% of all of my posts are about digital security, but I know some of you are still looking for holiday presents for that certain someone. My wife bought me the new DC-16 for my birthday.

That's right, a vacuum for my birthday. Take a look at the picture of this thing and tell me it is not awesome. I dare you. Don't believe? Forget the perpetually clogged, nasty "filter" on my old Dustbuster. The DC-16 has a canister that I empty. The DC-16 also has a trigger, not a power button. It looks even more weaponized when the crevice tool is attached instead of the combination accessory tool (pictured above).

Don't let the crybaby Amazon.com reviewers dismay you. Sure, it would be nice to be able to have a second battery pack for swappable charging. However, if you're draining the battery regularly it's a sign you need to pull out your regular vacuum and not rely on a handheld. I've never drained the battery cleaning up after our kids.

I expect to see the DC-16 appear in homemade sci-fi videos on YouTube any time now.

Wednesday, December 19, 2007

After Five Years, NSM Is Still More Than IDS

I've received a series of questions relating to Network Security Monitoring (NSM) recently, via email, blog comments, IRC questions, and so on. Just over five years ago (2 Dec 02) Bamm Visscher and I recorded a Webcast for SearchSecurity.com titled Network Security Monitoring Is More Than IDS. That URL links to a series of questions submitted in response to the podcast.

I still have a copy of our slides, which I just exported to .pdf and uploaded as bejtlich_visscher_techtarget_webcast_4_dec_02.pdf. Remarkably, I would hardly change any of the content. All of the arguments we made back then still hold today. The only real changes involve replacing one or two defunct Web sites.

Anyone who is trying to understand NSM will enjoy this presentation. Please post questions here, and I will either answer the comments directly or save them for a follow-on blog post. Thank you.

Tuesday, December 18, 2007

Does Failure Sell?

I often find myself in situations trying to explain the value of Network Security Monitoring (NSM). This very short fictional conversation explains what I mean. This exchange did not happen but I like to contemplate these sorts of dialogues.

NSM Advocate: I recommend deploying network-based sensors to collect data using NSM principles. I will work with our internal business units to select network gateways most likely to yield significant traffic. I will build the sensors using open source software on commodity hardware, recycled from other projects if need be.

Manager: Why do we need this?

NSM Advocate: Do you believe all of your defensive measures are 100% effective?

Manager: No. (This indicates a smart manager. Answering Yes would result in a line of reasoning on why Prevention Eventually Fails.)

NSM Advocate: Do you want to know when your defensive measures fail?

Manager: Yes. (This also indicates a smart manager. Answering No would result in a line of reasoning on why ignorance is not bliss.)

NSM Advocate: NSM will tell us when we fail. NSM sensors are the highest impact, least cost way to obtain network situational awareness. NSM methodologies can guide and validate preventative measures, transform detection into an actionable process, and enable rapid, low-cost response.

Manager: Why can't I buy this?

NSM Advocate: Some mainstream vendors are realizing a market exists for this sort of data, and they are making some impact with new products. If we had the budget I might propose acquiring a commercial solution. For the moment I recommend pursuing the do-it-yourself approach, with transition to a commercial solution if funding and product capabilities materialize.

Manager: Go forth and let your sensors multiply.


Now you know that it's fiction.

Notice the crux of the argument is here: Do you believe all of your defensive measures are 100% effective? As a statement, one would say Because prevention eventually fails, you should have a means to identify intrusions and expedite remediation. A manager hearing that statement is likely to respond like this.

Manager: Do you mean to tell me that all of the money I've spent on firewalls, intrusion prevention systems, anti-virus, network access control, etc., is wasted?

NSM Advocate: That money is not wasted. It's narrowed the problem space, but it hasn't eliminated the problem.

This is a tough argument to accept. When I worked at Foundstone the company sold a vulnerability management product. Foundstone would say "buy our product and you will be secure!" I worked for the incident response team. We would say "...and when you still get owned, call us." Which aspect of the business do you think made more money, got more attention, and received more company support? That's an easy question. How is a salesperson supposed to look a prospect in the eye and say "You're going to lose. What are you going to do about it?"

Many businesses are waking up to the fact that they've spent millions of dollars on preventative measures and they still lose. No one likes to be a loser. The fact of the matter is that winning cannot be defined as zero intrusions. Risk mitigation does not mean risk elimination. Winning has to be defined using the words I used to explain risk in my first book:

Security is the process of maintaining an acceptable level of perceived risk.

This definition does not eliminate intrusions from the enterprise. It does leave an uncomfortable amount of interpretation for the "acceptable level" aspect. You may have noticed that most of the managers one might consider successful are usually self-described or outwardly praised as being risk-takers. On the other side of the equation we have security professionals, most of whom I would label as risk-avoiders.

The source escapes me now, but a recent security magazine article observed that those closest to the hands-on aspects of security rated their companies as being the least secure. Assessments of company security improved the farther one was removed from day-to-day operations, such that the CIO and above was much more positive about the company's security outlook. The major factor in this equation is probably the separation between the corner office and the cubicle, but another could be the acceptable level of risk for the parties involved. When a CIO or CEO is juggling market risk, credit risk, geo-political risk, legal risk, and other worries, digital risk is just another item in the portfolio.

The difference between digital risk and many of the other risk types is the consequences can be tough to identify. In fact, the more serious the impact, the least likely you could be to discover the intrusion.

How is that possible? What causes more damage: a DDoS attack that everyone notices because "the network is slow," or a stealthy economic competitor whose entire reason in life is to avoid detection while stealing data?

Without evidence to answer the question are you secure?, managers practice management and defense by belief instead of management and defense by fact.

Saturday, December 15, 2007

Feds Plan to Reduce, Then Monitor

According to OMB directs agencies to close off most Internet links, by June 2008 the Federal government plans to reduce the number of Internet connections it maintains, and then monitor them more closely:

The Office of Management and Budget's Trusted Internet Connections (TIC) initiative likely is to be the last publicized program in the Bush administration's stepped-up focus on cybersecurity, some experts say. More importantly, the new initiative requires agencies to implement real-time gateway monitoring, which has been a deficit in federal network protection.

The TIC initiative mandates that officials develop plans for limiting the number of Internet connections into their departments and agencies. OMB officials want to reduce the number of gateways from the more than 1,000 to about 50, said Karen Evans, OMB's administrator for e-government and information technology.
(emphasis added)

This sounds promising. The story continues:

The initiative also asks chief information officers to develop a plan of action and milestones for participating in the Homeland Security Department's U.S. Computer Emergency Readiness Team's Einstein initiative. The program offers agencies real-time gateway monitoring capabilities and helps them react more quickly to security incidents. About 13 agencies voluntarily participate in the Einstein program.

"The reduction of access points to trusted Internet connections will improve our situational awareness and allow us to address potential threats in an expedited and efficient manner," Evans said. "While we optimize and improve our security, it is also our goal to minimize overall operating costs for services through economies of scale."


Reduction of gateways + enhanced monitoring = better, stronger, faster -- and cheaper.

The story With Internet gateways, less is more adds:

A June deadline for agencies to consolidate their Internet connections coincides with another OMB deadline. June is also when agencies must upgrade their backbone networks to run the next-generation Internet protocol, IPv6...

“The [TIC] initiative is saying, ‘We have to know what we own in order to protect it,’ ” Evans said. “We also must know we are managing risk at an acceptable level.”

Evans said the federal government has more than 1,000 gateways to the public Internet.

The target number is 50, but that is not an absolute number, she said. “We know 1,000 or more is not the way to do it. At a minimum, 50 is two per department.”

Fifty gateways is a reasonable number, Evans said, adding that the Defense Department has reduced its Internet gateway count to 18. The Homeland Security Department expects to have only two Internet gateways after it completes its OneNet initiative.

“The 50 or so points of presence [would] become the perimeter of the federal government,” Evans said.
(emphasis added)

Kudos to Karen Evans. I am hopeful that someone who realizes FISMA Is a Joke has begun steering the Federal government away from worthless documentation and towards real network security operations.

Wednesday, December 12, 2007

Incident Severity Ratings

Much of digital security focuses on pre-compromise activities. Not as much attention is paid to what happens once your defenses fail. My friend Bamm brought this problem to my attention when he discussed the problem of rating the severity of an incident. He was having trouble explaining to his management the impact of an intrusion, so he asked if I had given any thought to the issue.

What follows is my attempt to apply a framework to the problem. If anyone wants to point me to existing work, please feel free. This is not an attempt to put a flag in the ground. We're trying to figure out how to talk about post-compromise activities in a world where scoring vulnerabilities receives far more attention.

This is a list of factors which influence the severity of an incident. It is written mainly from the intrusion standpoint. In other words, an unauthorized party is somehow interacting with your asset. I have ordered the options under each category such that the top items in each sub-list is considered worst, and the bottom is best. Since this is a work in progress I put question marks in many of the sub-lists.

  1. Level of Control


    • Domain or network-wide SYSTEM/Administrator/root

    • Local SYSTEM/Administrator/root

    • Privileged user (but not SYSTEM/Administrator/root

    • User

    • None?


  2. Level of Interaction


    • Shell

    • API

    • Application commands

    • None?


  3. Nature of Contact


    • Persistent and continuous

    • On-demand

    • Re-exploitation required

    • Misconfiguration required

    • None?


  4. Reach of Victim


    • Entire enterprise

    • Specific zones

    • Local segment only

    • Host only


  5. Nature of Victim Data


    • Exceptionally grave damage if destroyed/altered/disclosed

    • Grave damage if destroyed/altered/disclosed

    • Some damage if destroyed/altered/disclosed

    • No damage if destroyed/altered/disclosed


  6. Degree of Friendly External Control of Victim


    • None; host has free Internet access inbound and outbound

    • Some external control of access

    • Comprehensive external control of access


  7. Host Vulnerability (for purposes of future re-exploitation


    • Numerous severe vulnerabilities

    • Moderate vulnerability

    • Little to no vulnerability


  8. Friendly Visibility of Victim


    • No monitoring of network traffic or host logs

    • Only network or host logging (not both)

    • Comprehensive network and host visibility


  9. Threat Assessment


    • Highly skilled and motivated, or structured threat

    • Moderately skilled and motivated, or semi-structured threat

    • Low skilled and motivated, or unstructured threat


  10. Business Impact (from continuity of operations plan)


    • High

    • Medium

    • Low


  11. Onsite Support


    • None

    • First level technical support present

    • Skilled operator onsite



Based on this framework, I would be most worried about the following -- stated very bluntly so you see all eleven categories: I worry about an incident where the intruder has SYSTEM control, with a shell, that is persistent, on a host that can reach the entire enterprise, on a host with very valuable data, with unfettered Internet access, on a host with lots of serious holes, and I can't see the host's logs or traffic, and the intruder is a foreign intel service, and the host is a high biz impact system, and no one is on site to help me.

What do you think?

Saturday, December 01, 2007

Expert Commentary on SPAN and RSPAN Weaknesses

It's no secret I am a fan of using taps instead of switch SPAN ports when instrumenting networks. Two excellent posts explain the weakness of using SPAN ports and RSPAN.

Both of these were written by Tim O'Neill, an independent consultant.

This is the simplest way for me to compare SPAN ports to taps: a SPAN port is a girlfriend, but a tap is a wife. It takes a real level of institutional commitment to install a tap, and the rewards are long-lasting. A SPAN port is a temporary fling subject to break-up (i.e., deactivation).

Furthermore, I really liked the blog post's emphasis on SPAN configuration as a change that must be allowed by the change control board in any semi-mature IT shop. The only CCB action needed for a tap is the initial installation. Any change to a SPAN port configuration should be authorized by the CCB. This is one of the reasons why very mature (and well-funded) IT shops use matrix switches for on-demand visibility, as a mentioned last year in Notes on Net Optics Think Tank.

Monday, November 26, 2007

Controls Are Not the Solution to Our Problem

If you recognize the inspiration for this post title and graphic, you'll understand my ultimate goal. If not, let me start by saying this post is an expansion of ideas presented in a previous post with the succinct and catchy title Control-Compliant vs Field-Assessed Security.

In brief, too many organizations, regulators, and government agencies waste precious time and resources devising and auditing "controls," regardless of the effect these controls have or do not have on security. They are far too input-centric; they should become more output-aware. They obsess over recording conditions they believe may be helpful while remaining ignorant of the "score of the game." They practice management by belief and disregard management by fact.

Let me provide a few examples from one of the canonical texts used by the control-compliant crowd: NIST Special Publication 800-53: Recommended Security Controls for Federal Information Systems (.pdf). The following is an example of a control, taken from page 140.

SI-3 MALICIOUS CODE PROTECTION


The information system implements malicious code protection.

Control: Supplemental Guidance: The organization employs malicious code protection mechanisms at critical information system entry and exit points (e.g., firewalls, electronic mail servers, web servers, proxy servers, remote-access servers) and at workstations, servers, or mobile computing devices on the network. The organization uses the malicious code protection mechanisms to detect and eradicate malicious code (e.g., viruses, worms, Trojan horses, spyware) transported: (i) by electronic mail, electronic mail attachments, Internet accesses, removable media (e.g., USB devices, diskettes or compact disks), or other common means; or (ii) by exploiting information system vulnerabilities. The organization updates malicious code protection mechanisms (including the latest virus definitions) whenever new releases are available in accordance with organizational configuration management policy and procedures. The organization considers using malicious code protection software products from multiple vendors (e.g., using one vendor for boundary devices and servers and another vendor for workstations). The organization also considers the receipt of false positives during malicious code detection and eradication and the resulting potential impact on the availability of the information system. NIST Special Publication 800-83 provides guidance on implementing malicious code protection.

Control Enhancements:
(1) The organization centrally manages malicious code protection mechanisms.
(2) The information system automatically updates malicious code protection mechanisms.


At first read one might reasonably respond by saying "What's wrong with that? This control advocates implementing anti-virus and related anti-malware software." Think more clearly about this issue and several problems appear.

  • Adding anti-virus products can introduce additional vulnerabilities to systems which might not have exposed themselves without running anti-virus. Consider my post Example of Security Product Introducing Vulnerabilities if you need examples. In short, add anti-virus, be compromised.

  • Achieving compliance may cost more than potential damage. How many times have you heard a Unix administrator complain that he/she has to purchase an anti-virus product for his/her Unix server simply to be compliant with a control like this? The potential for a Unix server (not Mac OS X) to be damaged by a user opening an email through a client while logged on to the server (a very popular exploitation vector on a Windows XP box) is practically nil.

  • Does this actually work? This is the question that no one asks. Does it really matter if your system is running anti-virus software? Did you know that intruders (especially high-end ones most likely to selectively, steathily target the very .gov and .mil systems required to be compliant with this control) test their malware against a battery of anti-virus products to ensure their code wins? Are weekly updates superior to daily updates? Daily to hourly?


The purpose of this post is to tentatively propose an alternative approach. I called this "field-assessed" in contrast to "control-compliant." Some people prefer the term "results-based." Whatever you call it, the idea is to direct attention away from inputs and devote more energy to outputs. As far as mandating inputs (like every device must run anti-virus), I say that is a waste of time and resources.

I recommend taking measurements to determine your enterprise "score of the game," and use that information to decide what you need to do differently. I'm not suggesting abandoning efforts to prevent intrusions (i.e., "inputs.") Rather, don't think your security responsibilities end when the bottle is broken against the bow of the ship and it slides into the sea. You've got to keep watching to see if it sinks, if pirates attack, how the lifeboats handle rough seas, and so forth.

These are a few ideas.

  1. Standard client build client-side survival test. Create multiple sacrificial systems with your standard build. Deploy a client-side testing solution on them, like a honeyclient. (See The Sting for a recent story.) Vary your defensive posture. Measure how long it takes for your standard build to be compromised by in-the-wild Web sites, spam, and other communications with the outside world.

  2. Standard client build server-side survival test. Create multiple sacrificial systems with your standard build. Deploy them as a honeynet. Vary your defensive posture. Measure how long it takes for your standard build to be compromised by malicious external traffic from the outside world -- or better yet -- from your internal network.

  3. Standard client build client-side penetration test. Create multiple sacrificial systems with your standard build. Conduct my recommendation penetration testing activities and time the result.

  4. Standard client build server-side penetration test. Repeat number 3 with a server-side flavor.

  5. Standard server build server-side penetration test. Repeat number 3 against your server build with a server-side flavor. I hope you don't have users operating servers as if they were clients (i.e., browsing the Web, reading email, and so forth.) If you do, repeat this step and do a client-side pen test too.

  6. Deploy low-interactive honeynets and sinkhole routers in your internal network. These low-interaction systems provide a means to get some indications of what might be happening inside your network. If you think deploying these on the external network might reveal indications of targeted attacks, try that. (I doubt it will be that useful due to the overall attack noise, but who knows?)

  7. Conduct automated, sampled client host integrity assessments. Select a statistically valid subset of your clients and check them using multiple automated tools (malware/rootkit/etc. checkers) for indications of compromise.

  8. Conduct automated, sampled server host integrity assessments. Self-explanatory.

  9. Conduct manual, sampled client host integrity assessments. These are deep-dives of individual systems. You can think of it as an incident response where you have not had indication of an incident yet. Remote IR tools can be helpful here. If you are really hard-core and you have the time, resources, and cooperation, do offline analysis of the hard drive.

  10. Conduct manual, sampled server host integrity assessments. Self-explanatory.

  11. Conduct automated, sampled network host activity assessments. I questioned adding this step here, since you should probably always be doing this. Sometimes it can be difficult to find the time to review the results, however automated the data collection. The idea is to let your NSM system see if any of the traffic it sees is out of the ordinary based on algorithms you provide.

  12. Conduct manual, sampled network host activity assessments. This method is more likely to produce results. Here a skilled analyst performs deep individual analysis of traffic on a sample of machines (client and server, separately) to see if any indications of compromise appear.


In all of these cases, trend your measurements over time to see if you see improvements when you alter an input. I know some of you might complain that you can't expect to have consistent output when the threat landscape is constantly changing. I really don't care, and neither does your CEO or manager!

I offer two recommendations:

  • Remember Andy Jaquith's criteria for good metrics, simplified here.


    1. Measure consistently.

    2. Make them cheap to measure. (Sorry Andy, my manual tests violate this!)

    3. Use compound metrics.

    4. Be actionable.


  • Don't slip into thinking of inputs. Don't measure how many hosts are running anti-virus. We want to measure outputs. We are not proposing new controls.


Controls are not the solution to our problem. Controls are the problem. They divert too much time, resources, and attention from endeavors which do make a difference. If the indications I am receiving from readers and friends are true, the ideas in this post are gaining traction. Do you have other ideas?

Friday, November 23, 2007

MPAA University Toolkit Phone Home

This is a follow-up to my story Examining the MPAA University Toolkit.

After reading the hysteria posted on the Slashdot story MPAA College Toolkit Raises Privacy, Security Concerns, I thought I would take a look at traffic leaving the box. Aside from traffic generated by the auto-start of Firefox, the only interesting event was the following. I captured it with my gateway Sguil sensor.

Sensor Name: hacom
Timestamp: 2007-11-23 21:27:04
Connection ID: .hacom_5136150487897024842
Src IP: 69.255.105.234 (c-69-255-105-234.hsd1.va.comcast.net)
Dst IP: 66.252.137.155 (Unknown)
Src Port: 39532
Dst Port: 80
OS Fingerprint: 69.255.105.234:39532 - UNKNOWN
[S4:61:1:60:M1460,S,T,N,W4:.:?:?] (up: 3 hrs)
OS Fingerprint: -> 66.252.137.155:80 (link: ethernet/modem)

SRC: GET /version.txt HTTP/1.1
SRC: Accept-Encoding: identity
SRC: Host: universitytoolkit.com
SRC: Connection: close
SRC: User-Agent: Python-urllib/2.5
SRC:
SRC:
DST: HTTP/1.1 200 OK
DST: Date: Fri, 23 Nov 2007 21:27:31 GMT
DST: Server: Apache/2.0.52 (Red Hat)
DST: Last-Modified: Fri, 12 Oct 2007 14:14:45 GMT
DST: ETag: "4f4002-7-57333f40"
DST: Accept-Ranges: bytes
DST: Content-Length: 7
DST: Connection: close
DST: Content-Type: text/plain; charset=UTF-8
DST:
DST: 1.2-RC3

That's it.

Examining the MPAA University Toolkit

I learned about the MPAA University Toolkit at Brian Krebs' always-excellent SecurityFix blog. If you want to know more about the user experience, please check out that post. Here I take a look at the monitoring software, focusing on Snort, operating on this application.

I downloaded the 534 MB peerwatch-1.2-RC5.iso and started it in a VMware Server session. I used ctrl-c and then 'sudo bash' to exit from the initial script presented within X, set a root password, then used 'apt-get ssh install' to install OpenSSH and thus enable root access. From this point forward I accessed the system using OpenSSH remotely to facilitate copying information into this blog post.

First, this looks like Ubuntu (Xubuntu, if you really care) Feisty Fawn, or 7.04.

root@ubuntu:~# uname -a
Linux ubuntu 2.6.20-15-generic #2 SMP Sun Apr 15 07:36:31 UTC 2007
i686 GNU/Linux

I was most interested in learning about Snort on this toolkit. I saw this version installed.

root@ubuntu:~# snort -V

,,_ -*> Snort! <*-
o" )~ Version 2.3.3 (Build 14)
'''' By Martin Roesch & The Snort Team: http://www.snort.org/team.html
(C) Copyright 1998-2004 Sourcefire Inc., et al.

Wow, that's old. It's probably patched base on the changelog. This is Snort installed via Debian/Ubuntu package:

root@ubuntu:~# dpkg --list | grep snort
rc snort 2.3.3-9
Flexible Network Intrusion Detection System
ii snort-common 2.3.3-9
Flexible Network Intrusion Detection System
ii snort-mysql 2.3.3-9
Flexible Network Intrusion Detection System
ii snort-rules-default 2.3.3-9
Flexible Network Intrusion Detection System

Let's see what the snort.conf looks like.

root@ubuntu:/etc/snort# cat snort.conf
var HOME_NET any
var EXTERNAL_NET !$HOME_NET
var DNS_SERVERS $HOME_NET
var SMTP_SERVERS $HOME_NET
var HTTP_SERVERS $HOME_NET
var SQL_SERVERS $HOME_NET
var TELNET_SERVERS $HOME_NET
var SNMP_SERVERS $HOME_NET

var HTTP_PORTS 80

var RULE_PATH /etc/snort/rules

preprocessor flow: stats_interval 0 hash 2
preprocessor frag2
preprocessor stream4: disable_evasion_alerts detect_scans
preprocessor stream4_reassemble

# preprocessor perfmonitor: time 300 file /var/snort/snort.stats pktcnt 10000

# (#DBSTART#)
output database: log, mysql, user=snort password=snort dbname=snort host=localhost
# (#DBEND#)

include classification.config
include reference.config

config flowbits_size: 256

include $RULE_PATH/bleeding-p2p.rules
include $RULE_PATH/local-ftp.rules
include $RULE_PATH/local-http.rules
include $RULE_PATH/local-smb.rules
include $RULE_PATH/p2p.rules

include threshold.conf

Excellent, another Snort installation where Snort is logging directly to a MySQL database. That must be the default provided by Debian/Ubuntu. Ouch. Thresholding and suppression are also enabled but the entire contents are commented out in the threshold.conf file.

Let's get a look at those rules.

bleeding-p2p.rules looks like an old copy of the bleeding-p2p.rules, perhaps from mid-year? I think there are 38 rules.

p2p.rules is a really old rule set:

# $Id: p2p.rules,v 1.17.2.1 2004/10/13 20:25:57 bmc Exp $

You may recognize these and the other Snort distributed-rules as being those that accompanied Snort 2.3.3, which pre-dates the new license for Snort rules.

local-ftp.rules is the first rule set written by whomever assembled this toolkit.

# cat local-ftp.rules
# 1 000 500 - 1 000 699

# active
alert tcp any 20 -> any any (msg: "FTP Download - MPEG Movie File - B2"; \
content: "|00 00 01 B2|"; depth: 6; rawbytes; \
sid: 1000501; rev: 1; \
)

alert tcp any 20 -> any any (msg: "FTP Download - MPEG Movie File - B3"; \
content: "|00 00 01 B3|"; depth: 6; rawbytes; \
sid: 1000502; rev: 1; \
)

alert tcp any 20 -> any any (msg: "FTP Download - MPEG Movie File - BA"; \
content: "|00 00 01 BA|"; depth: 6; rawbytes; \
sid: 1000503; rev: 1; \
)

alert tcp any 20 -> any any (msg: "FTP Download - MPEG Movie File - BB"; \
content: "|00 00 01 BB|"; depth: 6; rawbytes; \
sid: 1000504; rev: 1; \
)

alert tcp any 20 -> any any (msg: "FTP Download - MPEG-4 Video File"; \
content: "|00 00 00 18 66 74 79 70 6D 70 34|"; depth: 15; rawbytes; \
sid: 1000505; rev: 1; \
)

alert tcp any 20 -> any any (msg: "FTP Download - Quicktime Movie File - MOOV"; \
content: "|6D 6F 6F 76|"; depth: 10; rawbytes; \
sid: 1000506; rev: 1; \
)

alert tcp any 20 -> any any (msg: "FTP Download - Quicktime Movie File - MDAT"; \
content: "|6D 64 61 74|"; depth: 10; rawbytes; \
sid: 1000507; rev: 1; \
)

alert tcp any 20 -> any any (msg: "FTP Download - Audio Video Interleave (AVI) File - AVI"; \
content: "|41 56 49 20|"; depth: 6; rawbytes; \
sid: 1000508; rev: 1; \
)

alert tcp any 20 -> any any (msg: "FTP Download - Audio Video Interleave (AVI) File - RIFF"; \
content: "|52 49 46 46|"; depth: 6; rawbytes; \
sid: 1000509; rev: 1; \
)

alert tcp any 20 -> any any (msg: "FTP Download - Real Media File"; \
content: "|2E 52 4D 46|"; depth: 6; rawbytes; \
sid: 1000510; rev: 1; \
)

alert tcp any 20 -> any any (msg: "FTP Download - Windows Media File"; \
content: "|30 26 B2 75 8E 66 CF 11 A6 D9 00 AA 00 62 CE 6C|"; depth: 20; rawbytes; \
sid: 1000511; rev: 1; \
)

# passive
alert tcp any 1024: -> any 1024: (msg: "FTP PASV Download - MPEG Movie File - B2"; \
content: "|00 00 01 B2|"; depth: 6; rawbytes; \
sid: 1000512; rev: 1; \
)

alert tcp any 1024: -> any 1024: (msg: "FTP PASV Download - MPEG Movie File - B3"; \
content: "|00 00 01 B3|"; depth: 6; rawbytes; \
sid: 1000513; rev: 1; \
)

alert tcp any 1024: -> any 1024: (msg: "FTP PASV Download - MPEG Movie File - BA"; \
content: "|00 00 01 BA|"; depth: 6; rawbytes; \
sid: 1000514; rev: 1; \
)

alert tcp any 1024: -> any 1024: (msg: "FTP PASV Download - MPEG Movie File - BB"; \
content: "|00 00 01 BB|"; depth: 6; rawbytes; \
sid: 1000515; rev: 1; \
)

alert tcp any 1024: -> any 1024: (msg: "FTP PASV Download - MPEG-4 Video File"; \
content: "|00 00 00 18 66 74 79 70 6D 70 34|"; depth: 15; rawbytes; \
sid: 1000516; rev: 1; \
)

alert tcp any 1024: -> any 1024: (msg: "FTP PASV Download - Quicktime Movie File - MOOV"; \
content: "|6D 6F 6F 76|"; depth: 10; rawbytes; \
sid: 1000517; rev: 1; \
)

alert tcp any 1024: -> any 1024: (msg: "FTP PASV Download - Quicktime Movie File - MDAT"; \
content: "|6D 64 61 74|"; depth: 10; rawbytes; \
sid: 1000518; rev: 1; \
)

alert tcp any 1024: -> any 1024: (msg: "FTP PASV Download - Audio Video Interleave (AVI) File - AVI"; \
content: "|41 56 49 20|"; depth: 6; rawbytes; \
sid: 1000519; rev: 1; \
)

alert tcp any 1024: -> any 1024: (msg: "FTP PASV Download - Audio Video Interleave (AVI) File - RIFF"; \
content: "|52 49 46 46|"; depth: 6; rawbytes; \
sid: 1000520; rev: 1; \
)

alert tcp any 1024: -> any 1024: (msg: "FTP PASV Download - Real Media File"; \
content: "|2E 52 4D 46|"; depth: 6; rawbytes; \
sid: 1000521; rev: 1; \
)

alert tcp any 1024: -> any 1024: (msg: "FTP PASV Download - Windows Media File"; \
content: "|30 26 B2 75 8E 66 CF 11 A6 D9 00 AA 00 62 CE 6C|"; depth: 20; rawbytes; \
sid: 1000522; rev: 1; \
)

Anyone who has written Snort rules is probably going to question the false positive rate on this rule set, especially the "tcp any 1024: -> any 1024:" group. These are straight content matches, and the smaller strings like "|2E 52 4D 46|" are probably going to fire quite a bit on unintended traffic.

Here is local-http.rules.

root@ubuntu:/etc/snort/rules# cat local-http.rules
# 1 000 100 - 1 000 299

alert tcp any 80 -> any any (msg: "HTTP Download > 100M - MPEG Movie File - B2"; \
flow: established,from_server; \
content: "HTTP"; depth: 5; nocase; \
content: "200"; within: 8; \
content: "Content-Length\: "; within: 300; nocase; \
pcre: "/^[0-9]{9,}\r\n/R"; \
content: "|0d 0a 0d 0a|"; within: 100; \
content: "|00 00 01 B2|"; within: 6; \
sid: 1000101; rev: 1; \
)

alert tcp any 80 -> any any (msg: "HTTP Download > 100M - MPEG Movie File - B3"; \
flow: established,from_server; \
content: "HTTP"; depth: 5; nocase; \
content: "200"; within: 8; \
content: "Content-Length\: "; within: 300; nocase; \
pcre: "/^[0-9]{9,}\r\n/R"; \
content: "|0d 0a 0d 0a|"; within: 100; \
content: "|00 00 01 B3|"; within: 6; \
sid: 1000102; rev: 1; \
)

alert tcp any 80 -> any any (msg: "HTTP Download > 100M - MPEG Movie File - BA"; \
flow: established,from_server; \
content: "HTTP"; depth: 5; nocase; \
content: "200"; within: 8; \
content: "Content-Length\: "; within: 300; nocase; \
pcre: "/^[0-9]{9,}\r\n/R"; \
content: "|0d 0a 0d 0a|"; within: 100; \
content: "|00 00 01 BA|"; within: 6; \
sid: 1000103; rev: 1; \
)

alert tcp any 80 -> any any (msg: "HTTP Download > 100M - MPEG Movie File - BB"; \
flow: established,from_server; \
content: "HTTP"; depth: 5; nocase; \
content: "200"; within: 8; \
content: "Content-Length\: "; within: 300; nocase; \
pcre: "/^[0-9]{9,}\r\n/R"; \
content: "|0d 0a 0d 0a|"; within: 100; \
content: "|00 00 01 BB|"; within: 6; \
sid: 1000104; rev: 1; \
)

alert tcp any 80 -> any any (msg: "HTTP Download > 100M - MPEG-4 Video File"; \
flow: established,from_server; \
content: "HTTP"; depth: 5; nocase; \
content: "200"; within: 8; \
content: "Content-Length\: "; within: 300; nocase; \
pcre: "/^[0-9]{9,}\r\n/R"; \
content: "|0d 0a 0d 0a|"; within: 100; \
content: "|00 00 00 18 66 74 79 70 6D 70 34|"; within: 15; \
sid: 1000105; rev: 1; \
)

alert tcp any 80 -> any any (msg: "HTTP Download > 100M - Quicktime Movie File - MOOV"; \
flow: established,from_server; \
content: "HTTP"; depth: 5; nocase; \
content: "200"; within: 8; \
content: "Content-Length\: "; within: 300; nocase; \
pcre: "/^[0-9]{9,}\r\n/R"; \
content: "|0d 0a 0d 0a|"; within: 100; \
content: "|6D 6F 6F 76|"; within: 10; \
sid: 1000106; rev: 1; \
)
alert tcp any 80 -> any any (msg: "HTTP Download > 100M - Quicktime Movie File - MDAT"; \
flow: established,from_server; \
content: "HTTP"; depth: 5; nocase; \
content: "200"; within: 8; \
content: "Content-Length\: "; within: 300; nocase; \
pcre: "/^[0-9]{9,}\r\n/R"; \
content: "|0d 0a 0d 0a|"; within: 100; \
content: "|6D 64 61 74|"; within: 10; \
sid: 1000107; rev: 1; \
)

alert tcp any 80 -> any any (msg: "HTTP Download > 100M - Audio Video Interleave (AVI) File - AVI"; \
flow: established,from_server; \
content: "HTTP"; depth: 5; nocase; \
content: "200"; within: 8; \
content: "Content-Length\: "; within: 300; nocase; \
pcre: "/^[0-9]{9,}\r\n/R"; \
content: "|0d 0a 0d 0a|"; within: 100; \
content: "|41 56 49 20|"; within: 6; \
sid: 1000108; rev: 1; \
)

alert tcp any 80 -> any any (msg: "HTTP Download > 100M - Audio Video Interleave (AVI) File - RIFF"; \
flow: established,from_server; \
content: "HTTP"; depth: 5; nocase; \
content: "200"; within: 8; \
content: "Content-Length\: "; within: 300; nocase; \
pcre: "/^[0-9]{9,}\r\n/R"; \
content: "|0d 0a 0d 0a|"; within: 100; \
content: "|52 49 46 46|"; within: 6; \
sid: 1000109; rev: 1; \
)

alert tcp any 80 -> any any (msg: "HTTP Download > 100M - Real Media File"; \
flow: established,from_server; \
content: "HTTP"; depth: 5; nocase; \
content: "200"; within: 8; \
content: "Content-Length\: "; within: 300; nocase; \
pcre: "/^[0-9]{9,}\r\n/R"; \
content: "|0d 0a 0d 0a|"; within: 100; \
content: "|2E 52 4D 46|"; within: 6; \
sid: 1000110; rev: 1; \
)

alert tcp any 80 -> any any (msg: "HTTP Download > 100M - Windows Media File"; \
flow: established,from_server; \
content: "HTTP"; depth: 5; nocase; \
content: "200"; within: 8; \
content: "Content-Length\: "; within: 300; nocase; \
pcre: "/^[0-9]{9,}\r\n/R"; \
content: "|0d 0a 0d 0a|"; within: 100; \
content: "|30 26 B2 75 8E 66 CF 11 A6 D9 00 AA 00 62 CE 6C|"; within: 20; \
sid: 1000111; rev: 1; \
)

That's 11 rules. There are 22 more. The middle 11 have port 80 replaced by 3128. The final 11 have port 8080. What does that tell you? It means that you can avoid being detected by these rules if the remote Web server runs on a port other than 80, 3128, or 8080. Note also that the original snort.conf doesn't enable the http_inspect or http_inspect_server preprocessors. These rules are more raw content matches, although their specificity will reduce the number of times they fire. They also introduce more evasion options.

Finally, let's check out local-smb.rules.

root@ubuntu:/etc/snort/rules# cat local-smb.rules
# 1 000 300 - 1 000 499

alert tcp any 445 -> any any (msg: "SMB-445 Download > 100M - MPEG Movie File - B2"; \
flow: established,from_server; \
content: "SMB|2E|"; offset: 5; within: 4; nocase; \
content: "|00 00 01 B2|"; distance: 54; within: 4; \
sid: 1000301; rev: 1; \
)

alert tcp any 445 -> any any (msg: "SMB-445 Download > 100M - MPEG Movie File - B3"; \
flow: established,from_server; \
content: "SMB|2E|"; offset: 5; within: 4; nocase; \
content: "|00 00 01 B3|"; distance: 54; within: 4; \
sid: 1000302; rev: 1; \
)

alert tcp any 445 -> any any (msg: "SMB-445 Download > 100M - MPEG Movie File - BA"; \
flow: established,from_server; \
content: "SMB|2E|"; offset: 5; within: 4; nocase; \
content: "|00 00 01 BA|"; distance: 54; within: 4; \
sid: 1000303; rev: 1; \
)

alert tcp any 445 -> any any (msg: "SMB-445 Download > 100M - MPEG Movie File - BB"; \
flow: established,from_server; \
content: "SMB|2E|"; offset: 5; within: 4; nocase; \
content: "|00 00 01 BB|"; distance: 54; within: 4; \
sid: 1000304; rev: 1; \
)

alert tcp any 445 -> any any (msg: "SMB-445 Download > 100M - MPEG-4 Video File"; \
flow: established,from_server; \
content: "SMB|2E|"; offset: 5; within: 4; nocase; \
content: "|00 00 00 18 66 74 79 70 6D 70 34|"; distance: 54; within: 15; \
sid: 1000305; rev: 1; \
)

alert tcp any 445 -> any any (msg: "SMB-445 Download > 100M - Quicktime Movie File - MOOV"; \
flow: established,from_server; \
content: "SMB|2E|"; offset: 5; within: 4; nocase; \
content: "MOOV"; distance: 54; within: 8; nocase; \
sid: 1000306; rev: 1; \
)

alert tcp any 445 -> any any (msg: "SMB-445 Download > 100M - Quicktime Movie File - MDAT"; \
flow: established,from_server; \
content: "SMB|2E|"; offset: 5; within: 4; nocase; \
content: "MDAT"; distance: 54; within: 4; nocase; \
sid: 1000307; rev: 1; \
)

alert tcp any 445 -> any any (msg: "SMB-445 Download > 100M - Audio Video Interleave (AVI) File - AVI"; \
flow: established,from_server; \
content: "SMB|2E|"; offset: 5; within: 4; nocase; \
content: "AVI_"; distance: 54; within: 4; nocase; \
sid: 1000308; rev: 1; \
)

alert tcp any 445 -> any any (msg: "SMB-445 Download > 100M - Audio Video Interleave (AVI) File - RIFF"; \
flow: established,from_server; \
content: "SMB|2E|"; offset: 5; within: 4; nocase; \
content: "RIFF"; distance: 54; within: 4; nocase; \
sid: 1000309; rev: 1; \
)

alert tcp any 445 -> any any (msg: "SMB-445 Download > 100M - Real Media File"; \
flow: established,from_server; \
content: "SMB|2E|"; offset: 5; within: 4; nocase; \
content: "|2E 52 4D 46|"; distance: 54; within: 4; \
sid: 1000310; rev: 1; \
)

alert tcp any 445 -> any any (msg: "SMB-445 Download > 100M - Windows Media File"; \
flow: established,from_server; \
content: "SMB|2E|"; offset: 5; within: 4; nocase; \
content: "|30 26 B2 75 8E 66 CF 11 A6 D9 00 AA 00 62 CE 6C|"; distance: 54; within: 16; \
sid: 1000311; rev: 1; \
)

Notice all the port 445 instances? You can evade these if your SMB session uses port 139 TCP.

I thought it might be fun to test these rules. I decided to download a 108 MB .avi file to the toolkit host itself and see if would be observed.

file robert-morris.avi
robert-morris.avi: RIFF (little-endian) data, AVI, 640 x 480, 30.00 fps,
video: Motion JPEG, audio: uncompressed PCM (mono, 11024 Hz)

Hmm, no alerts. I have Sguil running on my gateway. Let's see what the start of a transcript for this session looks like.

Sensor Name: hacom
Timestamp: 2007-11-23 21:32:47
Connection ID: .hacom_5136151961070210685
Src IP: 69.255.105.234 (c-69-255-105-234.hsd1.va.comcast.net)
Dst IP: 164.106.251.250 (Unknown)
Src Port: 58172
Dst Port: 80
OS Fingerprint: 69.255.105.234:58172 - UNKNOWN
[S4:61:1:60:M1460,S,T,N,W4:.:?:?] (up: 3 hrs)
OS Fingerprint: -> 164.106.251.250:80 (link: ethernet/modem)

SRC: GET /docs/netsec/robert-morris.avi HTTP/1.0
SRC: User-Agent: Wget/1.10.2
SRC: Accept: */*
SRC: Host: 164.106.251.250
SRC: Connection: Keep-Alive
SRC:
SRC:
DST: HTTP/1.1 200 OK
DST: Date: Fri, 23 Nov 2007 21:38:16 GMT
DST: Server: Apache/2.0.52 (Red Hat)
DST: Last-Modified: Tue, 23 Aug 2005 21:46:31 GMT
DST: ETag: "37804f-6bfad96-ba9f7bc0"
DST: Accept-Ranges: bytes
DST: Content-Length: 113225110
DST: Connection: close
DST: Content-Type: video/x-msvideo
DST:
DST:
DST: RIFF....AVI LISTF...hdrlavih8...5...D.&......................I..
LISTt...strlstrh8...vidsmjpg............5...@B...........I...'..............
strf(...(...............MJPG....................LIST\...strlstrh8...auds....
.................+......\
DST: ..+...'..............strf.........+...+......IDIT....
FRI JUL 29 15:54:43 2005
DST: .LIST....INFOISFT....CanonMVI02..JUNK~...

After the HTTP response you see the download begin for the .avi. Presumably this would match, this rule?

"HTTP Download > 100M - Audio Video Interleave (AVI) File - RIFF"

Let's look at the two most important packets in the full content pcap file.

16:32:47.335530 IP 164.106.251.250.80 > 69.255.105.234.58172:
P 1:268(267) ack 133 win 1716 <nop,nop,timestamp 2163691250 1274738>
0x0000: 4520 013f e980 4000 3006 0fca a46a fbfa E..?..@.0....j..
0x0010: 45ff 69ea 0050 e33c f12d d653 a3ca 374e E.i..P.<.-.S..7N
0x0020: 8018 06b4 ce3b 0000 0101 080a 80f7 4ef2 .....;........N.
0x0030: 0013 7372 4854 5450 2f31 2e31 2032 3030 ..srHTTP/1.1.200
0x0040: 204f 4b0d 0a44 6174 653a 2046 7269 2c20 .OK..Date:.Fri,.
0x0050: 3233 204e 6f76 2032 3030 3720 3231 3a33 23.Nov.2007.21:3
0x0060: 383a 3136 2047 4d54 0d0a 5365 7276 6572 8:16.GMT..Server
0x0070: 3a20 4170 6163 6865 2f32 2e30 2e35 3220 :.Apache/2.0.52.
0x0080: 2852 6564 2048 6174 290d 0a4c 6173 742d (Red.Hat)..Last-
0x0090: 4d6f 6469 6669 6564 3a20 5475 652c 2032 Modified:.Tue,.2
0x00a0: 3320 4175 6720 3230 3035 2032 313a 3436 3.Aug.2005.21:46
0x00b0: 3a33 3120 474d 540d 0a45 5461 673a 2022 :31.GMT..ETag:."
0x00c0: 3337 3830 3466 2d36 6266 6164 3936 2d62 37804f-6bfad96-b
0x00d0: 6139 6637 6263 3022 0d0a 4163 6365 7074 a9f7bc0"..Accept
0x00e0: 2d52 616e 6765 733a 2062 7974 6573 0d0a -Ranges:.bytes..
0x00f0: 436f 6e74 656e 742d 4c65 6e67 7468 3a20 Content-Length:.
0x0100: 3131 3332 3235 3131 300d 0a43 6f6e 6e65 113225110..Conne
0x0110: 6374 696f 6e3a 2063 6c6f 7365 0d0a 436f ction:.close..Co
0x0120: 6e74 656e 742d 5479 7065 3a20 7669 6465 ntent-Type:.vide
0x0130: 6f2f 782d 6d73 7669 6465 6f0d 0a0d 0a o/x-msvideo....
16:32:47.336654 IP 164.106.251.250.80 > 69.255.105.234.58172:
. 268:1636(1368) ack 133 win 1716 #60;nop,nop,timestamp 2163691250 1274738#62;
0x0000: 4520 058c e982 4000 3006 0b7b a46a fbfa E.....@.0..{.j..
0x0010: 45ff 69ea 0050 e33c f12d d75e a3ca 374e E.i..P.<.-.^..7N
0x0020: 8010 06b4 b5f8 0000 0101 080a 80f7 4ef2 ..............N.
0x0030: 0013 7372 5249 4646 8ead bf06 4156 4920 ..srRIFF....AVI.
0x0040: 4c49 5354 4601 0000 6864 726c 6176 6968 LISTF...hdrlavih
0x0050: 3800 0000 3582 0000 44d0 2600 0000 0000 8...5...D.&.....
0x0060: 1000 0100 0e07 0000 0000 0000 0200 0000 ................
0x0070: c649 0100 8002 0000 e001 0000 0000 0000 .I..............
0x0080: 0000 0000 0000 0000 0000 0000 4c49 5354 ............LIST
0x0090: 7400 0000 7374 726c 7374 7268 3800 0000 t...strlstrh8...
0x00a0: 7669 6473 6d6a 7067 0000 0000 0000 0000 vidsmjpg........
0x00b0: 0000 0000 3582 0000 4042 0f00 0000 0000 ....5...@B......
0x00c0: 0e07 0000 c649 0100 1027 0000 0000 0000 .....I...'......
0x00d0: 0000 0000 8002 e001 7374 7266 2800 0000 ........strf(...
0x00e0: 2800 0000 8002 0000 e001 0000 0100 1800 (...............
0x00f0: 4d4a 5047 0010 0e00 0000 0000 0000 0000 MJPG............
0x0100: 0000 0000 0000 0000 4c49 5354 5c00 0000 ........LIST\...
0x0110: 7374 726c 7374 7268 3800 0000 6175 6473 strlstrh8...auds
0x0120: 0000 0000 0000 0000 0000 0000 0000 0000 ................
0x0130: 0100 0000 102b 0000 0000 0000 5c20 0a00 .....+......\...
0x0140: 102b 0000 1027 0000 0100 0000 0000 0000 .+...'..........
0x0150: 0000 0000 7374 7266 1000 0000 0100 0100 ....strf........
0x0160: 102b 0000 102b 0000 0100 0800 4944 4954 .+...+......IDIT
0x0170: 1a00 0000 4652 4920 4a55 4c20 3239 2031 ....FRI.JUL.29.1
0x0180: 353a 3534 3a34 3320 3230 3035 0a00 4c49 5:54:43.2005..LI
0x0190: 5354 1800 0000 494e 464f 4953 4654 0c00 ST....INFOISFT..
0x01a0: 0000 4361 6e6f 6e4d 5649 3032 0000 4a55 ..CanonMVI02..JU
0x01b0: 4e4b 7e06 0000 0000 0000 0000 0000 0000 NK~.............
...truncated...

Do you see it? The HTTP response code and the Content-Length statement appear in the first packet. The .avi begins in the second packet with RIFF. Snort doesn't fire an alert because all of the matches needed for the rule are not present in a single packet.

Technically, there's not much to worry about here -- at least not yet. I do worry about putting monitoring tools in the hands of people who don't know what they're doing and seeing them act on misconceptions. It's also important to identify the fact that this activity could violate wiretap and privacy laws.

Wednesday, November 21, 2007

Tap vs Lightning Strike

Earlier this year my lab suffered a near lightning strike. A tree right outside the lab was struck by lightning, causing damage to multiple electronic and electrical devices outside and inside the building.

Outside, the lightning disabled an exterior lighting system and my phone lines. Inside, the lightning took a severe toll on the lab. The cable modem to the outside world was destroyed. The NIC on the lab firewall facing the cable modem was fried, along with a second NIC in the firewall. The NIC on a sensor watching a tap between the cable modem and firewall was also destroyed. So far, this is a grim story.

I have one good piece of news to report, and it involves the tap I mentioned sitting between the cable modem and firewall. The tap survived the lightning strike. More precisely, the tap continued to pass traffic even when its monitoring interface was damaged.

Had the tap been receiving traffic from the modem or firewall, it would have continued to pass it. This truly amazed me. Frequently monitoring practitioners worry that inserting a tap in their network architecture will introduce a single point of failure. In my experience, all of the components around the tap are more likely to fail. A well-engineered tap will continue to pass traffic -- perhaps even when struck by lightning!

The tap that survived my lab lightning strike was built by Net Optics. Congratulations to the Net Optics engineering and manufacturing teams for building quality hardware.

Updating FreeBSD 7.0-BETA2 to 7.0-BETA3

Recently I posted FreeBSD Binary Upgrade News about developments with Colin Percival's FreeBSD Update tool. Today I performed a remote (via SSH) upgrade from FreeBSD 7.0-BETA2 to FreeBSD 7.0-BETA3 using FreeBSD Update. I document the process below so you can see how easy it is and for my future reference.

Here is uname output to show the OS version prior to upgrading.

# uname -a
FreeBSD myhost.mydomain.com 7.0-BETA2 FreeBSD 7.0-BETA2 #0:
Fri Nov 2 16:47:33 UTC 2007
root@logan.cse.buffalo.edu:/usr/obj/usr/src/sys/GENERIC i386

I wasn't sure if the version of FreeBSD Update packaged with FreeBSD 7.0-BETA2 would natively support this process, so I gave it a try.

# freebsd-update -r 7.0-BETA3 upgrade
usage: freebsd-update [options] command ... [path]

Options:
-b basedir -- Operate on a system mounted at basedir
(default: /)
-d workdir -- Store working files in workdir
(default: /var/db/freebsd-update/)
-f conffile -- Read configuration options from conffile
(default: /etc/freebsd-update.conf)
-k KEY -- Trust an RSA key with SHA256 hash of KEY
-s server -- Server from which to fetch updates
(default: update.FreeBSD.org)
-t address -- Mail output of cron command, if any, to address
(default: root)
Commands:
fetch -- Fetch updates from server
cron -- Sleep rand(3600) seconds, fetch updates, and send an
email if updates were found
install -- Install downloaded updates
rollback -- Uninstall most recently installed updates

Ok, that didn't work. Time to retrieve the new version from Colin's site.

# fetch http://www.daemonology.net/freebsd-update/freebsd-update-upgrade.tgz
freebsd-update-upgrade.tgz 100% of 21 kB 104 kBps
# fetch http://www.daemonology.net/freebsd-update/freebsd-update-upgrade.tgz.asc
freebsd-update-upgrade.tgz.asc 100% of 187 B 640 kBps

I decided to follow Colin's advice to check the signature of the upgrade file. To do that I needed to install GnuPG.

# pkg_add -r gnupg
Fetching ftp://ftp.freebsd.org/pub/FreeBSD/ports/i386/packages-7-current/Latest/gnupg.tbz... Done.
Fetching ftp://ftp.freebsd.org/pub/FreeBSD/ports/i386/packages-7-current/All/openldap-client-2.3.39.tbz... Done.

************************************************************

The OpenLDAP client package has been successfully installed.

Edit
/usr/local/etc/openldap/ldap.conf
to change the system-wide client defaults.

Try `man ldap.conf' and visit the OpenLDAP FAQ-O-Matic at
http://www.OpenLDAP.org/faq/index.cgi?file=3
for more information.

************************************************************

Fetching ftp://ftp.freebsd.org/pub/FreeBSD/ports/i386/packages-7-current/All/curl-7.16.3.tbz... Done.
Fetching ftp://ftp.freebsd.org/pub/FreeBSD/ports/i386/packages-7-current/All/pth-2.0.7.tbz... Done.
Fetching ftp://ftp.freebsd.org/pub/FreeBSD/ports/i386/packages-7-current/All/libiconv-1.11_1.tbz... Done.
Fetching ftp://ftp.freebsd.org/pub/FreeBSD/ports/i386/packages-7-current/All/gettext-0.16.1_3.tbz... Done.
Fetching ftp://ftp.freebsd.org/pub/FreeBSD/ports/i386/packages-7-current/All/libgpg-error-1.5.tbz... Done.
Fetching ftp://ftp.freebsd.org/pub/FreeBSD/ports/i386/packages-7-current/All/libgcrypt-1.2.4_1.tbz... Done.
Fetching ftp://ftp.freebsd.org/pub/FreeBSD/ports/i386/packages-7-current/All/libksba-1.0.1_1.tbz... Done.
Fetching ftp://ftp.freebsd.org/pub/FreeBSD/ports/i386/packages-7-current/All/dirmngr-0.9.7_2.tbz... Done.

###############################################################################
A T T E N T I O N

In order to use gpg-agent, you need to install a pinentry dialog.

The following ports of pinentry dialogs are available:

security/pinentry-curses (ncurses based dialog)
security/pinentry-gtk (GTK 1.2 based dialog)
security/pinentry-gtk2 (GTK 2.x based dialog)
security/pinentry-qt (QT based dialog)

###############################################################################

Wow, that installed more dependencies than I expected. Here I import the PGP keys from FreeBSD,org.

# rehash
# fetch http://www.freebsd.org/doc/pgpkeyring.txt
pgpkeyring.txt 100% of 1406 kB 142 kBps 00m00s
# gpg --import pgpkeyring.txt
gpg: /root/.gnupg/trustdb.gpg: trustdb created
gpg: key CA6CDFB2: public key "FreeBSD Security Officer " imported
gpg: key FF8AE305: public key "core-secretary@FreeBSD.org" imported
...edited...
gpg: key D069F2A0: duplicated user ID detected - merged
gpg: key D069F2A0: public key "Thomas Abthorpe " imported
gpg: Total number processed: 262
gpg: w/o user IDs: 1
gpg: imported: 261 (RSA: 36)
gpg: no ultimately trusted keys found

With the keys imported I verify the file I downloaded.

# gpg --verify freebsd-update-upgrade.tgz.asc freebsd-update-upgrade.tgz
gpg: Signature made Fri Nov 16 09:01:38 2007 EST using DSA key ID CA6CDFB2
gpg: Good signature from "FreeBSD Security Officer "
gpg: WARNING: This key is not certified with a trusted signature!
gpg: There is no indication that the signature belongs to the owner.
Primary key fingerprint: C374 0FC5 69A6 FBB1 4AED B131 15D6 8804 CA6C DFB2

Note I need to generate my own key and sign the FreeBSD Security Officer's key with my generated key if I want to avoid GPG's warnings, i.e.:

gpg --gen-key
gpg --sign-key security-officer@FreeBSD.org

Now I am ready to proceed with the upgrade.

# tar -xf freebsd-update-upgrade.tgz
# sh freebsd-update.sh -f freebsd-update.conf -r 7.0-BETA3 upgrade
Looking up update.FreeBSD.org mirrors... 1 mirrors found.
Fetching public key from update1.FreeBSD.org... done.
Fetching metadata signature for 7.0-BETA2 from update1.FreeBSD.org... done.
Fetching metadata index... done.
Fetching 2 metadata files... done.
Inspecting system... done.

The following components of FreeBSD seem to be installed:
kernel/generic src/base src/bin src/cddl src/contrib src/crypto src/etc
src/games src/gnu src/include src/krb5 src/lib src/libexec src/release
src/rescue src/sbin src/secure src/share src/sys src/tools src/ubin
src/usbin world/base world/dict world/doc world/games world/info
world/manpages world/proflibs

The following components of FreeBSD do not seem to be installed:
src/compat world/catpages

Does this look reasonable (y/n)? y

Fetching metadata signature for 7.0-BETA3 from update1.FreeBSD.org... done.
Fetching metadata index... done.
Fetching 1 metadata patches. done.
Applying metadata patches... done.
Fetching 1 metadata files...
Inspecting system... done.
Preparing to download files... done.
Fetching 1289 patches.....10....20....30....40....50....60....70....80....90....
...edited...
Applying patches... done.
Fetching 329 files... done.

The following files will be removed as part of updating to 7.0-BETA3-p0:
/etc/pf.conf
/usr/share/doc/es_ES.ISO8859-1/books/handbook/LEGALNOTICE.html
/usr/share/doc/fr_FR.ISO8859-1/books/handbook/x20872.html
/usr/share/doc/fr_FR.ISO8859-1/books/handbook/x20918.html
/usr/share/doc/fr_FR.ISO8859-1/books/handbook/x21123.html
/usr/share/examples/etc/pf.conf
/usr/src/etc/pf.conf

The following files will be added as part of updating to 7.0-BETA3-p0:
/boot/kernel/if_zyd.ko
/boot/kernel/if_zyd.ko.symbols
...edited...
/usr/share/examples/pf/pf.conf
/usr/src/share/examples/pf/pf.conf

The following files will be updated as part of updating to 7.0-BETA3-p0:
/bin/ps
/boot/kernel/3dfx.ko
...edited...
/usr/src/usr.sbin/wpa/wpa_supplicant/driver_freebsd.c
/var/named/etc/namedb/named.root

# sh freebsd-update.sh -f freebsd-update.conf install
Installing updates...
Kernel updates have been installed. Please reboot and run
"freebsd-update.sh install" again to finish installing updates.
# shutdown -r now

After a reboot I run the following.

# sh freebsd-update.sh -f freebsd-update.conf install
Installing updates... done.
# shutdown -r now

After a second reboot the system is completely upgraded.

$ uname -a
FreeBSD myhost.mydomain.com 7.0-BETA3 FreeBSD 7.0-BETA3 #0:
Fri Nov 16 22:20:33 UTC 2007
root@logan.cse.buffalo.edu:/usr/obj/usr/src/sys/GENERIC i386

That's excellent. The whole process took only a few minutes.

Tuesday, November 20, 2007

Network Monitoring: How Far?

In my January post The Revolution Will Be Monitored and elsewhere I discuss how network monitoring is becoming more prevalent, whether we like it or not. When I wrote my first book I clearly said that you should collect as much data as you can, given legal, political, and technical means because that approach gives you the best chance to detect and respond to intrusions. Unfortunately, I did not provide any clear guidance for situations where I think monitoring might not be appropriate. While this is by no means a political blog, I would not want my NSM approach to be taken as justification for monitoring and retaining every electronic transaction, especially beyond the security realm.

In that spirit I would like to point out three recent stories which highlight some of the contemporary problems I see with electronic monitoring.

First is Boeing bosses spy on workers. From the story:

Within its bowels, The Boeing Co. holds volumes of proprietary information deemed so valuable that the company has entire teams dedicated to making sure that private information stays private.

One such team, dubbed "enterprise" investigators, has permission to read the private e-mails of employees, follow them and collect video footage or photos of them. Investigators can also secretly watch employee computer screens in real time and reproduce every keystroke a worker makes, the Seattle P-I has learned...

"Employees should understand that the law generally gives employers broad authority to conduct surveillance, whether through e-mail, video cameras or other forms of tracking, including off the job in many cases."

The law grants companies the right to protect themselves from employees who break the law, such as by embezzling money or using the company warehouse to run a drug-smuggling ring.

The problem, [Ed] Mierzwinski [consumer program director at the federation of Public Interest Research Groups] said, is when companies use the surveillance tactics available to them to root out whistle-blowers.

"We need greater whistle-blower protections," he said. But, "if you're using the company's resources and you think it's protected because you're using Hotmail, think again."


My first point on this story is that I have never advocated NSM as a means to combat fraud, waste, and abuse by employees, let alone whistle-blowers. I have almost exclusively focused on external threats. I say let legal and human resources look for non-security policy violations.

My second point on this story is that I think the operative word here is surveillance. NSM is not a surveillance methodology. NSM does not advocate identifying a person of interest, then examining all traffic generated by or directed at that person. NSM is more channel- and system-centric. If I am going to conduct network surveillance of any type, I expect legal and human resources tasking. I do not engage in network surveillance for my own security purposes. I conduct NSM.

The next story is Cal-Ore Telecommunications on Solera Networks. This is a blog posting advertising the adoption of a packet capture appliance sold by Solera Networks to the Cal-Ore ISP in California. From the story:

Cal-Ore, a rural telephone company and ISP headquartered in Northern California, has been serving customers for more than 55 years. In order to comply with CALEA requirements, Charles Boening, Cal-Ore’s network manager considered three choices. First, they could do nothing and hope they never received a lawful intercept warrant request. Second, they could contract with a trusted third-party (TTP) that would perform any tapping services and bring them into compliance: at a six-figure price tag with ongoing fees. Or third, they could purchase a Solera DS 1000 from Solera Networks...

“We not only capture traffic that goes to the Internet, we can also use those extra Ethernet ports to capture traffic from other areas of our network,” Boening said...

While not being used to fulfill a warrant, Boening uses the Solera DS 1000 for complete network packet capture and storage. This has become an integral component to network management at Cal-Ore...

“We’ll hear from other providers telling us that we have a customer who is sending out spam,” said Boening. “Before I disconnect that customer, I need to verify it is a legitimate compliant. I use the Solera Networks box to find specific traffic over a period of time and put it into an analyzer, such as WireShark, to determine whether it is junk. If it is, I will then turn off the customer.”


When I read this I thought "This ISP is logging all traffic that customers send to the Internet?" I read their terms of service and found this:

Use of any Cal-Ore Telephone network service constitutes consent to monitoring at all times. If monitoring of any device in the Cal-Ore Telephone network reveals any evidence regarding violation of copyright laws, security regulations or any instance of unauthorized use of any system, this evidence and any other related information, including identification information about the user, can and will be provided to law enforcement officials.

It appears Cal-Ore is relying on the consent exception to the wiretap act to not break Federal law. They could also hope that their activity "is a necessary incident to the rendition of his service or to the protection of the rights or property of the provider of that service" and thereby receive another exception to the wiretap act.

However, California law is a little different. As noted in Applying the Wiretap Act to Online Communications after United States v. Councilman, California is a two-party consent state, meaning that both parties to the communication must give consent in order to make interception of a communication permissible. I am not a lawyer (I may have to rectify that situation at some point), but it sounds like the consent exception is lost when a Cal-Ore user who has not granted consent communicates via IM to any Cal-Ore user.

The third story is actually a set of articles posted by The Baltimore Sun about the National Security Agency and "cyber security." A slightly more recent article called In focus: Targeting Internet terror offers a few items of interest:

President Bush quietly announced yesterday his plans to launch a program targeting terrorists and others who would seek to attack the United States via the Internet, according to lawmakers and budget documents.

Bush requested $154 million in preliminary funding for the initiative, which current and former government officials say is expected to become a seven-year, multibillion-dollar program to track threats in cyberspace on both government and private networks...

At the White House, spokesman Sean Kevelighan would say only that the money would be used for "increased monitoring capabilities, as well as to increase the security of our networks."


I'm interested in this article because it and previous stories hint that the government might monitor private networks for security purposes. This would be quite a step if true.

Monitoring remains a hot topic, so I plan to keep my eye on these issues going forward.

Wednesday, November 14, 2007

Analyzing Protocol Hopping Covert Channel Tool

I enjoy analyzing covert channels, although my skills are far inferior to someone like Steven Murdoch. However, today via Packetstorm I learned of Protocol Hopping Covert Channel Tool by Steffen Wendzel. He wrote a text file describing his thoughts behind the tool called Protocol Hopping Covert Channels. Quoting the paper:

This paper describes a new way to implement covert channels. This is done by changing the protocol of the tunnel while the tunnel exists and even change the protocol on a randomized way without restarting the tunnel or reconnecting to the tunnel. A simple proof of concept tool called 'phcct' (protocol hopping covert channel tool) also known as 'takushi' (what is japanese for taxi) is available on my website http://www.doomed-reality.org. phcct implements only one (the easiest) version of such a randomized protocol hopping covert channel.

As soon as I read this I thought "this is so different from normal traffic, it will be easy to identify." I know that is true for manual inspection of traffic. I am not sure how automated tools would deal with it. The paper continues:

Do not forget the reason why doing this: It is to be stealth [sic]. Even if _one_ of the protocols you are using is recorded, the monitoring system will not collect ALL packets of ALL protocols in a network. This simply is a too huge amount of data. And yes, it makes the forensic analysis of network traffic much harder if there are multiple protocols used for a covert channel.

Apparently the author does not know about NSM or Sguil. Assuming you are performing this protocol hopping from a host on an enterprise network to a host on the Internet, a NSM sensor monitoring your Internet gateway will see and record this traffic.

I decided to give the author's proof of concept tool, phcct, a try. I compiled it statically on my Ubuntu laptop.

$ gcc -O -o phcct_s -fstack-protector-all -W -Wall -Wshadow -g
-ggdb *.c -lpthread -static

Next I copied the binary to a VM running a Ubuntu 7.10 as a live CD/.iso. I used this static version because I noticed the Ubuntu live CD did not have the required libraries to compile from source, but run as a statically compiled binary it worked fine.

Now I started phcct on each workstation and hit return once each side was running. My laptop is neely, 192.168.2.101.

root@neely:~/phcct# ./phcct_s -a 192.168.2.115
starting phcct (a.k.a. takushi) ...
please press return if the peer is setup up.
connecting ...
connected via http
connected via ftp-data
connected via plain proto
waiting for local connection on port 9999 ...

My VM is ubuntu, 192.168.2.115.

root@ubuntu:/home/analyst# ./phcct_s -a 192.168.2.101
starting phcct (a.k.a. takushi) ...
please press return if the peer is setup up.
connecting ...
connected via http
connected via ftp-data
connected via plain proto
waiting for local connection on port 9999 ...

Once each side of the tunnel was activated, I used Netcat on each system to connect to port 9999 on localhost. First, neely:

richard@neely:~$ nc -v localhost 9999
localhost [127.0.0.1] 9999 (?) open

Second, ubuntu:

analyst@ubuntu:~$ nc -v localhost 9999
localhost [127.0.0.1] 9999 (?) open

Now I was ready to send traffic. For example, I typed this on neely:

This is traffic from neely to ubuntu.

and it appeared on ubuntu:

This is traffic from neely to ubuntu.

Then I did the reverse. I repeated this cycle four times, for a total of five exchanges or ten total messages. When done I exited each Netcat session.

During this process I captured the traffic using Tshark. You can download it here.

I prefer to start the analysis by looking at session data. Here is what Argus 2.x thought of the traffic.

$ ra -nn -L0 -A -r ph.1.arg -s saddr daddr sport dport proto pkts bytes
SrcAddr DstAddr Sport Dport Type SrcPkt DstPkt SAppBytes DAppBytes
192.168.2.115 192.168.2.101.43598 2510 tcp 6 4 96 0
192.168.2.115 192.168.2.101.43198 80 tcp 5 3 281 0
192.168.2.115 192.168.2.101.52158 20 tcp 6 4 96 0
192.168.2.101 192.168.2.115.49586 80 tcp 4 4 281 0
192.168.2.101 192.168.2.115.45106 20 tcp 3 3 0 0
192.168.2.101 192.168.2.115.50200 2510 tcp 7 7 192 0

You can see the tool using destination ports 2510, 80 and 20. There are six sessions although one of them is empty. The five active sessions correspond to our five conversations.

Let's look at the traffic on each using Tcpflow. In the ls output I omit the first few fields for space purposes.

taosecurity:/home/analyst$ tcpflow -r ph.1.lpc
taosecurity:/home/analyst$ ls -al 192*
281 Nov 14 13:59 192.168.002.101.49586-192.168.002.115.00080
192 Nov 14 13:59 192.168.002.101.50200-192.168.002.115.02510
281 Nov 14 13:59 192.168.002.115.43198-192.168.002.101.00080
96 Nov 14 13:59 192.168.002.115.43598-192.168.002.101.02510
96 Nov 14 13:59 192.168.002.115.52158-192.168.002.101.00020

Notice the file sizes match the byte counts seen above. Here are the contents of each. Note the use of cat -tev to

taosecurity:/home/analyst$ cat -tev 192.168.002.101.49586-192.168.002.115.00080
GET / HTTP/1.1^M$
Host: google.de^M$
User-Agent: Mozilla/5.0^M$
Accept: text/xml^M$
Accept-Language: en-us;q=0.5,en;q=0.3^M$
Accept-Encoding: gzip,deflate^M$
Accept-Charset: ISO-8859-1,utf-8^M$
Keep-Alive: 300^M$
Connection: keep-alive^M$
Cookie: GPC=2FW=0:This is traffic from neely to ubuntu.$
^M$
^M$
taosecurity:/home/analyst$ cat -tev 192.168.002.101.50200-192.168.002.115.02510
^A^B^C0 FW=0:This is traffic from neely to ubuntu.$
^A^B^C1 FW=0:This is traffic from neely to ubuntu.$
^A^B^C3 FW=0:This is traffic from neely to ubuntu.$
^A^B^C4 FW=0:This is traffic from neely to ubuntu.$

taosecurity:/home/analyst$ cat -tev 192.168.002.115.43198-192.168.002.101.00080
GET / HTTP/1.1^M$
Host: google.de^M$
User-Agent: Mozilla/5.0^M$
Accept: text/xml^M$
Accept-Language: en-us;q=0.5,en;q=0.3^M$
Accept-Encoding: gzip,deflate^M$
Accept-Charset: ISO-8859-1,utf-8^M$
Keep-Alive: 300^M$
Connection: keep-alive^M$
Cookie: GPC=4FW=0:This is traffic from ubuntu to neely.$
^M$
^M$

taosecurity:/home/analyst$ cat -tev 192.168.002.115.43598-192.168.002.101.02510
^A^B^C0 FW=0:This is traffic from ubuntu to neely.$
^A^B^C1 FW=0:This is traffic from ubuntu to neely.$

taosecurity:/home/analyst$ cat -tev 192.168.002.115.52158-192.168.002.101.00020
^A^B^C2 FW=0:This is traffic from ubuntu to neely.$
^A^B^C3 FW=0:This is traffic from ubuntu to neely.$

At the end of the day we have messages exchanged using one real protocol (HTTP, with payload in the cookie field) and two pseudo-protocols (raw traffic on port 2510 TCP and attempted simulated FTP data traffic). The FTP data traffic isn't simulated properly because the SYN segments go to port 20 TCP.

$ tcpdump -n -t -r ph.1.lpc port 20
reading from file ph.1.lpc, link-type EN10MB (Ethernet)
IP 192.168.2.115.52158 > 192.168.2.101.20:
S 2100176842:2100176842(0) win 5840
IP 192.168.2.101.20 > 192.168.2.115.52158:
S 3955486862:3955486862(0) ack 2100176843 win 5792
IP 192.168.2.115.52158 > 192.168.2.101.20: . ack 1 win 183

IP 192.168.2.101.45106 > 192.168.2.115.20:
S 3970907263:3970907263(0) win 5840
IP 192.168.2.115.20 > 192.168.2.101.45106:
S 671130558:671130558(0) ack 3970907264
IP 192.168.2.101.45106 > 192.168.2.115.20: . ack 1 win 1460
IP 192.168.2.115.52158 > 192.168.2.101.20: P 1:49(48) ack 1 win 183
IP 192.168.2.101.20 > 192.168.2.115.52158: . ack 49 win 1448
IP 192.168.2.115.52158 > 192.168.2.101.20: P 49:97(48) ack 1 win 183
IP 192.168.2.101.20 > 192.168.2.115.52158: . ack 97 win 1448
IP 192.168.2.115.52158 > 192.168.2.101.20: F 97:97(0) ack 1 win 183
IP 192.168.2.115.20 > 192.168.2.101.45106: F 1:1(0) ack 1 win 181
IP 192.168.2.101.45106 > 192.168.2.115.20: F 1:1(0) ack 2 win 1460
IP 192.168.2.101.20 > 192.168.2.115.52158: F 1:1(0) ack 98 win 1448
IP 192.168.2.115.20 > 192.168.2.101.45106: . ack 2 win 181
IP 192.168.2.115.52158 > 192.168.2.101.20: . ack 2 win 183

Real active FTP data traffic comes from port 20 TCP.

Can this technique be improved? Sure. Is it tough to analyze? Possibly. If you use a packet-by-packet approach, you can see what's happening. For example, here are a few packets containing payloads. Notice the use of the Tshark display filter using the -R switch.

richard@neely:~$ tshark -r ph.1.lpc -x -R 'tcp.len >= 20'
19 62.570303 192.168.2.101 50200 192.168.2.115 2510 TCP 50200 > 2510
[PSH, ACK] Seq=1 Ack=1 Win=5840 Len=48 TSV=4768072 TSER=3038481

0000 00 13 02 4c 30 2d 00 13 02 4c 30 2d 08 00 45 00 ...L0-...L0-..E.
0010 00 64 24 88 40 00 40 06 8f e3 c0 a8 02 65 c0 a8 .d$.@.@......e..
0020 02 73 c4 18 09 ce eb e8 a2 75 28 9d e1 d0 80 18 .s.......u(.....
0030 05 b4 e3 5b 00 00 01 01 08 0a 00 48 c1 48 00 2e ...[.......H.H..
0040 5d 11 01 02 03 30 20 46 57 3d 30 3a 54 68 69 73 ]....0 FW=0:This
0050 20 69 73 20 74 72 61 66 66 69 63 20 66 72 6f 6d is traffic from
0060 20 6e 65 65 6c 79 20 74 6f 20 75 62 75 6e 74 75 neely to ubuntu
0070 2e 0a ..

21 70.543503 192.168.2.115 43598 192.168.2.101 2510 TCP 43598 > 2510
[PSH, ACK] Seq=1 Ack=1 Win=5856 Len=48 TSV=3055661 TSER=4752430

0000 00 13 02 4c 30 2d 00 13 02 4c 30 2d 08 00 45 00 ...L0-...L0-..E.
0010 00 64 13 0f 40 00 40 06 a1 5c c0 a8 02 73 c0 a8 .d..@.@..\...s..
0020 02 65 aa 4e 09 ce 7d 50 49 4a eb ba 86 09 80 18 .e.N..}PIJ......
0030 00 b7 70 7a 00 00 01 01 08 0a 00 2e a0 2d 00 48 ..pz.........-.H
0040 84 2e 01 02 03 30 20 46 57 3d 30 3a 54 68 69 73 .....0 FW=0:This
0050 20 69 73 20 74 72 61 66 66 69 63 20 66 72 6f 6d is traffic from
0060 20 75 62 75 6e 74 75 20 74 6f 20 6e 65 65 6c 79 ubuntu to neely
0070 2e 0a ..

23 84.876175 192.168.2.101 50200 192.168.2.115 2510 TCP 50200 > 2510
[PSH, ACK] Seq=49 Ack=1 Win=5840 Len=48 TSV=4773648 TSER=3053597

0000 00 13 02 4c 30 2d 00 13 02 4c 30 2d 08 00 45 00 ...L0-...L0-..E.
0010 00 64 24 89 40 00 40 06 8f e2 c0 a8 02 65 c0 a8 .d$.@.@......e..
0020 02 73 c4 18 09 ce eb e8 a2 a5 28 9d e1 d0 80 18 .s........(.....
0030 05 b4 92 56 00 00 01 01 08 0a 00 48 d7 10 00 2e ...V.......H....
0040 98 1d 01 02 03 31 20 46 57 3d 30 3a 54 68 69 73 .....1 FW=0:This
0050 20 69 73 20 74 72 61 66 66 69 63 20 66 72 6f 6d is traffic from
0060 20 6e 65 65 6c 79 20 74 6f 20 75 62 75 6e 74 75 neely to ubuntu
0070 2e 0a ..

25 88.388511 192.168.2.115 43598 192.168.2.101 2510 TCP 43598 > 2510
[PSH, ACK] Seq=49 Ack=1 Win=5856 Len=48 TSV=3060172 TSER=4770065

0000 00 13 02 4c 30 2d 00 13 02 4c 30 2d 08 00 45 00 ...L0-...L0-..E.
0010 00 64 13 10 40 00 40 06 a1 5b c0 a8 02 73 c0 a8 .d..@.@..[...s..
0020 02 65 aa 4e 09 ce 7d 50 49 7a eb ba 86 09 80 18 .e.N..}PIz......
0030 00 b7 19 c7 00 00 01 01 08 0a 00 2e b1 cc 00 48 ...............H
0040 c9 11 01 02 03 31 20 46 57 3d 30 3a 54 68 69 73 .....1 FW=0:This
0050 20 69 73 20 74 72 61 66 66 69 63 20 66 72 6f 6d is traffic from
0060 20 75 62 75 6e 74 75 20 74 6f 20 6e 65 65 6c 79 ubuntu to neely
0070 2e 0a ..

27 97.214303 192.168.2.101 49586 192.168.2.115 80 HTTP GET / HTTP/1.1

0000 00 13 02 4c 30 2d 00 13 02 4c 30 2d 08 00 45 00 ...L0-...L0-..E.
0010 01 4d e2 06 40 00 40 06 d1 7b c0 a8 02 65 c0 a8 .M..@.@..{...e..
0020 02 73 c1 b2 00 50 ec 6f 64 f6 27 b7 a8 06 80 18 .s...P.od.'.....
0030 05 b4 16 d2 00 00 01 01 08 0a 00 48 e3 1d 00 2e ...........H....
0040 5d 0f 47 45 54 20 2f 20 48 54 54 50 2f 31 2e 31 ].GET / HTTP/1.1
0050 0d 0a 48 6f 73 74 3a 20 67 6f 6f 67 6c 65 2e 64 ..Host: google.d
0060 65 0d 0a 55 73 65 72 2d 41 67 65 6e 74 3a 20 4d e..User-Agent: M
0070 6f 7a 69 6c 6c 61 2f 35 2e 30 0d 0a 41 63 63 65 ozilla/5.0..Acce
0080 70 74 3a 20 74 65 78 74 2f 78 6d 6c 0d 0a 41 63 pt: text/xml..Ac
0090 63 65 70 74 2d 4c 61 6e 67 75 61 67 65 3a 20 65 cept-Language: e
00a0 6e 2d 75 73 3b 71 3d 30 2e 35 2c 65 6e 3b 71 3d n-us;q=0.5,en;q=
00b0 30 2e 33 0d 0a 41 63 63 65 70 74 2d 45 6e 63 6f 0.3..Accept-Enco
00c0 64 69 6e 67 3a 20 67 7a 69 70 2c 64 65 66 6c 61 ding: gzip,defla
00d0 74 65 0d 0a 41 63 63 65 70 74 2d 43 68 61 72 73 te..Accept-Chars
00e0 65 74 3a 20 49 53 4f 2d 38 38 35 39 2d 31 2c 75 et: ISO-8859-1,u
00f0 74 66 2d 38 0d 0a 4b 65 65 70 2d 41 6c 69 76 65 tf-8..Keep-Alive
0100 3a 20 33 30 30 0d 0a 43 6f 6e 6e 65 63 74 69 6f : 300..Connectio
0110 6e 3a 20 6b 65 65 70 2d 61 6c 69 76 65 0d 0a 43 n: keep-alive..C
0120 6f 6f 6b 69 65 3a 20 47 50 43 3d 32 46 57 3d 30 ookie: GPC=2FW=0
0130 3a 54 68 69 73 20 69 73 20 74 72 61 66 66 69 63 :This is traffic
0140 20 66 72 6f 6d 20 6e 65 65 6c 79 20 74 6f 20 75 from neely to u
0150 62 75 6e 74 75 2e 0a 0d 0a 0d 0a buntu......

This tool demonstrates the importance of a few NSM concepts. First, intruders are unpredictable. (Remember I use the term "intruder" to mean anyone doing something you don't like on your network, i.e., any policy violater. Second, by collecting everything and investigating once you have indicators, you can find activity not observed by existing inspection and blocking systems. Third, there is no substitute for full content. Statistics are nice, sessions are better, but only full content reveals what's really happening. Even session tools can be fooled or misguided, or at least have their output subject to misinterpretation.

I expect to see additional iterations of this tool and technique.