Compliance Poll Analysis

A while ago, I did this quick poll on regulatory compliance – and here is the result analysis.

The “winners” are:

1. “No brainer” winner: PCI DSS with 59% – it is indeed ‘forevah’
2. ISO2700x is a surprising silver medalist with 36% (more than half of PCI?)
3. ITIL holds an even-more-surprising 3rd spot with 19% – at nearly 1/2 of ISO again
4. A bunch of supposedly “cool” regs share #4 spot with 12%-15%: FISMA, HIPAA, SOX
5. …and the same percentage (15%) is held by “I don’t care about that compliance sh*t”

Notable write-ins were:

• NIST (in general, I guess beyond just FISMA)
• Red Flag (financial)
• CFATS (?)
• PHIPA, MFIPPA  (?)
• EU Data Privacy laws

What does it tell us? What can we hypothesize based on our totally unscientific compliance poll?

• All this talk about PCI DSS impacting security at large is very real – now and likely in the near future. I might argue with Josh about whether the impact is positive or negative – but it is HUGE. It definitely goes way beyond retail and ecommerce.
• ISO27001 came back to life somehow. That’s probably a good thing….
• Not sure what the lesson from ITIL being #3 is – that folks from UK read my blog? :-)
• Finally, I think the people who don’t care about compliance split into two opposite camps: people who don’t EVEN CARE ABOUT COMPLIANCE (much less security) and people who care about security and operational excellence which gives them compliance [not for free, mind you!] So, 19% covers both of these camps.

Any other thoughts?
Possible related posts:

• All posts on polls and their analysis

Silly Compliance Poll

OK, some of you would say that I have weird hobbies, like writing books about  PCI DSS.

In any case, just for fun I am running this poll on compliance here. Please respond – violently is OK, if compliance brings this up in you :-)

Enjoy!

Possibly related posts:

• Old posts with fun polls and their analysis.

SANS Top 5 Essential Log Reports Update!

Some of you remember the project started at SANS Log Management Summit 2006 called “SANS Top 5 Essential Log Reports.” You can still grab the old document here [PDF]. Recently, I volunteered to create a 2010 version of SANS Top 5 Log Reports.
With help from others [to be credited when the project is complete, but definitely with help from somebody named MJR :-)] and some research into past efforts, I have identified the report types and specific examples below as candidates for a new Top 7 Essential Log Reports list – and now I need your help!
Initially, I wanted people to vote for 5 out of the 7 candidates, but let’s do it differently: just comment on the list below (blog comments, your own blogs – please post a li here, email, twitter, etc) or suggest your own most useful, most popular log reports or even report categories. There is no reason why we can’t have Top 7 or Top N>7 useful log reports :-)

NEW PROPOSED Top 7 Essential Log Reports

Top Log Report Candidate 1. Authentication and Authorization Reports
a. Login Failures and Successes
b. Attempts to gain unauthorized access through existing accounts
c. Privileged account access (success, failure)
d. VPN Authentication and other remote access (success, failure)
e. Please add more reports you find useful!
Top Log Report Candidate 2. Change Reports
a. Addition/Changes/Deletions to Users, Groups and Services
b. Change to configurations
c. Application installs and Updates
d. Please add more reports you find useful!
Top Log Report Candidate 3. Network Activity Reports [used to be called “Suspicious or Unauthorized Network Traffic Patterns” in the old Top 5 list]
a. Top Internal Systems Connecting Through Firewall // Summary of Outbound Connections
b. Network Services Transiting A Firewall
c. Top Largest File Transfers Through the Firewall
d. Internal Systems Using Many Different Protocols/Ports
e. Top Internal Systems With NIDS Alerts
f. Proxy Report on File Uploads
g. Please add more reports you find useful!
Top Log Report Candidate 4. Resource Access Reports
a. File
i. Failed File or Resource Access Attempts
b. Database
i. Top Database Users
ii. Summary of Query Types
iii. SELECT Data Volume
iv. All Users Executing INSERT/DELETE Commands
v. Database Backups
c. Email
i. Top Internal Email Addresses by Volume of Messages
ii. Top Attachment Types with Sizes
iii. Top Internal Systems Sending Spam // Top Internal Systems Sending Email NOT Through Mail Server
c. Please add more reports you find useful!
Top Log Report Candidate 5. Malware Activity Reports
a. Top systems with anti-malware events
b. Detect-only events from anti-malware tools (“leave-alones”)
c. Anti-virus protection failures by type
d. Internal malware connections (all sources)
e. Please add more reports you find useful!
Top Log Report Candidate 6. “Various FAIL”
a. Critical Errors
b. Backup failures
c. Capacity / Limit Exhaustion
d. System and Application Starts, Shutdowns and Restarts
e. Please add more reports you find useful!
Top Log Report Candidate 7. Analytic Reports  [Mostly Using “Never Before Seen” (NBS) aka “NEW Type/Object” Analysis]
a. NEW (NBS) IDS/IPS Alert Types
b. NEW (NBS) Log Entry Types
c. NEW (NBS) Users Authentication Success
d. NEW (NBS) Internal Systems Connecting Through Firewall
e. NEW (NBS) Ports Accessed
f. NEW (NBS) HTTP Request Types
g. NEW (NBS) Query Types on Database
h. Please add more NBS or other analytic reports you find useful!

So, please help this project by commenting via whatever means!!!

BTW, I think I perused all the previous efforts to distill log reports (such as this one), but feel free to point me to such things as well.

Finally, if you are a SIEM or log management vendor, please consider supporting the resulting reports in your products – after they are finalized by the community and released by SANS.

Possibly related posts:

• Simple Log Review Checklist

Logging Poll #9 Analysis: Log Security

This is the analysis of my last poll; the responses are here and also below.

First, the most obvious conclusion: people still don't care much about log security; I am saying that since this was BY FAR the least popular of my polls. Only 24 people responded, so everything below is pretty unscientific :-)  A good way to explain it: look at the recent media? Do these people care about their key business data and their customer data security? Nope. So, how on Earth do you make them care about securing their log data?

Second,  it is entirely unsurprising that 83% of respondents want "Authenticated access to log server." In fact, I'd opine that 100% of people want authenticated access to any of their servers :-) But, this was my "red herring" to set the baselines for the rest of the questions... 

However, this is where the buck stops: other security measures are notably less popular.

Third, "Logging all access to logs" is my favorite and I am happy to see it reported as popular. But do you really do it?  Do you log access to log server OR access to actual logs? Think about it... I think a lot of people who do the latter still answered "yes" to this one.

Fourth,  "Reliable / acknowledged network transfer of log data" and "Encryption of log data in transit " are two true "no-brainer" security features; they took the next spot at 45% and 50% of those who answered. They are simple, they are easy, they make  sense - and, obviously, they don't make logs entirely secure so you need to do more. Why only 50%? Where is THE OTHER 50%?!

Fifth, "all things crypto" are below 40%. "Cryptographic hashing of stored logs", "Cryptographic signing of stored log data" and "Encryption of stored log data" all hover at around 30%. I attribute them to general disregard of log security AND reliance on "system security" (separate server, etc) over "data security" measures for log protection.

Finally, I am embarrassed to say that I missed  the obvious security measure "Separate server for logging, not accessible from the Internet;" one of my readers added this using "Other security measures" choice. Indeed, this is a good point - and a good idea to do it. Another option mention there was "Destroy old logs." Amen to that too!

Possibly related posts:

• Top 11 Reasons to Secure and Protect Logs
• All other polls and their analysis

Poll #9 How Much Log Security Do You Need?

Yes! YES! Y-E-S! :-)

My next logging poll is out - with it I set out to figure out the old mystery of mine, why people don't protect their log data (e.g. see this lamentation "Top 11 Reasons to Secure and Protect Your Logs")

Vote away! As always, results will be posted.

Past polls and analysis are all here. Enjoy!

Logging Poll #8 Analysis: Needed Log Context

In my poll #8, I  asked a question: what information is most important when analyzing a particular log record. Live results are here and final count is also below:

What can we conclude?

First, good documentation never hurts :-) - indeed, the most popular information to look for when facing a new log record is documentation on what it means. While some software vendors are great in this regard, many other don't bother documenting their logs or document them only when customers complain.

Second, I was not sure that the second popular choice would be "Other logs from about the same time (this and other systems)."  This strongly points at huge value of cross-device log analysis (see this recent log entry on that),  where all the logs are consolidated and analyzed together (it goes without saying that time is synchronized OR at least corrected across those logs). Indeed, if you are confused about a log and documentation is not available, reviewing "what else was/is going on?" is smart. Trusting log time stamps across many systems is also key for that.

Third, having IP addresses in logs is great, but human-readable names are better: IPs in logs needs to be mapped to DNS or Netbios names. Indeed, given that often such names reveal where the system is, who might own it, what its function is, etc this information is not just a mapping, but true log information enrichment.

Fourth, so, what's next? The above 3 top responses are indeed universally useful, but the next choice digs deeper: flows, packets, connections and other network information does complement logs and is often studied in combination with logs (e.g. see a strange log entry then go see who connected to the system at that time or where the system itself connected to).

Fifth, next comes a group of pretty much everything else: other logs from the same system, logs about the same system as well as loosely defined 'similar' log entries. These come handy, but are not top choices. In fact,  from this I conclude that a lot of additional context information is needed to make sense of a confusing log entry.

Sixth, what was surprising? I thought that identity lookups (e.g. IP to real name or other user identity information) would score higher.  I also suspect that people were confused by "logs ABOUT the same systems" (what I meant is, for example, use firewall logs that mention the system which log we are now analyzing) and this should score higher.

Seventh, anything fun in the "Other" category? Yes, there were a few insightful ones: first, results of a Google search (supposedly for the info from the log entry in question)! Very true indeed. Also named were logs from the same daemon/program (how can I miss it?),  logs from previous incidents and information on the logging system owner.  All very useful indeed. Thanks for good ideas!

Finally, a brief message to people that work for a certain log-related vendor of ill repute who keep polluting my polls: if I catch you, I will kick you in the butt :-) Or, better, I will hammer you with a big and heavy log (you know, the wooden kind) over your miniscule heads ...

 

Past logging polls and their analysis:

Poll #7 "What tools do you use for Windows Event Log collection?" (analysis)

Poll #6 "Which Logs Do You LOOK At?" (analysis)

Poll #5 "What are your top challenges with logs?" (analysis)

Poll #4 "Who looks at logs in your organization?" (analysis)

Poll #3 "What do you do with Logs?" (analysis)

Poll #2 "Why collect logs?" (analysis)

Poll #1 "Which logs do you collect?" (analysis)

Poll #8 Log Analysis Context

So, my next poll is up - and it is fun: Which of the types of information below are most useful when trying to make sense of a log entry?

Vote here!

Past polls:

Poll #7 "What tools do you use for Windows Event Log collection?" (analysis)

Poll #6 "Which logs do you LOOK at?" (analysis)

Poll #5 "What are your top challenges with logs?" (analysis)

Poll #4 "Who looks at logs in your organization?" (analysis)

Poll #3 "What do you do with logs?" (analysis)

Poll #2 "Why collect logs?" (analysis)

Poll #1 "Which logs do you collect?" (analysis)

Windows Log Collection Poll Analysis

Now, my latest poll ("What tools do you use for Windows Event Log Collection and Analysis") was pretty popular (157 responses) and controversial as well; let's analyze it. The results are here and below as well.

 

So, what catches your eye first? Despite the fact that I was trying hard to list most of the tools that collect Windows logs known to humankind (and certainly, I thought I included ALL of the popular ones), response 'Other' is #1 by popularity. Now, the 'Other' option had a write-in field that is not visible online, but accessible to poll owner (i.e. me). What  dark and mysterious tool hides in there under the guise of 'Other'?  Well, this is where the controversy lies: out of 37 people who chose 'Other', 15 wrote in 'sp1unk.' Now, given that the Windows version was released only a couple of days before my poll, I refuse to believe that.

Second, as one can guess, using Snare agent for converting Windows event logs into syslog is the next popular (after 'Other'). This is definitely what I expected. Snare is a safe choice that everybody knows (but it is an agent)

Third, 'voting "no"' (i.e. 'We don't collect windows logs centrally') is next; in fact, it is not statistically different from the previous choice: Snare. This reflects the sad reality of Windows logging: people just do not collect them and then, when needed , they try to desperately reach for the logs stored on each server (and, obviously, often not finding them there). Will Windows 2008 (which does have its own WS-based log centralization system) change that? Probably!

Fourth, despite the fact that everybody hates agents, remote Windows collectors, such as ProjectLASSO, are less popular. In fact, most people who use a remote collector, use a commercial (WMI- or RPC-based) remote collector from their SIEM or log management vendor.

Fifth, OSSEC rises above the crowd of other remaining tools. This is definitely an interesting discovery as well.

Finally, on a somewhat humorous note, if one combines "We don't collect Windows logs centrally", "We ignore Windows logs" and "We are waiting for Windows to support syslog natively", the total count will reach 35% times and will exceed any other option, including 'Other', Snare, etc.

So, this poll reflects a sad state of affairs with Windows logging; let's hope that W2k8 will change that...

Technorati tags: logs, log management, logging, polls

Quick and Fun Poll on Behavior Change

While I am analyzing my previous poll, here is a quick and fun one: do we change our behavior when monitored? Vote away!

Poll #7: What tools do you use for Windows Event Log collection?

My next fun logging poll is here - please vote! It is about tools for centralized collection of Windows Event Log from servers and other systems. One of the somewhat surprising discoveries from my previous poll was that few people look at Windows logs; this poll drills down into it.

UPDATE: just looked at the results collected so far, and I would like to say this: why - oh - why some people want to turn an honest research effort into a vendor war? Ye bastards, :-) you know who you are ...

Past logging polls and their analysis:

Poll #6 "Which Logs Do You LOOK At?" (analysis)

Poll #5 "What are your top challenges with logs?" (analysis)

Poll #4 "Who looks at logs in your organization?" (analysis)

Poll #3 "What do you do with Logs?" (analysis)

Poll #2 "Why collect logs?" (analysis)

Poll #1 "Which logs do you collect?" (analysis)

Technorati tags: logs, logging, polls, log management

Logging Poll #6 "Which Logs Do You LOOK At?" Analysis

This poll on looking at logs  poll was relatively popular; lets see what we can learn (live results are also here).

First, what are the top 3 log types that people look at? They are:

1. Unix/Linux server syslog
2. Web server logs
3. Firewall logs

How does that compare with the top 3 log types that people collect (see picture showing results from my previous poll below)?

These are:

1. Unix/Linux server syslog
2. Firewall logs
3. Web server logs

Huh? They are the same - doesn't it just make sense? What are the possibilities here?

a. People only collect the logs they plan to look at, OR

b. People look at logs they collect (duh!).

Strangely, I find a) unlikely; I think most people collect more than they can review and that the incident/issue response and compliance needs drive collection more than review or analysis.

Another observation is that all of the "big 3" log types are useful for security, operations and compliance and not just for security (like NIDS/NIPS logs). Is that why they are so popular?

Second, I was fearful that "I only look at whatever logs needed for the incident/issue investigation" will win. It didn't!!! This to me indicates that proactive log review is not as unpopular as I feared. Good! It is working.

Third, obviously, nobody (well, 4%...) looks at all logs they collect.

Fourth, much more people look at Unix/Linux logs than Windows server logs (factor of 3x); this is not entirely unexpected and my next poll will drill down into this.\

Finally, I am SHOCKED that people don't look at NIDS/NIPS logs (only 11% do). People, what's wrong with you? :-) Why have you deployed those beasts if you don't look at what they produce? Then again, maybe you haven't :-(

Next poll coming up!

Technorati tags: logging, logs, polls

Monthly Blog Round-Up - February 2008

I saw this idea of a monthly blog round-up and I liked it. In general, blogs are a bit "stateless" and a lot of good content gets lost since many people, sadly, only pay attention to what they see today.

So, here is my next monthly "Security Warrior" blog round-up of top 5 popular posts and topics.

1. Finally, one post I wrote this month bumped the "anti-virus saga" from the #1 popular spot: Welcome to the Platform Club! :-) post discusses requirements for a log management platform (and makes fun of some folks in the process ...)
2. Now pushed to the #2 spot, next is the topic of anti-virus efficiency. Here are the posts: Answer to My Antivirus Mystery Question and a "Fun" Story, More on Anti-virus and Anti-malware, Let's Play a Fun Game Here ... A Scary Game, The Original Anti-Virus Test Paper is Here!, Protected but Owned: My Little Investigation, A Bit More on AV  and Closure (Kind of) to the Anti-Virus Efficiency/Effectiveness Saga
3. Next are again my Top11 logging lists:  Top 11 Reasons to Collect and Preserve Computer Logs and  Top 11 Reasons to Look at Your Logs (the third list, Top 11 Reasons to Secure and Protect Your Logs, was not quite that popular - I long argued that, sadly, few people care about log security yet). A new one was also added to the list: Top 11 Reasons to Analyze Your Logs. Check it out!
4. PCI compliance is still all the rage! So, MUST-DO Logging for PCI? post was propelled to a place in my Top5 popular posts list. It discusses the fact that there is no "easy list" of what you MUST do to comply.
5. My logging polls are hot as well. Specifically, the analysis of my newest poll  (Logging Poll #5 "Top Logging Challenges" Analysis) is popular.

See you in March - I will continue to make logs popular, research new log analysis methods and make fun of some people (of course!) :-)  

Possibly related posts / past monthly popular blog round-ups:

• Monthly Blog Round-Up - January 2008  
• Monthly Blog Round-Up - December 2007  
• Monthly Blog Round-Up - November 2007  
• Monthly Blog Round-Up - October 2007  
• Monthly Blog Round-Up - September 2007
• Monthly Blog Round-Up - August 2007

 

Technorati tags: monthly, chuvakin

Poll: What logs do you actually LOOK at?

This is my 6th logging poll (vote here now!)- links to the previous five polls below.

This one is deceptively similar to the #1 below, but it is not. This poll is What logs do you actually LOOK at? and not Which Logs Do You Collect? In other words, are you a log packrat? Are you collecting and never using the log data? You are making a mistake, if you don't.

Past polls:

Poll #5 "What are your top challenges with logs?" (analysis)

Poll #4 "Who looks at logs in your organization?" (analysis)

Poll #3 "What do you do with Logs?" (analysis)

Poll #2 "Why collect logs?" (analysis)

Poll #1 "Which logs do you collect?" (analysis)

Logging Poll #5 "Top Logging Challenges" Analysis

OK, this poll WAS fun! The raw results are here and below:

What do we learn from this? Sadly, this poll was less popular than I hoped, so the results are not as statistically significant. Still, we can draw some fun conclusions from the data.

First, what are the top challenges? It is with great regret :-) that I report that the #1 challenge is exactly the one I thought it would be: We collect logs but don't have time/resources to look at them. Yes, automated "analysis challenge" has only become more of a challenge as we deploy more tools that enable log collection on a massive scale (e.g. 75,000 logs/second). I dare to predict that we will finally have to tackle this one in the next year or two. In fact, this challenge rears it ugly head via another popular response, Lack of log analysis tools, which made Top 5 responses.

Second, even though I didn't have any predictions about the #2 entry, but I was surprised: No way to effectively search all logs is a  very close #2 (obviously, 1 vote is not statistically significant here). Indeed, log searching is an elusive little problem, especially when we want to do it fast and on a large pool of logs. Even though I think we need to search less and discover more, the need to search logs will be with us forever (and, no, I don't think you need a special product just to search logs, Raffy :-))

Third, I am happy to report that this poll shows that we finally broke the back of "the beast" of  not having logs. Responses that point at not having logs (e.g. Logging is not enabled, We don't know what logging we must enable,  etc) are not terribly popular (then again, maybe it is due to self-selection of my enlightened blog readers ...)

Fourth, infrastructure! Specifically, No infrastructure to manage the log volume we have is very popular as well (#4). This proves the point that I used to not take very seriously in the past (by mistake): when megabytes become gigabytes and those flow into terabytes, many things that used to trivial (e.g. moving logs from A to B, saving logs to disk, etc) become grand engineering challenges... Indeed, to manage high volume of logs you need a scalable log management solution (example :-))

Sixth, as I lamented, few care about log security (this counts as laments, I guess).  Secure storage of logs is only bothering a few people. One word: yet! ;-) As of today, stored log hashing + (sometimes!) log transport encryption + (rarely!) encrypted archives are the state of the art.

Next poll is coming up!

Technorati tags: logging, log management, polls

Hey! Vote on a poll!

Hey, hey, hey - why are you ignoring my most fun poll ever? :-) Vote on!

Poll: What are your top challenges with logs and logging?

This poll is especially fun: What are your top challenges with logs and logging? Vote here.

Past polls were:

Poll #4 "Who looks at logs in your organization?" (analysis)

Poll #3 "What Do You Do With Logs?" (analysis)

Poll #2 "Why Collect Logs?" (results so far, my analysis)

Poll #1 "Which Logs Do You Collect?" (analysis)

 

Technorati tags: polls, logs, logging, log management

Take This SANS/LogLogic Log Management Survey!

Here is fun survey on log management, check it out: "How do organizations use their log data? What are their challenges in log data analysis? What are their perceptions versus their practices? Take the third annual SANS/LogLogic Log Management Survey and help us find out. "

Logging Poll #4 "Who Looks at Logs?" Analysis

Time to analyze my final 2007 poll on logs. In it, I asked who actually looks at logs at the organization. Here is what came up: results are here and also included below.

 

What can we conclude from this?

First, a "duh" conclusion is in order! No matter how many times one can utter the word "compliance," logs are still most useful for mundane (one would hope! :-)) system administration. Yes, indeed, sysadmins are the primary consumers of logs - yesterday, today, and - likely! - tomorrow as well.

Second, I am saddened by the fact that application developers have not warmed up to logs, at least no en masse (and not according to this limited poll...). I am guessing when they start thinking of logging when creating their applications, they will be more aware of the fact that you can troubleshoot the applications using logs ...

Third, incident response team showing that low is some kind of fluke, I am sure. Everybody knows that logs are indispensable during incident response (yes, even if only a little logging was enabled or even logging defaults left in place, logs often reveal answers unobtainable via any other mechanisms)

Am I reading too much into this? Hey, maybe I am! :-) Then again, I am a former theoretical physicist - thus, I can explain anything!

Next poll coming soon!

Technorati tags: logging, log management, poll

Poll: Who looks at logs in your organization?

Here is my next poll about logs: Who looks at logs at your organization?

Vote here!

Also, my past polls and analysis are here.

Logging Poll #3 "What Do You Do With Logs?" Analysis

So, the results of my 3rd poll are ready: live results are here, picture is also in this post. This sure was fun!

First, this poll way more popular than my previous "why" poll. Yes, it seems like people do hate to wonder "why" :-)

Second, what are  the two choices, that are by far the most popular? They are:

• Store raw logs on a server (23%)
• Search raw logs (grep) when needed (24%)

Yes, this is the "state of the art" of logging:   collection of raw logs and "as needed" grep aka "slow and painful" search. In fact, the above answers might not even be given by the same people: some might be grepping logs on the individual servers, while others collect them on syslog servers and never touch them. That is why being in log management business is such a great thing: you have nearly the whole world to evangelize about the value of logs and log management tools.

Third, what's the next most popular idea of analyzing logs? It is "Run my own log analysis tool" at 10% of the respondents. Indeed, the movement started by the "enlightened" Leopold von Sacher-Masoch  still lives and thrives: people choose the Build->Suffer approach to log management often enough ...

Fourth, next come my somewhat self-inflicted surprise: apart from commercial log management (at 4%) and rolling one's own (discussed above at 10%), I added the option of "Use other log analysis tools"   which captured 7% of the vote. But what does that mean? I have no idea!

Fifth, I am NOT surprised by the lack of popularity of the rule-based correlation tools, such as SIEM (at 2%). When I made my decision to join LogLogic, I had to ponder this one really, really hard. Sorry to use this post to rant, but my conclusion at the time (which is also valid now) was that "SIEM is for some, log management is for everybody." This poll confirms this further.

Finally, all my logging polls and analysis are here. Next one is coming up!

Technorati tags: logging, polls, log management