By the gods, he lives! (Part 2: Secret weapons that never were)

So,

Last post I had talked about MDC3, and my role along with the rest of the unallocated team in making it to the finals. Sadly, we didn't win, but even worse, we never got to utilize the secret weapons we were planning on bringing with us. Once again, I had my involvement on the secret weapons front.

Most of our weapons weren't exactly that secret -- we had plans to bring massive password cracking wordlists with us from freely available sources, and essentially bring a package of updated tools with us.

My job was to re-spin Kali with the most up-to-date tools available for use, as well as provide us with some (not-so) secret weapons that are tailored towards CTF usage. Unfortunately, they outlawed use of your own special software or tools for the competition (in my opinion, that didn't keep others from downloading their tools and using them, but we played by the rules, regretfully)

So I worked at this shell script only to find out we couldn't use it for our CTF event. Well, our loss is now your gain. I present to you, updater.sh

What does updater.sh do?

Updater is a script specifically for Kali Linux that performs the following tasks:

.5: checks to see if you're root. If you're running on Kali, you should be, but still.

1. Generates new ssh host keys for your Kali installation, starts the ssh daemon and makes it start on boot -- this was something that was somewhat of an annoyance to me. Kali has every other service and capability known to man on it, but doesn't have sshd started by default. Well, I fixed that.

2. Adds the Kali Linux bleeding edge repository -- There isn't that much to say that hasn't already been said here. Basically there are certain tools that update frequently that were included by default with Kali. The bleeding edge repo was Kali's answer to those tools. Right now, the bleeding repo only has a small subset of tools, but dammit I want the latest and greatest, so I added the bleeding repo to Kali's /etc/apt/sources.lst

3. The almighty apt-get update and apt-get -y upgrade (dist-upgrade) -- What would an update script be without apt-get update and upgrade? I know, this is braindead simple stuff, but I wanted to automate it regardless. Just to give you fair warning: make sure you have ample time when running this script. This is the longest portion of the script. Why? Default Kali install pulls 400+ packages of varying size (some over 100+mb), then has to install all that.

You might wonder, why the hell would I choose to use dist-upgrade as opposed to just upgrade? I noticed when I ran the updates initially there was a number of packages that were "held back". Doing my homework, held back packages can be force updated via dist-upgrade when updating packages. I don't see a reason to hold back security tool packages, so I opted for the simplest way to fix it.

4. Msfupdate -- So you've updated your OS installations core tools and bleeding edge tools, what next? Kali's heart is metasploit. So msfupdate seemed logical to me. Takes a moment or two, but not nearly as drawn out as the apt-get package pull and the benefits are definitely noticable if you have any tasks where you'll be using or relying on the framework.

5. If you have the flash installer package for linux (.tar.gz) and/or the Nessus .deb package (with a home or professional feed provided to the script), they are automatically installed -- drop the adobe flash play for linux package in the root ("/") directory and/or the nessus installer package and provide the shell script with a home or professional feed key, and the script will auto install both.

Some of the more security conscious saw flash player and probably did one of these , but flash is required for a wide variety of web consoles (nessus included), so I included the option of installing it if the script detects you downloaded it and have it in the right place. If you don't want it installed, just don't download flash. That simple.

6. Couple of minor customizations -- As a part of the updater, some packages that I found to be missing on a deafault Kali install I configured apt-get to install:

armitage -- The popular graphical front-end to the metasploit framework by Raphael Mudge of Strategic Cyber. Also installs the teamserver (armitage collaboration) program. Was very surprised this was not installed by default.
mimikatz -- a popular program for dumping plaintext passwords -- basically if a user is logged in, lsass stores a plaintext copy of that user's password within lsass' memory space in memory. mimikatz can dump it and reveal the passwords stored.
terminator - a terminal emulator program that supports tabbed terminal sessions and also has a unique feature that allows you to split your terminal screen vertically and horizontally as many times as you want. For those with an OSX background, think of this as the Linux version of iterm/iterm2
unicornscan - a leightweight asynchronous network scanner. Doesn't have all the fancy bells and whistles of nmap, but sometimes all you want is a list of open ports. unicornscan does this fast and it does it well.
zenmap - the graphical front-end to the ever popular nmap scanner. Was also very surprised that this was not installed by default as well.

7. A collection of scripts and tools from other CTF veterans and pentesters alike who were willing to share their armaments, including:

1. The github collection of Cortana .cna scripts for automated red-teaming and post-exploitation. Cortana is touted as a "Red team force multiplier". Translated from cyber-ese, Cortana is a bot platform for armitage. Basically you provide scripts to Cortana and they can do things ranging from updating functionality in Armitage and allowing you to do new things to, automated scanning, to automatically throwing exploits at new targets seen on the network that have certain points open and/or listening services to post-exploitation modules to maintain persistence to hard-won hosts. If you're familiar with the sleep language, (The strange perl dialect that much of Armitage is written in along with java) then you can write your own, or simply modify the pre-existing scripts to suite your needs.

2. Unsploitable tools: Unsploitable is a collection of tools/scripts that when you feed them the results of a nessus scan will tell you what exact patch will fix the vulnerabilities listed. Good for situations where you have to perform triage quickly.

3. Defense Tools for the Blind (DTFTB): a set of scripts for unix systems that sort of allow you to ghetto rig security for unix systems and kinda triage the system/provide emergency defensive countermeasures for when you don't have the time or resources to fully patch out your vulnerabilities. For example, one script is a while true that scans ps output for /bin/sh or nc or netcat and kills the associated processes, killing post-exploit shells and simple backdoors.

4.New custom build of smbexec by Brav0hax. Basically this is a copy of the smbexec functionality found in metasploit re-written to get around some security products catching metasploit smbexec and flagging it as a non-legitimate service installation.

That's it. Lot of simple things, that make Kali that much more awesome and makes it that much easier for you to gear up for the CTF or pentest.

As with the autosnort scripts, the script tells you what it's doing and gives you success or failure indicators. If there's a failure the script bails.

The script logs the entirety of output for each of the installation steps to a dedicated log file in /var/log named kali_updater.log

If you're looking for a challenge, troll through it, figure out what failed, fix it, contact me, tell me my script is broken and how you fixed it. If you're lazy or not looking for a challenge, tell me my script is broken and ship me a copy of the updater log.

As usual, along side any post where I announce a new tool or script, it's available via github

My final installation in the 'what the hell have you been doing series: BSIDES DC!

By the gods, he lives! (Part 1, MDC3)

Hello again,

It's been quite a long time since I've posted anything here or posted any updates on github for autosnort OR H1N1 for that matter. well, let me tell you what I've been up to lately, this'll probably be over multiple posts, so I hope you're ready to be shotgunned with updates  >:) so let's start with MDC3.

For those who don't know about MDC3, what it is and why I'm interested in it, it's a yearly event sponsored by SAIC among other organizations. It's mainly a "suitcon", where managers in industry meet up, and a bunch of security vendors come on down to the Baltimore convention center with whitepapers and statistics and other things as to why you want to buy their wares.

In addition to that, MDC3 hosts a yearly cyber challenge event that I use as a sort of whet stone for keeping myself sharp. There are 5 rounds in total. The first 4 rounds are done online/offline with friends at a place of your choosing, with the final round at the "suitcon" live.

Out of the four online/offline rounds, the first two are practice rounds while the next two rounds are qualifier rounds for the finals. The practice and qualifier rounds are very similar -- either you're given VM images to examine, or you are given forensic and crypto challenges to perform. -- things ranging from "Here's a VM with an operating system of some sort running on it. It's owned in various ways and has several security holes. Close the holes." to "Here's a PCAP, answer these questions", where the questions range from "What was the admin on this box doing while the system was being compromised?" to "The attacker used FTP to exfil data off the system. Give us the name and an md5sum of the files downloaded."

We skipped the practice rounds this year, though in retrospect, I kinda wish we had gotten the files so I could practice more. So for the first qualifier round, we ended getting a windows server VM that was owned in various ways. The server was running IIS and had simple TCP services enabled. In addition to that, no patches, no security countermeasures, no firewalls, several netcat backdoors, user applications (firefox and chrome installed -- plus firefox was backdoored with another netcat listener), etc.

The round was scored out of 20 points or 20 vulnerabilities to remediate. Our team ended up getting 19 out of 20 and passing the round. Essentially all we did was more or less manually revert the host back to a clean,default install with DISA STIG recommendations applied, full patches, all services disabled, user account passwords changed and other common sense things you would do to a system that was compromised. We had no requirements to keep IIS running or core services on the VM that were required to be running, so we just uninstalled everything.

Some of you may be running, how would SAIC know what was going on with the VM and whether or not you remediated something? They had a really crappy java client called cybernexs installed on the VM with a special account running a service that would start on boot, that you would register to a scoring server. This service would phone home and say what you had fixed.

Theoretically, you could reverse engineer the client and make it pretend to call back stating everything was fixed, but we never got around to doing that. I know I'm not skilled enough to do that, but there were some on our team that probably could have done it.

Back to the competition. So overall I'll call round 1 a success, however it's disappointing: In a real-live production environment, your server is responsible for providing a service (at minimum 1 service, sometimes many), the fact that we were given free reign to uninstall all services and applications installed on the server flies in the face of reality.

So the second round this year was network forensics and crypto.  We were given a packet capture to analyze and the following questions:

1. What files were transferred from the victim?

2. What directory were files initially copied to?

3. What is MD5 hash of files transferred to the attacker? (Use lowercase letters)

4. What port is the backdoor listening on?

5. What was the admin doing during attack?

6. What is the name of the backdoor user?

7. What is the password of the backdoor user?

8. What is root's password?

9. What software on the victim allowed the attack?

10. Repair the zip file and answer the challenges within.

I managed to do most of these questions easily. The only tools I used this were Wireshark, Network Miner, md5sum and DiskInternals zip repair utility. Managed to answer all the questions except question 8.... a portion of the pcap showed the attack managed to get a copy of /etc/shadow. Even after throwing JohnTheRipper at it for 24 hours straight, I never cracked the password or got any hint in the pcap as to what root's password was. Bummer.

Question 10 was a multi-parter. Part of the forensic challenge indicates there were files transferred during the attack. One of the files was a zip file. It was in plaintext and was easy to single out with wireshark and tcp stream separation. After exporting the bytes of the TCP stream, I was able to reconstruct it as a zip file. The problem was that the zip file was broken. Another one of our team members who knew more about forensics than me was able to manually fix it. There was some sort of discrepency between the zip file's size and the zip file record's reported size that needed to be repaired. I ended up using diskinternal's zip file repair utility and that fixed it for me. That was as far as I got there, I relied on other team members to resolve the crypto challenges since I'm absolutely terrible at crypto and cipher schemes.

We did well enough as a team to make it to the finals. The format of the final round is completely different from the previous rounds. It's essentially a king of the hill style CTF challenge. There are several systems (with more being added over the duration of the event) that are vulnerable and very exploitable. The goal is to get on the system, and plant your flag (a flag.txt file with a hash that represents your team in a specific directory on the host) and keep it there. You can lose shell access to a system, but as long as your flag is there, the system is still yours.

Well, I was in charge of exploiting windows systems, and here was my run-down: Most of the Windows 2000, w2k3 and XP systems were vulnerable to some fort of MS08-067 (If you don't know what this is, it's essentially a 4+ year old SMB exploit for windows versions ranging from windows NT to Server 2003 as well as several service pack revisions, it's pretty old, but is still used as a point of reference for many security conferences and classes because it's very reliable and a shining example of Remote Code Execution exploitation) MS03-026 (this exploit is mainly focused around windows 2000, early versions of XP and early versions of server 2003. It's an RPC DCOM exploit that is also another popular remote code execution exploit used as an example in lots of training courses.), etc.

The most unique windows box I had to throw against in the network, another team member found was vulnerable to file upload via webdav, and handed off to me to finish owning. There's a metasploit module that exploits the file upload and uploads an ASP script which then gives you a meterpreter shell as an unpriveleged user. At that point, you could plant the flag as an unpriv'd user, or you could be like me, rummage through armitage and metasploit's post-exploit modules and find a privilege escalation that would give you SYSTEM access...As an attacker, why would I want system access? Well, there are a number of reasons, we'll get into that in a moment. after several unsuccessful tries, I managed to get system through ms11-080. I wish I was smart enough understand how it worked, but the gist of it is that it's a driver that executes in ring 0 (kernel space, SYSTEM space) and through some voodoo magic or another, you load shellcode into memory and tell this driver to refer to your shellcode, and BAM you have a system shell (smarter hackers will skewer me... but I did what I could with what I know.)

So, what was the point of gaining system access the windows boxes? system access enables you to utilize a post-exploitation module known as hashdump. hashdump allows you to dump the contents of the SAM (Security Accounts Manager) database. This contains hashes for all the users on the system, from the lowliest of users to all system administrators.

What good are hashes without having the cracked, you may ask. Well, in windows, you don't necessary need to crack the hashes in order to be able to utilize them. A common exploit for windows is a technique called "Pass the Hash". PTH exploits a weakness in windows filesharing that allows you to authenticate to a remote windows system by simply giving the remote system a copy of that user's hash as opposed to the actual password and having the remote system do hash checking. Pass the hash can be used to do anything that user can do -- map drives, execute commands, upload and execute metasploit payloads, etc.

So, when you exploit a windows system, the first thing you want to do is esclate privs and get system access as soon as you can. After you have system access, you want to dump hashes and get hashes for admin users, since it's extremely easy to gain system access as an admin user. Once you have hashes for admin users, this allows you to use psexec/pass the hash to maintain access to an exploited host, or easily regain access to the host if you've somehow lost your initial access.

So, in spite of getting hashes for multiple systems and repeatedly managing to get back on as SYSTEM, we didn't get very far in terms of placement, because others would simply beat us to our targets and had better methods of planting their flag and keeping it there. I didn't know enough about acls and cacls to figure out how to fix file permissions to undo what they did to their flag file to leave it on the host, then drop our file and lock it to where other teams couldn't modify it. It was really frustrating. Also, in addition to that, I would repeatedly lose my meterpreter shells in spite of doing things the right way (e.g. migrate processes and load meterpreter into a system process that an adversary would not want to kill, etc.)

While I was thrashing windows boxes, others were doing everything they could in linux and unix hosts. And once again, we would get access (managed to plant our flag briefly on one), and lose it, then the hole would be closed, locking us out of that host.

So, in spite of all things, and not having won the competition, I was satisfied because of the fact that I did manage to pop some windows boxes and be a massive annoyance to other teams. I was annoyed because either the network was incredibly unstable, or the hosts we were attacking were incredibly unstable. I'm guessing a combination of the two, but the systems we attacked and got on to would get REALLY bogged down after a little while. I was also disappointed. SAIC didn't give us much swag this year for making it to the finals. Last year, we got challenge coins for making it to the finals, this year, we got a craptastic backpack. registration for the event was something like 300 dollars per team in the professional level. What gives? Sigh. Oh well.

So, that's a run-down of my MDC3 adventures.

Part 2: the secret weapons that never were. Look forward to a script release...

Project H1N1: An introduction.

Hello Autosnort viewers, today I'm going for a change of pace and going to introduce a bold, new project that could have a monstrous impact on the IDS and Network Intelligence realm. I'd like to introduce groundbreaking for the new Unallocated Space collaboration project, H1N1:
----------
H1N1 – Abstract:

H1N1 is a project that is meant to integrate the PRADS Passive Operating System Detection with Snort, the popular Intrusion Detection System package. The end goal is a system where Passive OS fingerprinting and IDS both work off one another to present a bigger picture:

-    Snort benefits would benefit from Passive OS fingerprinting by having the Passive OS fingerprinting software inform it as to what operating systems are in its HOME_NET, and use that information to build Preprocessor policies, specifically frag3 and stream5 reassembly policies, that control how snort handles reassembling traffic for those hosts. Eventually this gathered information could be used to make snort rule recommendations based on hosts, client service banners observed in the HOME_NET as well.

-    PRADS would (eventually…) benefit from snort, by existing as a preprocessor in the “stack”, benefitting from snort’s native ability to reassemble fragmented packets and reassemble TCP streams for both Operating System identification and Client/Server banner grabbing from the wire.

-    Users would be able to review information for assets in their HOME_NET detected off the wire, alone with their intrusion events, offering greater network intelligence and a bigger picture.

The problem is, I’m a complete derp when it comes to anything programming or development. I know shell scripting, I’ve done some introductory python, but beyond that, I’m pretty much helpless. The key to completing this project lies in the Unallocated space motto: Teach, Learn, Build -- We'll teach one another something, We will learn in order to fill in the gaps of knowledge we may lack in making this project a reality, and the end goal will be to Build something awesome.

So that's the general abstract and overall goal of this project. Yesterday afternoon, I did a presentation about the project at my local hackerspace, Unallocated Space  The goal was to find others in my area who would want to collaborate and be interested in making the project a reality. Below is an overview of the discussion and the powerpoint slides I brought with me:
 ----------
H1N1 – Day 1: Introductions, Teach, Learn, Build

The first day was primarily an introduction to the technology behind the project. Unallocated motto was integrated into today’s talk, in introducing H1N1:

-    Teach:
o    I introduced the audience the open-source IDS/IPS software snort, if they weren’t already familiar, as well as how it worked. After that I explained the difference between Active and Passive OS fingerprinting, then delved deeper into the various techniques used to passively fingerprint operating systems that have been researched and well-established by minds much greater than my own; such as TCP SYN flag options, Client/Server banners, ICMP fingerprints, etc. I discussed the weaknesses of Passive OS fingerprinting – network devices modifying traffic between the source and destination, duplicate fragments/segments, etc.

o     I then set the stage for H1N1 – why hasn’t there been a project that integrates these two technologies together and have them work together? Most of the more well known solutions are commercial, while some of the Open-Source solutions aren’t quite there yet, in terms of usability .

-    Learn:
o    I want to make this project a reality. But I next to 0 programming knowledge. I know how to “hello world” in three separate languages, but unless I have an active challenge, or a reason to extend my knowledge beyond that, I tend to lose interest and my knowledge drops off nearly immediately. H1N1 is going to be a challenge, A challenge that I don’t know where to even begin to tackle.

o    This is where Learning comes into play. I want to learn how to develop in different programming languages. I’m more than willing to learn from anyone willing to teach. If you know how to do it, show me. If you don’t have the time, tell me to “Go learn about X concept in Y language then come back to me.” Once I understand it, I’ll teach others. We all benefit and gain knowledge in this way. Knowledge is power, and we all deserve to wield it.

-    Build: The project is ambitious, but I want to start with baby steps. Let’s start with the basics.
o     What passive OS identification software do we want to use?
o    Where are we going to host the code, and eventually the alpha builds?
o    Let’s make sure that Snort and PRADS can actually sniff off the same interface without causing the box to explode?
o    The current Prads2db script is in perl… Hell, most of the prads utils are in perl. Should we fix that? Can we make them better?
o    We can output the prads data to a database, to host attribute XML, but currently, there’s no web interface to display this data on.
o    (This is probably something that will happen much later) Is there host timeout functionality integrated in with PRADS? If not, how can we make it?
o    Automating the prads2snort script (rewritten in something that isn’t perl), and having it run on a regular interval, given to snort, and passing snort SIGHUP to read the new hosts identified in HOME_NET
----------
Finally, We ended the conversation with a review of the slides, Q + A and what direction to take the project -- a review of the Build portion of my slides where I wanted to establish project milestones for H1N1:
----------
H1N1 Q + A:

o     What passive OS identification software do we want to use?

Answer: We all primarily agreed that PRADS has the most features out of the options presented, and it made the most sense to build off it.

o    Where are we going to host the code, and eventually the alpha builds?

Answer: For the time being, we all have infrastructure in our own homes that we can build snort and prads on, Forgotten offered to create an unallocated space github account for hosting the project code there. Eventually, when we have a workable alpha release, or beta release, or SOMETHING, we can talk to hunter and UAS Keyholders to either get a physical box or virtual machine to demo things.

o    Let’s make sure that Snort and PRADS can actually sniff off the same interface without causing the box to explode?

Answer: This should be very easy for us to do. compile/build a snort sensor (snort, barnyard2, a web interface of choice), then compile PRADS. Set them both to run on startup (either via rc init script, or via rc.local), throw traffic at it. If it doesn't explode in a fit of rage, I'm going to call it a victory.

o    The current Prads2db script is in perl… Hell, most of the prads utils are in perl. Should we fix that? Can we make them better?

Answer: There's a general fear and loathing of perl that's understandable, along with a love of python. Additionally there's been talk at our first session that it would be trivial to write a parser to drop this information into a database. Any volunteers for reviewing the prads2db.pl script as well as the prads2snort.pl script, and prads.sql data and determining whether or not porting it all to python would be feasible?

o    We can output the prads data to a database, to host attribute XML, but currently, there’s no web interface to display this data on.

Answer: Ruby on rails has been suggested. I don't know enough about web application technology to make this happen, but I'm not opposed to trying this. Another technology that I've been curious about is node.js.. A lot of new up and coming infosec projects have used node.js, and it's supposed to be incredibly fast. I wanna learn more.

o    (This is probably something that will happen much later) Is there host timeout functionality integrated in with PRADS? If not, how can we make it?

Answer: It was suggested that we turn this into a rolling time window sort of thing that's user configurable. Instead of taking the first fingerprint we see and associating it to an IP address, we can keep fingerprints associated to an IP address seen over a period of time, and based on results over X time period, Have Y% confidentiality that this IP address is this type of operating system. This is likely something that will have to come much further down the line, as it seems to be something that will require considerable effort to code.

o    Automating the prads2snort script (rewritten in something that isn’t perl), and having it run on a regular interval, given to snort, and passing snort SIGHUP to read the new hosts identified in HOME_NET

Answer: This should be fairly straightforward to do: Basically read the database, and generate a host_attributes.xml the same way the current perl scripts read the prads log and generate output.

----------
That just about sums up the discussion about this project. This project is still very much in its infancy. No hard timelines for ANY of the milestones established have been set. The reason for that is that I want this to be a project that anyone can come into with any level of programming knowledge, say "this part of the project interests me", take charge of it, learn the concepts necessary to complete it, come back to the group, teach what they learned, and implement in a way that helps meet the overarching goal of the project -- Teach, Learn, Build. To make this happen, setting hard timelines isn't feasible since everyone learns at different rates.



I hope to drum up support for this project as far and as widely as I possibly can. I'm not looking for anyone to do the code and solve the problems for me, or folks wanted to collaborate on this project. Even if your contribution to this project is "This task would be best suited to this programming language. Go learn about this concept in this language and come back to me."

A good friend of mine, Forgotten has started a collaboration github account on github, by the namesake of this project:

https://github.com/Unallocated/H1N1

Currently there isn't much there other than the readme; I'm hoping to change that if not on my own, then with your help, slowly but surely.

Hope to see you on the list of contributors,

DA_667

autosnort: compatible with Linux Mint

This is a really quick post. I've installed Linux mint on my person desktop at home, and out of curiousity, decided to try to run autosnort on mint to verify whether or not any of my build scripts were compatible. Well, the ubuntu autosnort script is 100% compatible with mint.

I'm aware that mint is primarily a desktop operating system, but if you're interested in running it on a Mint desktop as a sort of personal single workstation IDS, here's what to do:

1. Get a root shell: sudo /bin/bash
2. wget http://www.github.com/da667/Autosnort/Archive/master.zip
3. unzip master.zip
4. copy the autosnort-ubuntu-[date].sh script and an interface installation script to /root
5. run "bash autosnort-ubuntu-[date].sh"
6. run through the prompts as normal.
7. At the very end, when the script asks you whether or not you want to reboot, choose NO.
8. sudo vi or sudo gedit /etc/rc.local and modify the ifconfig eth0 line.
It initially reads: ifconfig eth0 up -arp -multicast promisc
make it read: ifconfig eth0 up promisc
9. save the file.
10. either reboot, or run "sudo bash /etc/rc.local"
11. Open firefox, or the web browser of your choice. type "localhost" in the address bar. You should be greeted by the web UI of your choosing.
12. ps -ef | grep snort will confirm whether or not snort and/or barnyard2 are running.

Yes, I know I've said in the past that managing your IDS on the same interface you're sniffing on, or having your sniffing interface be addressable isn't a good thing, but those are for DEDICATED IDS installations, this little "hack" is meant for users who want to drop snort on their personal workstation as an added layer of security.

Cheers,

DA_667

Changing of the seasons; RHEL/CentOS code push/update; Autosnort milestones posted.

    So here we are, at the end of August. Defcon has came and went, summer vacation is coming to close, and labor day is right around the corner. I don't consider summer officially over until Labor Day has come and passed, but why postpone it?

On that note, I've been busy in the lab, and have a whole host of updates that I'm dropping tonight for Autosnort. 99% of these updates are specific to the RHEL/CentOS release, that has been long neglected -- much longer neglected than the other distro scripts. The last update you all saw for RHEL/CentOS was sometime in April. The last major blog post you saw was in regards to the 9 layers of hell I traversed to make Snorby work on CentOS with SELinux enabled, then I disappeared for summer break. So, without further adieu, here's what's happening:

RHEL/CentOS:
- Updated the entire look and feel of the main autosnort installation script. CentOS/RHEL users now have the metasploit like prompts just like the Debian and Ubuntu users. Only things the user should be aware of are printed to the screen now:

    - Status updates are in blue (e.g. [*] this indicates what autosnort is doing currently)
    - Notifications are in yellow (e.g. [*] this indicates something the user needs to pay attention to (such as a prompt, or something they may want to note somehow))
    - Successful modifications are in green (e.g. [*] this indicates whatever autosnort was doing was successful)
    - Unsuccessful modifcations/installations are in red (e.g. [*] something bad happened. You'll probably want to get a hold of me and give me the log files for your installation, so it can be resolved)

- Speaking of logging, the installations scripts no longer spew output all over your screen buffer. Thanks to neat trick I picked up from stack exchange, the output of every major command is saved in two separate log files in /var/log:

    -/var/log/autosnort_install.log -- contains output from all the major commands ran from the main autosnort installation script
    -/var/log/[interfacename]_install.log -- contains output from all the major commands ran from the interface installation script for the interface you chose to install.

- The main autosnort installation script and all the web interface installation scripts have been updated with the new metasploit-like prompts and the background logging. This includes:

    -autosnort-CentOS-[date].sh
    -snortreport-CentOS.sh
    -base-CentOS.sh
    -aanval-CentOS.sh
    -syslog_full-CentOS.sh
    -snorby-CentOS.sh

- All web interface installation scripts for RHEL derivatives have had their DocumentRoot and Directory directives reconfigured to serve out the web interface of your choice. This means all you have to do is point your web browser to the ip address of your sensor's management interface, and provided you reconfigured ip tables to allow port 80 inbound, your web interface will pop up automatically.
- All web interface installation scripts for RHEL-based distros are 100% compatible with SELinux
- All web interface installation scripts for RHEL-based distros have had the ownership of DocumentRoot changed to the apache user and group
-Fixed minor grammatical and syntactical errors littered throughout the script.    

Ubuntu/Debian:
- Apparently at some point between now and june, the passenger output directory for the mod_passenger.so binary changed the name of the directory from "libout" to "buildout". sigh. consistency is awesome, don't you agree? I only discovered this during testing passenger during the CentOS testing process.
- In an effort to make the mysql installs uniform between all autosnort builds and promote better security, I've made the mysql-server installation for Ubuntu and Debian silent, but now, just like with the CentOS script, the /usr/bin/mysql_secure_installation script is ran as a part of autosnort. huzzah for better secured databases.
- Same as the CentOS script, found minor grammatical and syntactical errors littered all over the script. Found and fixed what I could.

Code push should be happening tonight before init 1 for the night. As usual, the scripts are open source and are available on the autosnort github:

https://github.com/da667/Autosnort

While we're on the topic of github, I've posted some milestones that officially lay out where I want to take autosnort next -- things I want to accomplish with it. Have a look, see if I'm missing something:

https://github.com/da667/Autosnort/issues/milestones?with_issues=no

And last, but not least... The new project ideas I spoke of in my last blog post? I'm holding a meeting at our local hackerspace about one of them in particular, the passive OS fingerprinting project, trying to get some helping hands with it and make it more of a success. If you're in the Maryland/DC/VA area, come to unallocated space on the 25th so we can get a project outline hashed out.

Cheers,

DA_667

End of my Summer vacation; new projects to consider -- want feedback!

Dear autosnort users,

I just got back from Defcon. Summer is almost over. You're probably wondering if this lazy louse is going to get off his ass and start scripted again. Well, yes, but, I have some details to announce.

1. I will definitely be continuing the Autosnort project. I never really had the intention of taking the summer off, but now I'm kinda realizing that it makes sense to do this. Summer is practically the only time I get to spend with other hackers locally and at Defcon, and one of the few times a year that I get to see family back home.

First on my order of things to do is to finish testing on CentOS for pulled pork. I released the selinux module I painstaking build through a couple of weekings of gnashing teeth and cursing, but I want to have the selinux module generated by the snorby install script for centOS instead of having use have to bring another item with them for the installation. Right now, I have the script echoing out the source of the module that I got to work and building that generated file. It looks kludgy in the code, but it works wonderfully.

2. I have an ambitious project that I really want to get some help doing. I was a former Sourcefire employee. That being said, I'm familiar with RNA or what they call firesight right now. The general gist of this technology is to gain insight as to what operating systems are running in your network, and make one's snort ruleset more efficient based on that -- You're not going to care about X11 exploit rules if you're not running Linux boxes with publically listening X servers, the same as you wouldn't care about seeing alerts about SMB/windows malware attacks if you have your HOME_NET defined as a network that is solely a groupe of freeBSD servers not running samba.

p0f -- passive OS fingerprinting has been around for years, at least a few decades. I was present at shmoocon. The project had been idling for a while up until a group of folks decided to request access to oversee the project and continue on with it. This was a talk at shmoocon over a year ago. The project is alive and kicking.

What I want to be able to do: take p0f signatures, have them monitor a defined network, and develop a measure of confidence as to what is running in your home network.

take this intelligence and use it modify pulledpork -- enable/disable sids based on p0f results, set ip defragmentation and tcp stream reassembly policy by majority of X hosts in your HOME_NET, etc.

the thing is, I'm not quite sure where to start. If anyone wants to help me with this in any way... I want to make this happen.

3. Malware analysis wiki. this inspired by my friend @forgottensec. Forgotten ended up providing the Capture the Flag community the CTF wiki. Well, I want to provide a resource for the information security community as well:

There seems to be information scattered to the seven corners of the  internetz. I want to gather as much information as I possibly can into one way for malware analysis techniques. For example, I don't the first thing about using debuggers to debug malware, but I know lots of interesting things to look for, when it comes to dynamic malware analysis. Why not put it some place publically accessible and easily contributed to?

Let me know what your thoughts are on these new developments I'm thinking about, until then... I'm gonna enjoy the end of my summer before I have to get my lazy ass back to work

Still Out here!

Hello Autosnort Users,

This is just a quick update to inform you all that I'm still out here and still consider autosnort an active endeavor. I guess you could say that I'm on summer break, and it wouldn't be too far off. I went home some time in July for vacation, been studying Python after hours at work, Enjoying the steam summer sales with my sparse spare time, and I'm looking forward to going to Defcon/Bsides in the coming week.

In spite of rumored... hostilities real, imagined, thinly-veiled or otherwise, I'm a hacker and security researcher first, and above all else. I enjoyed Defcon the last time I went, and I'm sure this time will be just as enjoyable.

Until then, I ask that you be patient.

Cheers,

DA_667

Autosnort-CentOS progress -- SELinux, ROR, passenger, and Snorby all working in Harmony

I've been working pretty hard on getting The CentOS autosnort build up to par with the Debian and Ubuntu autosnort builds, and for the most part, I'm nearing the end of the road there.

Aside from work and my regular everyday life, I've been spending my weekends mostly working on getting Snorby to run on CentOS with SELinux enabled. I'm a firm believer that in this day and age you shouldn't have to turn off SELinux to make your application work.

According to Wikipedia, SELinux has been around since the year 2000 and mainlined into the 2.6 kernel since 2003. It's been close to over ten years since SELinux was rolled out. Turning it off isn't a solution. It's a work-around, and pardon my french, an incredibly shitty and lazy work-around at that.

Now, I've made it no secret that I do not like ROR at all. This could be for a number of reasons -- I'm a newb programmer, I haven't developed webapps, I don't know what I'm talking about -- take your pick. Do not take this as a personal attack if you love ROR for one reason or another. Ruby OTOH has given us pretty interesting things, such as the Metasploit Framework.

In troubleshooting this how to get Snorby and SELinux to behave I had to learn how to write SELinux policy modules. As a side note, I would like to thank whoever on github posted this gist:

https://gist.github.com/2-718/708659

The code that I am dropping below is HEAVILY based on this individual's work getting passenger to work under SELinux without turning it off. I don't know who you are, but I salute you.

Let me give you a bit of background. SELinux is incredibly modular, there are several different ways you can tell SELinux to allow an application to do certain things, access certain resources, etc. You can use chcon to tell SELinux to allow a process to access files (think of this as another layer on top of the file access controls that we all know and love with chmod/chown), there are booleans you can set that, almost windows registry-like tell SELinux to allow a process to be able to interact with a mysql database or connect over the network.

Then there is the ability to write your own modules via selinux development tools to load into SELinux and in turn load into the kernel. You would generally do this if the application is very complex and needs to get its hands into a lot of places.

The research I did on this over the weekend varied. Most places suggest just running the application over and over again -- each iteration, just running audit2allow to determine what it is that httpd wants access to and building a policy module out of it.

I kept doing this, and more and more things would be allowed, then of course, the next thing in the way would break. It felt like dependency hell all over again.

Eventually I found that block of code by someone who had apparently been here before and decided to iterate through audit2allow probably hundreds of times to develop a single policy for passenger. I took it, ran with it, and improved it. Over the last few evenings I expanded upon it, and finally got Snorby to a point to where it will run with 0 errors and full functionality with SELinux enforcing and in targeted mode.

Have a look; I'll be posting this code to github soon, but I wanted to copy it here just to give you an idea of what the hell this application has access to on your system:

### begin code ###
module passenger 1.0;

# Not an expert at SELinux module building, but this is similar to library declarations in C programming -- these are things that the module needs to be able to do and contexts the module needs to be able to understand/communicate with

require {
        type init_t;
        type initrc_t;
        type system_cronjob_t;
        type mysqld_t;
        type usr_t;
        type syslogd_t;
        type system_dbusd_t;
        type abrt_dump_oops_t;
        type dhcpc_t;
        type kernel_t;
        type auditd_t;
        type udev_t;
        type mysqld_safe_t;
        type postfix_pickup_t;
        type sshd_t;
        type crond_t;
        type getty_t;
        type anon_inodefs_t;
        type httpd_tmp_t;
        type devpts_t;
        type user_devpts_t;
        type httpd_sys_script_t;
        type security_t;
        type httpd_t;
        type unconfined_t;
        type selinux_config_t;
        type hi_reserved_port_t;
        type httpd_sys_content_t;
        type httpd_sys_rw_content_t;
        type var_t;
        type cert_t;
        type postfix_qmgr_t;
        type postfix_master_t;
        class file { getattr read create append write execute execute_no_trans open };
        class process { siginh signal noatsecure rlimitinh setpgid getsession };
        class unix_stream_socket { read write shutdown };
        class chr_file { read write append ioctl };
        class capability { setuid dac_override chown fsetid setgid fowner sys_nice sys_resource sys_ptrace kill };
        class fifo_file { setattr create getattr unlink };
        class sock_file { write getattr setattr create unlink };
        class lnk_file { read getattr };
        class udp_socket name_bind;
        class dir { write read search add_name getattr };
}

#This stuff below is more of an access control list -- these are things the contexts below are requesting to be able to do in order to run properly.
#============= httpd_sys_script_t ==============
allow httpd_sys_script_t abrt_dump_oops_t:dir { search getattr };
allow httpd_sys_script_t abrt_dump_oops_t:file { read open };
allow httpd_sys_script_t anon_inodefs_t:file { read write };
allow httpd_sys_script_t auditd_t:dir { search getattr };
allow httpd_sys_script_t auditd_t:file { read open };
allow httpd_sys_script_t cert_t:dir { search getattr };
allow httpd_sys_script_t cert_t:file { read getattr };
allow httpd_sys_script_t cert_t:lnk_file read;
allow httpd_sys_script_t crond_t:dir { search getattr };
allow httpd_sys_script_t crond_t:file { read open };
allow httpd_sys_script_t devpts_t:chr_file { read write };
allow httpd_sys_script_t dhcpc_t:dir { search getattr };
allow httpd_sys_script_t dhcpc_t:file { read open };
allow httpd_sys_script_t getty_t:dir { search getattr };
allow httpd_sys_script_t getty_t:file { read open };
allow httpd_sys_script_t httpd_sys_content_t:fifo_file setattr;
allow httpd_sys_script_t httpd_sys_content_t:sock_file { create unlink setattr };
allow httpd_sys_script_t httpd_sys_rw_content_t:file { execute execute_no_trans };
allow httpd_sys_script_t httpd_t:dir { search getattr };
allow httpd_sys_script_t httpd_t:file { read open };
allow httpd_sys_script_t httpd_t:unix_stream_socket { read write };
allow httpd_sys_script_t httpd_tmp_t:fifo_file setattr;
allow httpd_sys_script_t httpd_tmp_t:sock_file { write create unlink setattr };
allow httpd_sys_script_t init_t:dir { search getattr };
allow httpd_sys_script_t init_t:file { read open };
allow httpd_sys_script_t initrc_t:dir { search getattr };
allow httpd_sys_script_t initrc_t:file { read open };
allow httpd_sys_script_t kernel_t:dir { search getattr };
allow httpd_sys_script_t kernel_t:file { read open };
allow httpd_sys_script_t mysqld_safe_t:dir { search getattr };
allow httpd_sys_script_t mysqld_safe_t:file { read open };
allow httpd_sys_script_t mysqld_t:dir { search getattr };
allow httpd_sys_script_t mysqld_t:file { read open };
allow httpd_sys_script_t postfix_master_t:dir { search getattr };
allow httpd_sys_script_t postfix_master_t:file { read open };
allow httpd_sys_script_t postfix_pickup_t:dir { search getattr };
allow httpd_sys_script_t postfix_pickup_t:file { read open };
allow httpd_sys_script_t postfix_qmgr_t:dir { search getattr };
allow httpd_sys_script_t postfix_qmgr_t:file { read open };
allow httpd_sys_script_t self:capability { setuid chown fsetid setgid fowner dac_override sys_nice sys_resource sys_ptrace kill };
allow httpd_sys_script_t self:process { setpgid getsession };
allow httpd_sys_script_t sshd_t:dir { search getattr };
allow httpd_sys_script_t sshd_t:file { read open };
allow httpd_sys_script_t syslogd_t:dir { search getattr };
allow httpd_sys_script_t syslogd_t:file { read open };
allow httpd_sys_script_t system_cronjob_t:dir getattr;
allow httpd_sys_script_t system_dbusd_t:dir { search getattr };
allow httpd_sys_script_t system_dbusd_t:file { read open };
allow httpd_sys_script_t udev_t:dir { search getattr };
allow httpd_sys_script_t udev_t:file { read open };
allow httpd_sys_script_t unconfined_t:dir { search getattr };
allow httpd_sys_script_t unconfined_t:file { read open };
allow httpd_sys_script_t unconfined_t:process signal;
allow httpd_sys_script_t user_devpts_t:chr_file { read write append ioctl };
allow httpd_sys_script_t usr_t:file execute;
allow httpd_sys_script_t var_t:dir { write read add_name };
allow httpd_sys_script_t var_t:file { read getattr create append };
#============= httpd_t ==============
allow httpd_t hi_reserved_port_t:udp_socket name_bind;
allow httpd_t httpd_sys_content_t:fifo_file { create unlink getattr setattr };
allow httpd_t httpd_sys_content_t:sock_file { getattr unlink setattr };
allow httpd_t httpd_sys_script_t:process { siginh rlimitinh noatsecure };
allow httpd_t httpd_sys_script_t:unix_stream_socket { read write shutdown };
allow httpd_t httpd_tmp_t:fifo_file { create unlink getattr setattr };
allow httpd_t httpd_tmp_t:sock_file { getattr unlink setattr };
allow httpd_t security_t:dir search;
allow httpd_t self:capability { fowner fsetid };
allow httpd_t selinux_config_t:dir search;
allow httpd_t var_t:file { read getattr };
allow httpd_t var_t:lnk_file { read getattr };

### end code ###

Admittedly there are a number of things above that make sense that Snorby would need access to -- sockets, file execution, making its own fifos, managing its own files and process execution options, etc. but then there are other things that just made me go WTF. Why does the webapp need access to crond? to getty sessions? pts? sshd? cronjobs? files and directories under the unconstrained context? dhcp files? auditd files?

I could be blowing this entirely out of proportion, but I worked with snortreport, BASE and aaval, and there were a minimal number of things I had to enable to get each of those interfaces to work under SELinux. Maybe PHP is just better integrated into SELinux, or maybe SELinux knows that PHP is beyond hope and doesn't even try to constrain it. I don't know, but reading through the module and seeing what it has access to is kinda crazy to me.

Okay, so now that I'm off my soapbox, you're probably wondering when to expect a code push for CentOS.

The answer is: soon. I know that's not the answer you all want to hear, but it's the best I can give right now -- The rewrite of the main script is mostly done. At this point, I'm working on re-writing the child shell scripts, prettying them up, and also ensuring that Snorby plays nice.

I'm going to re-iterate again, that this isn't any sort of a personal attack on any developer of any web interface, or against any platform, just my (probably ill-informed) opinion based on my observations.

Until next time,

DA_667

Addendum: I thought I would add a note stating that the I've posted the code above into an SELinux module that is available via autosnort's github under the CentOS autosnort directory, in a subdirectory called  "PolicyModules". the directory contains passenger.te as well as a PolicyModulesNotes.txt file that details how to actually compile the passenger.te file into an SELinux module. This is for those who may not necessary care to use autosnort, may not care to use Snorby, but may need some help trying to figure out what accesses Ruby, Rails, and Passenger want on SELinux enabled systems.

I hope that it is some help to the Linux community.

New Autosnort Release, support for more distros and much more!

Hello,

Lots to talk about, so let's get started.

I finally got around to finishing testing for the new, improved, and much cleaned up build of Autosnort that I had been hyping for a little while. For those who didn't see the last blog post, or the screen shot, I've done a bit of housekeeping with Autosnort.

Instead of puking the output for every single command all over the screen buffer, Autosnort's output has been significantly minimized and cleaned up where it could be cleaned up. Output is now very metasploit-like:

[*] are things that the script is doing
[*] are things the user should pay attention to
[*] are successful results
[*] are unsuccessful results

"Okay, so what if something goes wrong? All I have is a generic 'something went wrong' comment." Well, instead of hoping whatever you need was caught in the screen buffer, autosnort now logs the output of the ENTIRE installation to /var/log/autosnort_install.log, and /var/log/[interface you chose to install here -- for example, snort_install.log], So now, instead of having to hope the screen buffer caught the relevant bits of a problem with my script, you now have an entire log of what the script was actually doing for review, or to send to me to debug the script and get it to work properly.

The script will now actually tell you where I installed things, instead of you having to hunt and guess (sorry about that..)

Additionally, this release sees support for Ubuntu 13.04 and Debian 7. Currently in the process of Verifying support for Kali Linux (Backtrack's newer, younger sibling), but since it's all supposed to based off Debian and follows the Debian software methodology, it should be pretty straightforward.

The child installer scripts all got this little makeover, and as previously mentioned also have logging built in, and write to their own log files in /var/log.

The only thing of note that changed here in addition to the minimizing of output would be that I finally added a prompt to run Aanval's BPU processes on boot via rc.local. That's something I've been meaning to do for a while.

So now that this is over with, the next step would be for me to focus on Getting CentOS updated -- Snorby and the cleaned up autosnort script. That is next up on my plate before enabling anymore features on any platform. period.

After that is finished, here's what I'd like to focus on:

- All web interfaces should have SSL/HTTPS enabled by default. No exceptions.
- Remove database/barebone sensors should have a method to secure database transactions to the "Master Console"
+ some point way far down the line, I want to begin experimenting with p0f. The project has recently gotten some new blood (as mentioned in a recent shmoocon), and there are some interesting possibilities I can think of where p0f and snort could play incredibly well off of one another:
--have p0f run for a set period of time, sample traffic off the network, fingerprint operating systems and write results to a database
--script something out that takes the database results and helps write frag3/stream5 reassembly policies based off p0f results.

Same as always, the scripts are available via The Autosnort Github Repo

That's about it for now. Happy snorting, hope you enjoy the new release!

DA_667

Autosnort: MSF edition

Working on some changes to Autosnort.

After seeing Darkoperator's MSF Installer script, I wanted to change how Autosnort operates a little bit.

You see, one of the cool things about metasploit is that It will log to hell and back if things are or are not working, but while you're actually running it, It only tells you what you need to know:
 "This is what I'm trying to do right now"
"This good thing happened"
"This bad thing happened"
"This is something to be aware of."

Keeping this in mind, and heavily ganking the notification printing code from darkoperator, in combination with a crazy named pipe trick I found via stack overflow, I'm changing how autosnort presents information.

As it stands currently, Autosnort, to put it bluntly, pukes all over the terminal/screen buffer. If something exploded in a bloody mess or failed to install correctly, you may see it and may be able to capture it in a screen shot and report it back to me, or you may not. Digital gods only know.

The way I've re-written the script, the only thing that gets printed to the screen are status messages indicating what the script is doing, good/bad messages to let you know if something failed or succeeded, and notifications that are sort of a "I need your input here" or "Pay attention to this!"

Apt-get (with the exception of the mysql-server installation that REQUIRES user input), configuration, compiler and all other output is redirected to log files.

For those of you who like seeing autosnort puke all over the screen or actually seeing what's going on behind the scenes, I found a nifty hack via stack overflow that uses mkfifo and named pipe magic to log everything from the script into a log file. No more running script to log output, no more screenshots, if you have a problem, you can e-mail me the output from the main autosnort script and/or the child interface installation scripts.

As of right now, I'm piloting this out on Debian, and it just looks slick. So far, I have the main script re-written and snorby's installation script re-written in this format. The other interface installers should fall in line easily enough, Snorby is the most complicated install I scripted out.

I haven't officially released either scripts out to github because I'm working on a couple other issues as well:

-- Support for both Debian 6 AND Debian 7 in the autosnort-debian main script
The only problem I'm seeing here at this point is that libmysqlclient.so (for barnyard2) installs in three different places in Debian 6 32/64 and Debian 7 32/64:
/usr/lib (Deb 6 32 and 64 bit)
/usr/lib/i386-linux-gnu (Deb 7 32)
/usr/lib/x86_64-linux-gne (Deb 7 64)

so I'm testing out a work-around involving the 'find' command and 'dirname' to fix this once and for all.

-- Support for Ubuntu 13.04
It's a short-term release, but some want to see support for it. It also appears to have an issue finding libmysqlclient.so

My theory is, if I can get find/dirname tested/working on Debian 6/7, likely I can get it to work between Ubuntu 12 and 13.04, hell I might be able to merge the Debian and Ubuntu autosnort scripts into a single script (since this was one of slight differences between the two).

So there's that.

I'm going to try and include a screenshot down here to preview what to expect, but so far, Blogger has somehow managed to not get my photos attached to my blog posts every time I've tried to use them.

First try:

Second try:
Via my twitter

Not snort-related, but a good thing to know: How to tunnel VNC over SSH

So, this isn't strictly snort or autosnort related, but I figured I would post it anyhow. I primarily use SSH to manage my autosnort testing sensors in my virtual environment, but I also use vnc over SSH to manage my Attack system (A Kali Linux box -- was formerly Backtrack 5 r3) and thought I would share my experience on how to set up vnc server over SSH and have it look somewhat decent.

I've found a blog post here and a blog post there about how to tunnel VNC over ssh, but nothing really comprehensive. So I thought I'd share my knowledge on the bits of wisdom I've found strewn all over the net. If you have anything to contribute, or anything to add, please do so.

To begin, VNC is old and insecure; It encrypts absolutely nothing. This is why we're going to wrap it over SSH. In addition to a lack of encryption, VNC4's password functionality only allows 8-character passwords. This means the password can be broken fairly easily, even with a random password.

Tunneling VNC over SSH allows you to use stronger authentication (keys or huge long passwords, or both) in additional to the VNC password, setting up a basic form of multifactor authentication.

Most debian-based distro package sources include vnc server. It can be installed via:

apt-get install vnc4server

Afterwards, you will need to run vnc4server manually or vnc4passwd to specify a password for the current user for using VNC. again, the password can only be 8 chars long.

Additionally, in the user's .vnc folder (e.g. ~/.vnc or /home/[user]/.vnc/ ) is a file entitled xstartup

xstartup must be executable, and looks something like this:

#!/bin/sh
# Uncomment the following two lines for normal desktop:
#unset SESSION_MANAGER
#exec /etc/X11/xinit/xinitrc

[ -x /etc/vnc/xstartup ] && exec /etc/vnc/xstartup
[ -r $HOME/.Xresources ] && xrdb $HOME/.Xresources

#xsetroot -solid grey
#vncconfig -iconic &
#x-terminal-emulator -geometry 80x24+10+10 -ls -title "$VNCDESKTOP Desktop" &
#x-window-manager &

 Change the file to look something like this:

#!/bin/sh
# Uncomment the following two lines for normal desktop:

unset SESSION_MANAGER
exec /etc/X11/xinit/xinitrc

[ -x /etc/vnc/xstartup ] && exec /etc/vnc/xstartup
[ -r $HOME/.Xresources ] && xrdb $HOME/.Xresources

#xsetroot -solid grey
#vncconfig -iconic &
#x-terminal-emulator -geometry 80x24+10+10 -ls -title "$VNCDESKTOP Desktop" &
#x-window-manager &
startx &;

if the .vnc directory doesn't exist, try running "vnc4server" to start up vnc. This should give you a .vnc directory and the file above. After starting it, immediately kill it via kill -9 or killall vnc4server.

Next, we need to make an init script to start up vnc at boot. I found this init script via

http://www.isnull.com.ar/2010/03/vnc4server-boot-script-working-in.html

I've made some minor improvements, however. Copy these contents to /etc/init.d/vncserver:



# /etc/init.d/vnc4server
#
# Some things that run always

touch /var/lock/vnc4server

# Carry out specific functions when asked to by the system
case "$1" in

start)
echo "Starting script vnc4server "
su root -c 'vnc4server -depth 24 -pixelformat RGB888 -localhost -geometry 1440x900 '
;;

stop)
echo "Stopping script vnc4server"
su root -c 'killall Xvnc4'
;;

*)
echo "Usage: /etc/init.d/vnc4server {start|stop}"
exit 1
;;

esac

exit 0

The line that is doing the all the work of running the vnc server is this line:

 vnc4server -depth 24 -pixelformat RGB888 -localhost -geometry 1440x900

This line is essentially saying "start the vnc server with 24-bit color depth, with this RGB mapping and a resolution of 1440x900, and only allow it to listen on 127.0.0.1" (This last part is important! do not forget the -localhost line!) You can change the -geometry line to any resolution you wish, but I had to figure out the -depth and -pixelformat lines on my own, and mainly through trial and error.



next you'll have to make the file executable (chmod u+x /etc/init.d/vncserver)

after making the file executable, run:

update-rc.d vncserver defaults

or

update-rc.d vncserver start [runlevels you want the server to run on]

finally, the file make the xinitrc file executable. I've found that vnc won't work properly unless this file is executable:

chmod u+x /etc/X11/xinit/xinitrc

The higher the bit depth and resolution, the more packets you need to be able to push over the network (and your SSH tunnel) to keep things running smoothly. For this reason, I recommend that if you're going to use this configuration that it be over a high-speed link, or a local network. try changing the depth to 16, or making the resolution lower if you find the VNC session to be laggy.

So to summarize what we just did above we:

installed vnc4server

set up a vnc password for your user

made an init script for vnc4server

made all necessary preparations to host vnc4 server.


In this section I'll show you how to SSH tunnel to your VNC server either from windows or linux.

We'll start with windows. I use an awesome program called MRemoteNG for managing remote connectivity via a number of remote protocols -- VNC, RDP, SSH, .ICA (citrix), HTTP/HTTPS (xulrunner-- a bit buggy), etc. I can't stress how awesome MRemoteNG is for managing multiple connections. For those worried about having the app store connection information, it also allows you to encrypt the connection configuration files to keep your creds safe.


Download and Install MRemoteNG. Open it up and click Tools::Options:: and click advanced. Click "Launch Putty"


Here is where we're going to configure the portforwarding settings. Under Category, go down to Connections, click the plus sign on SSH, then click tunnels.

This page looks confusing, but it's really simple.

The only things you need to modify are

Source Port: set this to any arbitrary port number you want. Doesn't really matter. Just remember what it is. If you can't remember, you can always come back to this page.



For destination type this in exactly: 127.0.0.1:5901

The more intrepid may ask "uh.. why am I putting localhost in as the destination?"

Because this is localhost on the REMOTE server (e.g. the system where VNC is listening on loopback (127.0.0.1) only. So what this tunnel is saying is:

"After this user has successfully authenticated to me, have the client open up source port:[source port you choose]. If you get a connection on [source port you chose], funnel it over port 22. When it reaches the remote end, connect it to 127.0.0.1:5901 where the VNC server is listening."

Save your putty session by scrolling up, clicking on sessions, putting in a name under "saved sessions" then clicking save. Afterwards, you can close putty.


You'll need to create two mremoteng sessions here:


the first Mremote session, select ssh version tool as the protocol, enter the IP address (user name and/or password if you want) and under PuTTY session, click the drop down and save the session you made.


the second Mremote session, you'll choose VNC as the protocol. Enter the IP address "127.0.0.1" and the source port that you entered when you created the putty session above. For the password, enter the password you entered when you ran vnc4passwd or vnc4server previously.


Next, double click on your ssh session in mremoteng, then double click on your localhost VNC session. You should have your remote system's desktop.

If you're doing this over Linux, it's not that much harder to do. Make sure your linux  connecting to the remote SSH (and VNC) server has an ssh client and a vncviewer application of some sort.


Open up a terminal window and type:

ssh -L 5901:localhost:5901 username@[hostname/ipaddress]

The -L line above is the secret sauce and works exactly the same as the what the putty session does:

[Source Port to listen on locally]:[target remote ip address]:[remote port to connect to]

So we tell SSH to open a listener on 5901 if your local workstation gets a connection on 5901, forward it over the SSH connection and send it to 5901 on the remote server's loopback address.

Next, just tell your vnc client to connect to 127.0.0.1:5901 and you're done.




Note: Some of the more intrepid among you may notice that there's now a listener on 6001/tcp. I've searched long and hard for the reasoning behind this. 6001/tcp is the default port for the first X server screen to use when X is told to listen for remote X client connections.


Some of you are probably freaking out. you don't want your server to allow connections to X windows. Don't freak out just yet; Sure enough, if you get NMAP scanned, NMAP will say the port is open. Netcat will connect and promptly get dropped. PuTTY RAW sessions stay open (probably because the remote X server has no idea what the heck the PuTTY RAW session is trying to do.) No remote X clients will be able to connect to the x server. While you are on your vnc session, If you type "xhost" in a terminal window "access control enabled, only authorized clients can connect" should be the response you get. In the olden days, way before even my time, No one thought to tunnel X over SSH. so they would use xhost to authorize remote users to connect to that system's X windows server. The format was usually xhost +{ip address} to allow a remote ip address to connect to your X server. Even more common was for most to just type xhost + (no ip address). This is the same as saying "Allow anyone to connect to my X server". You can probably imagine why this isn't a good idea.


So why am I telling you about this? Because by default, your X server (the one that VNC uses), for some stupid reason  has your X11 X windows server listen on 6001/tcp. From what googling I've done, there's no reason for this, and there's no way to configure X (without modifying the source code) to just listen on localhost. I've tried to find a solution that isn't "turn on your firewall, or just configure firewall rules to block it." That's a work-around, not a solution (A viable work-around, by the way, but not the point.) and I haven't find any (other than the just a firewall rule)


For those of you who aren't firewall gurus, you're in luck. by default, xhost doesn't authorize ANY remote users at all to connect to your X server. That means, as soon as anyone tries to connect to your X server, the session is dropped by X's own authorization system. The bottom line is: Don't freak out; no one is allowed to connect to it. You're safe. But if you know how to make X listen on 127.0.0.1 only, I'd be very interested...

Final Note: On some sshd servers, TCP Forwarding is disabled. TCP forwarding is the magic by which all this tunneling happens. Meaning if it's disabled, then none of this works. Check /etc/ssh/sshd_config if you see the line "AllowTcpForwarding no" it means tcp forward is turned off. Conversely, if it is set to yes, then you're good to go. If you don't see this line in your sshd_config file, then you're good to go as well. It's implicitly allowed by default.


 

Troubleshooting your Snort deployment

I recently had a discussion with a user new to snort and autosnort who wanted to know some troubleshooting techniques that can be done to see if snort is working and/or snorby or the web interface of your choosing is working. After thinking about it, I decided it might be a good idea to post my troubleshooting steps here to assist users of snort and snorby in general to determine if things are working okay.

Before we begin here, realize that there are a number of moving parts with any IDS/IPS deployment.  Understanding the basics of networking (OSI/TCPIP stacks, collision domains and/or broadcast domains) really help you in understanding why snort can or cannot see traffic for certain parts of your network. If you have no way of copying traffic from the network segment you want to protect, then snort will not be able to see anything.

Now that I got that part out of the way, there are a couple of basic components in any snort deployment:

1) the network

How busy is your network? If the network isn't particularly active, it may take a while before you see any sort of alerts from snort.

2) the method you are using to pass network traffic to snort

How are you copying traffic to snort's sniffing interface? Do you have an old hub? a professional grade network tap or hobbyist network tap? A switch that can do port mirroring/span ports? Is snort sitting in a VM? does that virtual machine have a way to sniff traffic off of the vswitch?

Any IDS/IPS deployment needs a reliable method to clone/copy traffic off of the network. If you plug snort into a regular switch port or SOHO router, you're usually only going to see broadcast traffic (usually UDP broadcasts, CDP broadcasts, Spanning tree traffic, and ARP requests) and will be absolutely useless to you.

You need to have the equipment or means to copy traffic to snort somehow.

3) Snort itself and how it's configured

If you have a light ruleset (say the balanced ruleset from pulledpork, or the connectivity over security ruleset), you won't be seeing very many alerts, unless those rules are a really big deal, like big enough to be included in those rulesets, and/or if your network is particularly infested with bad traffic.

4) Barnyard2, how it's configured and/or the database it is logging to

Autosnort is configured to use barnyard2 and log to a local database. normally this is the "snort" database, but some web interfaces require their own database (e.g. Aanval uses the aanval database and snort database, snorby uses the snorby database). Does barnyard2's database user have the right access to insert events to the database? does the database user have the right credentials?

For other barnyard2 logging options, such as syslog, is the remote host allowing connections from your sensor's management interface? is the SIEM solution configured to log syslog events properly?

So, to summarize IDS/IPS deployments have a number of parts that have to work together to provide you with a working solution: The network itself, the method you are using to copy network traffic to snort, snort or the IDS/IPS solution you are using and what its configured to alert on, and the method by which you are logging intrusion events (barnyard2/database/syslog).

Here are some basic troubleshooting methods I've devised:

Basic Steps:

1) are snort processes running? If you're using Snorby or another specialized logging interface, are the processes that interface relies on running?

ps -ef | grep snort

ps -ef | grep delayed

you are looking for this output from ps -ef | grep snort:

root      1853  1258  0 17:54 pts/0    00:00:00 grep snort
snort     1883     1  0 May18 ?        00:00:47 /usr/local/snort/bin/snort -D -u snort -g snort -c /usr/local/snort/etc/snort.conf -i br0
root      1912     1  0 May18 ?        00:00:53 /usr/local/bin/barnyard2 -c /usr/local/snort/etc/barnyard2.conf -d /var/log/snort -f snort.u2 -w /var/log/snort/barnyard2.waldo -D

you are looking for this output from ps -ef | grep delayed:

root      1869  1258  0 17:55 pts/0    00:00:00 grep delayed
root      2161     1  0 May18 ?        00:11:01 delayed_job

If snort and barnyard2 BOTH are not running, check /var/log/messages (cat /var/log/messages [or /var/log/syslog depending on your distro] | grep snort) to see if you can determine why. Look for error, critical or fatal messages as they indicate reasons why snort or barnyard2 refused to run. We'll talk about snorby's delayed_job process in a moment.

2) ifconfig -a

-- which interface is snort sniffing traffic on? is it in promiscuous mode?

You're looking for output that looks something like this on the interface that snort is sniffing traffic on, pay attention to the lines in bold face:

eth1      Link encap:Ethernet  HWaddr 00:0c:29:8b:bb:ca
          inet6 addr: fe80::20c:29ff:fe8b:bbca/64 Scope:Link
          UP BROADCAST RUNNING NOARP PROMISC  MTU:1500  Metric:1

The most important lines above are UP RUNNING PROMISC, because the interface must be up, must be running, and likely should be in promiscuous mode to pick up traffic. the NOARP line isn't entirely necessary, but it's there to ensure that the sniffing interface doesn't respond to arp queries in case attackers are probing the network to try and find your IDS installations.

3) How are you passing traffic to snort? is snort's sniffing interface attached to a switch span? a hub? a network tap? if it's a virtual machine on ESXi or something of that nature, is the vswitch snort is sniffing off of configured to allow promiscuous mode network cards?


If you're in a corporate environment, and are a proficient network administrator, I'm going to assume you know what a switch span port is. These are also referred to as mirror ports. Most professional grade switches have an option to copy ALL traffic seen on a switch to a single interface. You would plug snort's sniffing interface into this mirror interface for snort to see traffic for that entire network segment.

If you're a security reasearcher and don't have money for a procurve or cisco switch that does port mirroring, here's some awesome alternatives:

http://routerboard.com/RB260GS

This is a 5-port switch from microtik. Very very affordable and capable of port mirroring.

Alternatively, if you want a bigger name-brand switch, I personally use netgear's GS108T switch for my home network, since it was the cheapest switch I was aware at the time that allowed port mirroring:

http://www.newegg.com/Product/Product.aspx?Item=33-122-381&IsVirtualParent=1

An alternative to span/mirror ports are network taps. On the professional/corporate side, Net Optics, Gigamon and several other vendors offer network tap solutions. Often ridiculously priced, but without all of the possible problems that come with setting of span/mirror ports. These devices are solely dedicated to creating exact clones of network traffic passing over the wire and nothing else.

For those on the cheap side, there are several results via google on how to make your own network tap, but be aware that these sorts of taps are usually for 10/100 networks and usually do not support tapping gigabit networks.

Finally for vmware environments, I posted a specialized tutorial on how to "hack" vmware player on how to bridge traffic from one physical interface or another, using the vmnetconfig.exe from vmware workstation (check the pdf document available in my autosnort github if you're interested...)

But for professional deployments or network labs running vmware ESX/ESXi, it's pretty easy to configure a vswitch to allow promiscuous mode nics. That's really all you have to do is change vswitch security settings, allow promiscuous mode NICs and just configure the snort VM to sniff off of that vswitch.


4) run tcpdump on snort's sniffing interface:

tcpdump -i ethX not arp and not port 22

X is the ethernet interface number snort is sniffing traffic on. the "not arp and not port 22" is telling tcpdump to ignore ssh traffic and arp broadcasts.

while tcpdump is running, on another system on the network, try doing something to generate network traffic (ping google.com, go to a web page, etc.). did tcpdump see your traffic? If not, you may need to investigate how you're mirroring traffic to snort.

5) Check and see if the snort.u2 file is greater than 0 bytes:

ls -al /var/log/snort

check the output of the ls command. You're looking for the snort.u2 file, it should look something like this:

-rw-------  1 snort snort 67261 May 19 17:39 snort.u2.1368917933

the numbr "67261" in the middle is the file's size. If your snort.u2 file is 0 bytes, that means snort hasn't generated any alerts.

So those are the basic steps. You confirm that snort is running, confirm that the interface is in promisc mode, and confirm that mirroring device is actually pushing traffic to where snort can see it, and confirm snort is logging events.

Snorby-specific troubleshooting steps:

Here's a couple of things you can try to confirm that barnyard2 and snorby are working together:

1) ps -ef | grep delayed

- You are looking for an entry that looks something like this:

root      2161     1  0 May18 ?        00:11:00 delayed_job

if the delayed_job process is NOT running, run this command:

cd /var/www/snorby && ruby script/delayed_job start

then run this command to see if the sensor_cache needs to be updated:

cd /var/www/snorby && rails runner 'Snorby::Jobs::SensorCacheJob.new(false).perform; Snorby::Jobs::DailyCacheJob.new(false).perform'

note: I do not know if you need root permission to run the commands above, but if the above commands fail without root privileges, run the command:
 sudo su -

to get a shell with root privileges and re-run those commands.

2) you'll want to check the contents of barnyard2.conf and snorby's database.yml to make sure they match:

cat /usr/local/snort/etc/barnyard2.conf

on the log mysql line, check these values and make sure they match:

log,mysql, user=snort password=[password you gave the snort database user] dbname=snorby host=localhost

cat /var/www/snorby/config/database.yml

verify these lines in database.yml:

snorby: &snorby
  adapter: mysql
  username: snort
  password: [same password in the barnyard2.conf file]
  host: localhost

development:
  database: snorby
  <<: *snorby

test:
  database: snorby
  <<: *snorby

production:
  database: snorby
  <<: *snorby

the most important lines to check are the dbname: and database: line under production are the same database, and that the password= and password: lines are the same to make sure that barnyard2 and snorby agree on the correct password to use to log to and read from that same database.

If the database lines and password lines match, at the very least, we know that barnyard2 and snorby are talking to the right database.

3) One last check you can perform is to verify that the snort user has the privileges to talk to the snorby database. run this command:

mysql -usnort -p[snort database user's password, no spaces] snorby -e "show tables;"

your output should look like this:

+-------------------+
| Tables_in_snorby  |
+-------------------+
| agent_asset_names |
| aggregated_events |
| asset_names       |
| caches            |
| classifications   |
| data              |
| delayed_jobs      |
| detail            |
| encoding          |
| event             |
| events_with_join  |
| favorites         |
| icmphdr           |
| iphdr             |
| lookups           |
| notes             |
| notifications     |
| opt               |
| reference         |
| reference_system  |
| schema            |
| search            |
| sensor            |
| settings          |
| severities        |
| sig_class         |
| sig_reference     |
| signature         |
| tcphdr            |
| udphdr            |
| users             |
+-------------------+

This confirms that the snorby database exists and that the schema was configured properly during installation.

4) If the snort.u2 file in /var/log/snort is NOT 0 bytes, and you confirmed the snort database user was able to see the tables in the snorby database, there's one last set of commands you can run:

mysql -usnort -p[snort database user's password] snorby -e "select * from event\G;"

if you get "empty set" as the return value from that command, snorby doesn't have anything. If you get a bunch of data from this command, snorby should have events in the "events" tab on the web interface.

you can also run this command:

mysql -usnort -p[snort database user's password] snorby -e "select count(*) from event;"

you should get something like this:

+----------+
| count(*) |
+----------+
|     5284 |
+----------+

I have a boatload of events, because I have a testing suite that causes snort to trigger alerts like crazy, but basically, you're looking for the count number to NOT be 0. If the count is 0 there's no events for snorby to report on.


Hope this helps you all out

Cheers,

DA_667