INF526: Secure Systems Administration SIEM and Intrusion

INF526: Secure Systems Administration SIEM and Intrusion Detection Prof. Clifford Neuman Lecture 12 5 April 2017 OHE100C

Class Presentation Schedule 4/5 Mohammed Alsubaie – SIEM and Intrusion Detection 4/12 Vishnu Vadlamani - Network Monitoring/Attack Forensics 4/19 Andrew Gronski - Accreditation and acceptance testing 2

Detecting Intrusions Visibility of artifacts from intrusions – Passive attacks Difficult to detect because technically no changes made. – But in some cases – evidence of connecting to a network might be present. – Honeytokens – Intrusions are really all Active attacks Identifying Subversions Identifying traffic that is indicative of an intrusion – Best point of observation is outside affected system 3

Where Detection Occurs Events visible in the network – New network peer entities is evidence – Significant change in frequency or bandwidth is evidence Events on the Compromised System – Changes to system binaries are evidence of subversion – Changes to accounts and privileges are evidence – Changes to the running processes and CPU share are evidence – Creation of new files are evidence The above are anomalies and an administrators role is to sort though them. 4

Events Visible in the Network Network Monitoring uses a system to monitor a computer network for abnormalities and notifies an administrator or other system components when an issue is identified. – Enterprise health monitoring Network Monitoring tools allow us to see the devices connected to our network and traffic between them. Packet analyzers can capture network traffic for viewing through an event management system. Such data is useful for intrusion detection. 5

Network Monitoring and Secure SA Secure administration of a network and systems requires visibility of current activities and status of devices, and the ability to examine past activities. In particular: – Ensures Availability is maintained Ability to correct system issues, and to identify potential DoS attacks – Ensures Non-Repudiation Gives administrator a history of who did what, and when While not directly involved in enforcement, it also assists in ensuring other principles (Integrity, Confidentiality, Authentication, Authorization) have not been breached. – Can identify suspicious activity that may indicate a security compromise. Slide by Josh McCamey INF526 Summer 2016 6

How Does it Work Enterprise Network Monitoring systems usually check the health of systems on the network by using a “ping” signal to test whether ports respond as expected. – Period between pings can be anywhere from minutes to hours – Checks response times and up-time – May also check for things like consistency and configuration of specific equipment or software on the network Can check just about any network protocol (e.g. HTTP/S, SNMP, FTP, SMTP,POP3, IMAP, DNS, SSH, TELNET, SSL, TCP, etc ) – In the case of a protocol like SMTP/POP3, system may send a test message periodically through one protocol while receiving through another to the health of servers the message passes through. Slide by Josh McCamey INF526 Summer 2016 7 check

How Does it Work A failure occurs when a test operation or “ping” times out – When a failure is recorded the Network Monitor produces an “action” – System can use any available means to notify an administrator (e.g. e-mail, text, phone call, pop-up notification on local machine, even an audible alarm) Advanced networks may have automatic fail-safes that off-load a server’s duty to the rest of the network to make the failure transparent to the user – This is critical for big companies/sites that can’t afford down-time. Slide by Josh McCamey INF526 Summer 2016 8

How Does it Work Packet analyzers, on the other hand, capture packets going across an interface – Sometimes these are also called “packet sniffers” – Close relationship to forensics and intrusion detection – Can be used to determine the nature of connections and identify malicious activity Wide variety of visualization, recording and analysis capabilities depending on thetool used Packet analyzers and health monitors not necessarily mutually exclusive – Many different tools, some can “do-it-all”, depending on your definition of “all” Slide by Josh McCamey INF526 Summer 2016 9

Common Tools (not a compressive list) ntop Microsoft Message Analyzer MRTG PRTG Nagios OpenNMS SolarWinds Spiceworks IT Management Advanced IP Scanner Capsa Fiddler NetworkMiner Pandora FMS Zenoss Core The Dude Splunk Angry IP Scanner Icinga Total Network Monitor NetXMS Xymon And WIRESHARK Slide by Josh McCamey INF526 Summer 2016 10

Microsoft Message Analyzer Successor to Microsoft Network Monitor – Free, 32-bit and 64-bit: Windows 7, Windows 8, Windows 8.1, Windows Server 2008 R2, Windows Server 2012, Windows Server 2012 R2, and Windows 10 clients Capture, display, and analyze protocol messaging traffic Trace and assess system events and other messages from Windows components Main Features: – – – – – – – – – Integrated “live” event and message capture at various system levels and endpoints Parsing and validation of protocol messages and sequences Automatic parsing of event messages described by ETW manifests Summarized grid display – top level is “operations”, (requests matched with responses) User controlled “on the fly” grouping by message attributes Ability to browse for logs of different types (.cap, .etl, .txt) and import them together Automatic re-assembly and ability to render payloads Ability to import text logs, parsing them into key element/value pairs Support for “Trace Scenarios” (one or more message providers, filters, and views) Slide by Josh McCamey INF526 Summer 2016 11

Microsoft Message Analyzer Slide by Josh McCamey INF526 Summer 2016 12

Nagios Network Analyzer Popular paid IT admin tool to monitor for configuration issues, high load, etc. Focused on high-level goals rather than low level packet inspection – – – – – – Netflow Analysis, Monitoring, and Bandwidth Utilization Security and reliability monitoring and user notifications Advanced visualization capabilities Custom application monitoring Bandwidth monitoring and threshold settings with alert capability Advanced user management to allow for multi-tenancy Linux focused Specialized versions for machines running in virtual environments like VMWare Slide by Josh McCamey INF526 Summer 2016 13

Nagios Network Analyzer Slide by Josh McCamey INF526 Summer 2016 14

Open NMS Free & open-source enterprise grade network monitoring & management tool Runs on Linux/Win/Mac “Network Management Application Platform” – Made to be customized and have network monitoring solutions/applications built on top of it Core feature set – – – – – – – – Java-based framework API-extendable notification mechanism including SMTP, Slack & Mattermost, Jabber and XMPP, Microblogs (i.e. Twitter) Ticketing integration framework including Request Tracker, BMC Remedy, IBM Tivoli Service Request Manager, Atlassian JIRA, extensible ticketing API Alarm forwarding to external applications for handling or analysis (ElasticSearch), “Northbound” API and JMS, AQMP integration “Southbound integration” enables gathering of performance measurements and logging messages from network applications and devices (test connections, check resource configurations, etc.) Slide by Josh McCamey INF526 Summer 2016 15

Open NMS Slide by Josh McCamey INF526 Summer 2016 16

NTopNG Network traffic probe that shows network usage similar to Linux “top” command – – – – – – – – – – – – – – – – Full-featured network analysis tool, free & paid versions, Portable, made to run on any Unix platform (&Win x64) Uses a web server to provide a web browser interface Sort network traffic according to many criteria including IP address, port, Layer 7 protocol (app layer), throughput, and more Show network traffic and IPv4/v6 active hosts. Produce long-term reports about various network metrics such as throughput, application protocols Top X talkers/listeners, top ASs, top Layer 7 applications For each communication flow report network/application latency/round trip time (RTT), TCP stats (retransmissions, packets OOO, packet lost), bytes/packets Store on disk persistent traffic statistics in RRD format. Geolocate hosts and display reports according to host location. Discover application protocols by leveraging on nDPI, ntop’s DPI framework. Characterise HTTP traffic by leveraging on characterisation services provided by Google and HTTP Blacklist. Show IP traffic distribution among the various protocols. Analyse IP traffic and sort it according to the source/destination. Display IP Traffic Subnet matrix (who’s talking to who?) Report IP protocol usage sorted by protocol type. Produce HTML5/AJAX network traffic statistics. Slide by Josh McCamey INF526 Summer 2016 17

NTOP NG Slide by Josh McCamey INF526 Summer 2016 18

Wireshark World's foremost network protocol analyzer (self-claimed) – – – – – – – – – – – - Standard for analyzing traffic for many industry and educational purposes – - Deep inspection of hundreds of protocols, with more being added all the time - Live capture and offline analysis - Multi-platform: Runs on Windows, Linux, OS X, Solaris, FreeBSD, NetBSD, and many others - Captured network data can be browsed via a GUI, or via the TTY-mode TShark utility - Rich VoIP analysis - Read/write many different capture file formats (too many to show here) - Capture files compressed with gzip can be decompressed on the fly - Live data can be read from Ethernet, IEEE 802.11, PPP/HDLC, ATM, Bluetooth, USB, Token Ring, Frame Relay, FDDI, and more - Decryption support for many protocols, including IPsec, ISAKMP, Kerberos, SNMPv3, SSL/TLS, WEP, and WPA/WPA2 - Coloring rules can be applied to the packet list for quick, intuitive analysis - Output can be exported to XML, PostScript , CSV, or plain text - Similar to tcpdump w/GUI - Paid add-ons (SteelCentral) can make more effective for large-scale monitoring Slide by Josh McCamey INF526 Summer 2016 19

WireShark Slide by Josh McCamey INF526 Summer 2016 20

Deployment of Network Monitoring Switch Spanning Port Can miss traffic during burst periods Network Tap Usually a temporary /emergency solution Not commonly found, used when options A&C not possible Inline Makes all network connectivity dependent on sensor health Other Bottom line, your network monitoring device must be deployed in a way that enables it to see/monitor all relevant traffic. Slide by Josh McCamey INF526 Summer 2016 21

Summary of Network Monitoring There are multiple focuses of Network Monitoring – – – Focus on Health Internally vs. Externally Geared All play a role in Secure Systems Administration Works either via actively inquiring about network devices or passively observing – – Ping and inquiries Packet Inspection/Recording/Analysis Many different ways for admins to interact with systems and receive notifications – – Advanced Visualization Notifications, Automated Response A tool for every use case (including frameworks to build your own) Can be as high-level or as granular as you need Close relationship between monitoring and IDS/Forensics Slide by Josh McCamey INF526 Summer 2016 22

Intrusion Detection Signature Based – Specific characteristics of known attacks and attack tools are maintained, and network based IDS’s scant traffic looking for these patterns. – Host based IDS’s scan system looking for these patterns. – Administrator installs IDS and keeps signatures up-todate. Anomaly Based – In the examples from the previous slide many anomalous events will be normal or innocuous. – Administrator must add to patters of normal events or biome overwhelmed with false positives. 23

Gathering Information Useful Reading: SANS Institute InfoSec Reading Room - Logging and Monitoring to Detect Network Intrusions and Compliance Violations in the Environment by Sunil Gupta. Data Sources – – – – Syslog SNARE agent Dedicated monitor on host Bro for network event data Collected by – An event collection system E.g. a SNARE Server 24

Application of NM to Intrusion Detection An intrusion detection system (IDS) monitors network activity for malicious behavior or policy violations – Clearly, network monitoring and intrusion detection go hand in hand Multiple ways to detect intrusions – Can use a unified application to analyze traffic and identify anomalies – Have a separate application analyze logs generated by a network monitor Many applications can provide this functionality – Later We’ll demonstrate using our “de facto standard” (self-proclaimed): Wireshark – Examples taken from “Intrusion Detection Using Wireshark” by S. Gupta and R. Mamtora – (International Journal of Advanced Research in Computer Science and Software Engineering, Volume 2, Issue 11,2012.) Slide by Josh McCamey INF526 Summer 2016 25

Wireshark ID Example Using Wireshark - Expert Info - Each will contain the following: – – – – Chat (grey): information about usual workflow, e.g. a TCP packet with the SYN flag set Note (cyan): notable things, e.g. an application returned a "usual" error code like HTTP 404 Warn (yellow): warning, e.g. application returned an "unusual" error code like a connection problem Error (red): serious problem, e.g. Malformed Packet Malformed packets can indicate malicious activity Slide by Josh McCamey INF526 Summer 2016 26

Wireshark ID Example Using Wireshark - Chats TCP connections should each have SYN, SYN ACK, ACK sequence Sequence of SYNs could indicate DoS Slide by Josh McCamey INF526 Summer 2016 27

Wireshark ID Example Same thing can be done with Wireshark - Flow Graph Continuous barrage of SYNs on various ports with no other messages (i.e. ACK) indicates DoS Slide by Josh McCamey INF526 Summer 2016 28

Wireshark ID Example Using Wireshark - Conversations Shows all interaction between two endpoints (e.g. two IP addresses) Add a column to each row called “conclusion” which contains either “normal” or “abnormal”. Set all values to “abnormal” and update to “normal” when behavior is verified (e.g. packet from A B and B- A that is not equal to zero, which reflects a connection being established) Can apply any data mining technique such as association rule, classification, clustering, etc. to find rules that reflect connection legitimacy. Figure on bottom-right shows that if the port number is less than 1066 and packet sent is greater than 3,then all data sent is normal; otherwise, it’s abnormal. Slide by Josh McCamey INF526 Summer 2016 29

Visualizing the Data Security Incident Event Management (SIEM) – Collects data from many sources Host, network, and application – Converts data into a common database format – Enables search and analysis of the collected data – Provide visualization tools that allow one to push down on and query for statistics from its database of events. – Examples: Snort, many many commercial products Qradar, OSSIM, others 30

Filtering the Data Signature Based – Looking at known attacks Anomaly Based – How to define normal – Baseline for system – Baseline for network Log Analysis and Correlation – Categorize events (as authorized) – Drops from view of things to be monitoried Reduces false positives Lets brainstorm on how to do this. – Examples from your exercises? 31

Brainstorming: How and What Unexpected Flows – What might this indicate, Where might we see these – Filtering the expected Some events should only occur following specific administrative action. – Consider privileged operations on systems – What does root do Forking / creation of new processes Logins DNS Lookups Access to specific files Installations of software – new executables 32

Monitoring from the Outside If sensors on affected system, send data to SIEM as soon as possible. – Actions leading to a breach are moved off system before breach compromises reporting. Monitor from Host OS/Hypervisor – Breach of guest does not affect such reporting. Monitor network activity through appliance or embedded device. – Requires compromise of separate device to subvert reporting. 33

Taking Action The purpose of intrusion detection is to provide the administrator with actionable intelligence. – Actions are taken based on that insight. Actions – Shut down the affected systems – Restore to a known state – Block access from parts of the internet Some actions can be taken automatically – Simply closing a vulnerability might be too late. 34

IDS and SIEM Mohammad AlSubaie

Outline Motivation Intrusion Detection Systems – Types – Detection Methods Security Information and Event Management – Features – Workflow

Motivation No system is 100% secure We can’t prevent all break-ins There will always be new holes, new attacks, and new attackers We need some way to cope

Defense in Depth Single defenses can and will fail Instead, we need multiple layers of defense of different designs and objectives

Network Defense in Depth 1 2 3 4 5 6 7 8 1. Advance Threat Protection e.g. FireEye, Cisco/IronPort 2. Intrusion Detection Systems e.g. McAfee 3. Web Security e.g. Fortinet 4. Email Security e.g. Bluecoat, Cisco/IronPort 5. Forensics Analysis e.g. RSA/NetWitness 6. Data Loss Prevention (DLP) e.g. TrendMicro 7. Next-Gen Firewalls e.g. Palo Alto Networks 8. Security Event Monitoring e.g. HP/ArcSight

Network Defense in Depth 1 2 3 4 5 6 7 8 1. Advance Threat Protection e.g. FireEye, Cisco/IronPort 2. Intrusion Detection Systems e.g. McAfee 3. Web Security e.g. Fortinet 4. Email Security e.g. Bluecoat, Cisco/IronPort 5. Forensics Analysis e.g. RSA/NetWitness 6. Data Loss Prevention (DLP) e.g. TrendMicro 7. Next-Gen Firewalls e.g. Palo Alto Networks 8. Security Event Monitoring e.g. HP/ArcSight

Intrusion Detection System A device or software application that monitors a network or systems for malicious activity or policy violations. [CIA violation] Types of Intrusion Detection Systems – Network-based IDSs – Host-based IDSs

Network-based IDS Deploying sensors at strategic locations – E.G., Packet sniffing via tcpdump at routers Inspecting network traffic – Watch for violations of protocols and unusual connection patterns Monitoring user activities – Look into the data portions of the packets for malicious command sequences May be easily defeated by encryption – Data portions and some header information can be encrypted

Network-based IDS Countermeasure Alarms Policy Event Analyzer Event stream Event Storage Filtered packet stream Configuration Packets Collector Packet stream Sensor Sensor SYSTEM Sensor Sensor

Firewall versus Network IDS What is the difference between firewalls and network-based Intrusion Detection Systems?

Firewall vs. Network-based IDS What is the difference between firewalls and network-based Intrusion Detection Systems? Firewall – Active filtering – Fail-close Network-based IDS – Passive monitoring – Fail-open FW IDS

Host-based IDS Using OS auditing mechanisms – EventLog: Windows Event Logs, System, Security, Application. – Netstat: Read information about network usage. – Health: Read information about CPU, Memory, and Swap usage. – Ps: Read information about running processes Monitoring user activities – E.g. analyze shell command Monitoring execution of system programs

Host-based IDS Configuration Policy Alarms Detector Vulnerability Scanning Audit logs SYSTEM Countermeasure Actions

IDS detection Methods Knowledge-based Specification-based Behavior-based

Evaluation of IDS Accuracy Completeness Performance Fault tolerance Timeliness

Knowledge-based IDS High accuracy, but low in completeness Drawback: need regular update of knowledge (malware signatures) High performance with minimal processing power

Specification-based IDS Manually develop specifications that capture the legitimate system behavior. Any deviation from it is an intrusion. Pro: can avoid false-positives since the specification can capture all legitimate behavior. Con: hard to develop a complete and detailed specification.

Behavior-based IDS High in completeness, but low accuracy Detect intrusion by observing a deviation from the normal or expected behavior of the system or the users Can detect attempts to exploit new and unforeseen vulnerabilities (zero-day attacks)

Typical Corporation Environment Remote User Internet Intrusion Prevention System Router Database Server Web Server App Server DC Server EPO Server Email Server Switch Switch DNS Server

Typical Corporation Environment Remote User Internet Intrusion Prevention System Router Database Server Web Server App Server DC Server EPO Server Email Server Switch Switch DNS Server

Typical Corporation Environment Remote User Internet Intrusion Prevention System Router Database Server Web Server App Server DC Server EPO Server Email Server Switch Switch DNS Server

Security Information and Event Management (SIEM) SIEM is a system combining Security Information Management (SIM) and Security Event Management (SEM). SEM deals with – – – – Real-time monitoring Correlation of events and threat intelligence Notifications Console views SIM deals with – Long-term storage – Analysis and reporting of log data

SIEM Features * Source “ManageEngine” https://www.manageengine.com/products/eventlog/manageengine-siem-whitepaper.html

SIEM Workflow Collect Data from Log Sources Correlate Events Alerts Security Incidents Generate Security and Compliance Reports Archive Logs for Forensic Analysis

SIEM Workflow Collect Data from Log Sources Correlate Events Alerts Security Incidents Generate Security and Compliance Reports Archive Logs for Forensic Analysis Methods of collecting data from sources Data Source Collector Data Source Aggregation: to gather data together as a whole in singular repository Normalization: to create consistent records by type and format

Normalization Original log format from source 1 10:32, 12/3/2017, alsubaim, ad.corporate.com, error, failed login attempt Original log format from source 2 12:45, 3/23/2017, malicious code detected, host1.corporate.com, alsubaim Normalized logs 10:32, 12/3/2017, alsubaim, ad.corporate.com, failed login attempt 12:45, 23/3/2017, alsubaim, host1.corporate.com, malicious code detected

SIEM Workflow Collect Data from Log Sources Correlate Events Alerts Security Incidents Generate Security and Compliance Reports Archive Logs for Forensic Analysis Link events to identify attacks Event based: – a single event identifies an attack Rule based: – If X Y Z then do A – If X repeated 3 times within an hour, then do Y Anomaly based: – If the traffic on port X exceeds the standard deviation of historic traffic patterns, then there may be a problem

SIEM Workflow Collect Data from Log Sources Correlate Events Generate Security and Compliance Reports Alerts Security Incidents Archive Logs for Forensic Analysis Severity Low Medium High Critical Notification: upon identifying a threat, notifications are sent to the security administrators (SOC) Automated Response: the majority of SIEM tools can execute external scripts to react on identified threats. (Change to FW rules, issue a Remedy ticket)

Security Operations Center (SOC)

SIEM Workflow Collect Data from Log Sources Correlate Events Alerts Security Incidents Generate Security and Compliance Reports Archive Logs for Forensic Analysis

SIEM Workflow Collect Data from Log Sources Correlate Events Alerts Security Incidents Generate Security and Compliance Reports Archive Logs for Forensic Analysis Collected log data is stored for future forensic investigations. Not equivalent to Log Management Solutions

SIEM vs. LM Functionality Log collection Log pre-processing Log retention Reporting Analysis Alerting and notification Other features Security Information and Event Management Security related logs Parsing, normalization, categorization, and enrichment Retain parsed and normalized data Security focused reporting Correlation, threat scoring, event prioritization Advanced security focused reporting Incident management, analyst workflow, context analysis, etc. Log Management All logs Indexing, parsing, or none Retain raw log data Broad use reporting Full text analysis, tagging Simple alerting on all logs High scalability of collection and storage

HP ArcSight (SIEM)

Why SIEM Implementation Fails? Lack of planning – No defined scope Faulty Deployment Strategies – Incoherent log management data collection – High volume of irrelevant data can overload the system Operational – Lack of management oversight – Assume plug and play Security is a process not a product

Thank You! Question?

Second Exercise - Criminal Enterprises Chosen because of differences in the high level principles. – Not because I expect you to implement these kinds of systems in your future endeavors . – But you may be called upon to break some of these systems if later employed by government organizations. Your organization must: – – – – – Accept Bitcoin as payment (not really, but it must accept something that stands in for bitcoin) Manage an inventory of stolen account identifiers with passwords Control access to such information Prevent collection of evidence or intelligence by third parties. Note, do not deal in any illegal goods, but use dummy information to stand in for such goods. Also, do not use terms associated with such illegals goods or information in communications, make up new names for this dummy information. 70

Teams for Second Exercise Group A – Enrolled: Srivastav, Binkulaib, Dmytrisin, Alsubaie, Bubshait, Gronski – Advisors: haiboz, jaskaran, rajattan Group B – Enrolled: Guerrerobognoli, Li, Wright-Walker, Jackoski, Vadlamani, Zhang – Advisors: sajmani, slakkadi, tosirisu, wang907 71