This project was born out of the partnership between QCRI, and various government and public stakeholders involved in the National Security Operations Center (NSoC) initiative led by the Ministry of Interior (MOI). As part of this partnership, QCRI established and manages the National Cyber Security Research Lab (NCSRL) which comprises three units: (i) The Secure Data Lab, an isolated lab that stores sensitive NSoC logs collected from various government agencies and public organizations. The lab has a direct connection to the NSoC datacenter via a point-to-point GNET link, allowing it access to all NSoC logs and providing security intelligence to participating stakeholders. (ii) The Cyber Range serves as a training ground for students, practitioners and security operators on cyber security defensive and offensive techniques, and as a testing ground for researchers to advance the state-of-the-art in cyber security technologies and for stakeholders to better understand the security posture of their organizations by testing an emulated version of it in the cyber range. (iii) The Security Operations Center (SoC) that supports the cyber range adversary emulation and defense games and also used to replay data received from the NSoC for ground-truth mining.

1. Spam and phishing in logs: We have built a prototype of the compromised user account detection mechanism, named SIJIL, using autoencoders. We have also developed a novel approach for content-agnostic detection of spam and phishing emails. Stakeholders across all sectors in Qatar can benefit from the outcomes of this project by: (1) having continuous assessment and monitoring of their critical cyber assets, (2) enhancing their preventive and detective cyber security posture with tailored and advanced agile technologies, and (3) gaining better visibility into their cyber space through sharing of data, intelligence, resources, and experience.Studying the MOI logs, we observed thousands of Tor and VPN connections, some of which were related to malware and botnets. This isn’t surprising as hundreds of malware variants have evolved to use VPNs and Tor to hide their presence and thwart active take down operations. C&C servers are increasingly operated from hidden servers to protect their IPs and locations and prevent network admins from observing coordinated botnet operations.
2. Tor malware problem: Simple blocking of Tor connections is not feasible, so we explore if it is possible to differentiate between benign and malicious Tor connections. We rely on traffic analysis, which is the process of examining traffic patterns (packet sizes, directions, timings, etc) to infer more (sensitive) information about traffic, thereby reducing the expected privacy provided by encryption or by proxy-based anonymization. Not only are we able to identify malicious encrypted Tor connections, we show that this can be done for “zero-day” malware variants, which have not been seen by our models. Our approach is summarized as follows.First, we build and update a repository consisting of hundreds of thoroughly verified fresh malicious Tor binaries. We inspect their traffic and ensure they are indeed malicious and use Tor for their operation. For creating a benign dataset, we build various binaries that simulate different profiles of traffic browsing (using the Tor browser) with different settings and loads. We deploy our (benign and malware) binaries independently on a sandbox environment and collect thousands of traffic instances over months. We also incorporate Tor traffic instances generated using various applications that were independently collected and published. Second, we characterize Tor-based malware binaries through the analysis of different families and network traffic exchange. We identify several characteristics that differentiate between malicious Tor communication and typical benign Tor browsing sessions. We craft and extract novel combinations of connection- and host-level traffic features from benign and malicious traffic. We then create (1) binary classifiers that can distinguish malware from benign connections, and (2) multi-label malware classifiers that can identify the malware class (ransomware, worm, trojan, etc.) based on their Tor communication.Finally, we craft several experiments to demonstrate the usefulness of our approach, feature categories, and models. In testing our models, we build datasets that reflect realistic scenarios where malware connections comprise a minimal percentage of all Tor connections. Our experiments show that we can identify the following: (1) malware connections, (2) malware class, and (3) zero-day malware. Furthermore, we experiment on real-world enterprise network logs which we obtain from our partners, and we are able to red flag suspicious connections despite missing some features due to the unavailability of raw PCAPs