TIBCO DataSynapse launches its Cloud Adapter for autoscaling in Azure

Since our launch of the TIBCO DataSynapse GridServer Engine image in the Azure Marketplace back in August, we have continued to pursue feature updates that improve the integration and experience of scheduling jobs and tasks into Azure. With the increased regulatory requirements of the Fundamental Review of the Trade Book (FRTB), having the ability to burst into Azure for the use of additional compute capacity is a hot topic within the Financial Services industry.

This week, TIBCO DataSynapse released the High Performance Adapter (HPCCA) to extend the functionality of GridServer 6.2.0 and enable customers to temporarily increase their compute capacity in the cloud.

The hybrid burst scenario takes a step forward with the ability to dynamically create and provision both Linux and Windows VMs directly into Azure. By implementing HPCCA as a Manager Hook into the Broker, it is able to review Broker events and take the right course of action through a simple cloud management algorithm.

Based on the number of events experienced, a formula predicts the number of VMs required and a customer created image is initiated. HPCCA can configure the current deployment and remote-start the Engine Daemons ready to process these events.

HPCCA also has the ability to shut down the Engine Daemons as soon as the events begin to decrease. By utilizing the same set of Azure APIs, the Engine idle time following event execution triggers this action.

If you would like to hear more about this and you based in the New York City area, please join us Wednesday, April 19th at the Microsoft office to view a demonstration.

Register now for the TIBCO and Microsoft Azure Workshop!

TIBCO DataSynapse GridServer is a service execution platform for dynamically scaling any application at any time across grid infrastructure. Due to its improvement in productivity, performance and uptime it is used heavily within Financial Services for parallel computing certain risk applications.
Quelle: Azure

How Azure Security Center helps reveal a Cyberattack

The Azure Security Center (ASC) analysts team reviews and investigates ASC alerts to gain insight into security incidents affecting Microsoft Azure customers, helping improve Azure Security alerts and detections. ASC helps customers keep pace with rapidly evolving threats by using advanced analytics and global threat intelligence.

Although we have come a long way as far as cloud security is concerned, even today security factors are heavily discussed as companies consider moving their assets to the cloud. The Azure Security Center team understands how critical it is for our customers to be assured that their Azure deployments are secure, not only from advanced attacks but even from the ones that are not necessarily new or novel. The beauty of ASC lies in its simplicity. Although ASC uses machine learning, anomaly detection, and behavioral analysis to determine suspicious events, it still addresses simple things like SQL brute force attacks that Bad Guys/Script Kiddies are using to break into Microsoft SQL servers.

In this blog, we’ll map out the stages of one real-world attack campaign that began with a SQL Brute Force attack, which was detected by the Security Center, and the steps taken to investigate and remediate the attack. This case study provides insights into the dynamics of the attack and recommendations on how to prevent similar attacks in your environment.

Initial ASC alert and details

Hackers are always trying to target internet connected databases. There are tons of bad guys trying to discover IP addresses that have SQL Server running so that they can crack their password through a brute force attack. The SQL database can contain a wealth of valuable information for the attackers, including personally identifiable information, credit card numbers, intellectual property, etc. Even if the database doesn’t have much information, a successful attack on an insecurely configured SQL installation can be leveraged to get full system admin privileges.

Our case started with an ASC Alert notification to the customer detailing malicious SQL activity. A command line “ftp -s:C:zyserver.txt” launched by the SQL service account was unusual and flagged as by ASC Alerts.

The alert provided details such as date and time of the detected activity, affected resources, subscription information, and included a link to a detailed report of the detected threat and recommended actions.

 

 

Through our monitoring, the ASC analysts team was also alerted to this activity and looked further into the details of the alert. What we discovered was the SQL service account (SQLSERVERAGENT) was creating FTP scripts (i.e.: C:zyserver.txt), which was used to download and launch malicious binaries from an FTP site.

The initial compromise

A deeper investigation into the affected Azure subscription began with inspection of the SQL error and trace logs where we found indications of SQL Brute Force attempts. In the SQL error logs, we encountered hundreds of “Audit Login Failed” logon attempts for the SQL Admin ‘sa’ account (built-in SQL Server Administration) which eventually led up to a successful login.

These brute force attempts occurred over TCP port 1433, which was exposed on a public facing interface. TCP port 1433 is the default port for SQL Server.

Note: It is a very common recommendation to change the SQL default port 1433, this may impart a “false sensation of security”, because many port scanning tools can scan a “range” of network ports and eventually find SQL listening on ports other than 1433.

Once the SQL Admin ‘sa’ account was compromised by brute force, the account was then used to enable the ‘xp_cmdshell’ extended stored procedure as we’ve highlighted below in a SQL log excerpt.

The ‘xp_cmdshell’ stored procedure is disabled by default and is of particular interest to attackers because of its ability to invoke a Windows command shell from within Microsoft SQL Server. With ‘xp_cmdshell enabled, the attacker created SQL Agent jobs which invoked ‘xp_cmdshell’ and launched arbitrary commands, including the creation and launch of FTP scripts which, in turn, downloaded and ran malware.

Details of malicious activity

Once we determined how the initial compromise occurred, our team began analyzing Process Creation events to determine other malicious activity. The Process Creation events revealed the execution of a variety of commands, including downloading and installing backdoors and arbitrary code, as well as permission changes made on the system.

Below we have detailed a chronological layout of process command lines that we determined to be malicious:

A day after the initial compromise we began to see the modification of ACLS on files/folders and registry keys with use of Cacls.exe (which appears to have been renamed to osk.exe and vds.exe).

Note: Osk.exe is the executable for the Accessibility On-Screen Keyboard and Vds.exe is the Virtual Disk Service executable, both typically found on a Windows installation. The command lines and command switches detailed below, however, are not used for Osk.exe or VDS.exe and are associated with Cacls.exe.

The Cacls.exe command switches /e /g is used to grant the System account full(:f) access rights to ‘cmd.exe’ and ‘net.exe’.

A few seconds later, we see the termination of known Antivirus Software using the Windows native “taskkill.exe”.

This was followed by the creation of and FTP script (c:zyserver.txt ) which was flagged in the original ASC Alert. This FTP script appears to download malware (c:stserver.exe) from a malicious FTP site and subsequently launch the malware.

A few minutes later, we see the “net user” and “net localgroup” commands used to accomplish the following:

a.    Activate the built-in guest account and add it to the Administrators group

b.   Create a new user account and add the newly created user to the Administrators group

A little over 2 hours later, we see the regini.exe command which appears to be used to create, modify, or delete registry keys. Regini can also set permissions on the registry keys as defined in the noted .ini file. We then see, regsvr32.exe silently (/s switch) registering dlls related to the Windows shell (urlmon.dll, shdocvw.dll) and Windows scripting (jscript.dll, vbscript.dll, wshom.ocx).

This is immediately followed by additional modification of permissions on various Windows executables. Essentially resetting each to default with the “icacls.exe” command.

Note: The /reset switch replaces ACLs with default inherited ACLs for all matching files.

Lastly, we observed the deletion of “Terminal Server” fDenyTSConnections registry key. This is a registry key that contains the configuration of Terminal Server connection restrictions. This led us to believe that malicious RDP connections may be the next step for the attacker to access the server. Inspection of logon events did not reveal to us any malicious RDP attempts or connections, however:

Disabling of Terminal Server connection restrictions by overwriting values in the “Terminal Server” registry key
reg.exe ADD "HKLMSYSTEMCurrentControlSetControlTerminal Server" /v fDenyTSConnections /t REG_DWORD /d 00000000 /f" 

We also noticed and Scheduled task created. This task referenced a binary named “svhost.exe” to be launched out of the C:RECYCLER folder, which is suspicious.

Note that the legitimate “svchost.exe” files located in the “WindowsSystem32” and “WindowssysWOW64”. Svchost.exe running from any other directory should be considered suspicious.

Persistence mechanism – Task Scheduler utility (schtasks.exe) used to set a recurring task
C:WindowsSystem32schtasks.exe /create /tn "45645" /tr "C:RECYCLERsvchost.exe" /sc minute /mo 1 /ru "system 

Recommended remediation and mitigation steps

Once we understood the extent and the details of the attack, we recommended the following remediation and mitigation steps to be taken.

First, if possible, we first recommended the backup and rebuild the SQL Server and reset all user accounts. We then implement the following mitigation steps to help prevent further attacks.

1. Disable ‘sa’ account and use the more secure Windows Authentication

To disable ‘sa’ login via SQL, run the following commands as a sys admin

ALTER LOGIN sa DISABLE

GO

2. To help prevent attackers from guessing the ‘sa’ account, rename the ‘sa’ account
To rename the ‘sa’ account via SQL, run the following as a sys admin:

ALTER LOGIN sa WITH NAME = [new_name];

GO

3. To prevent future brute force attempts, change and harden the ‘sa’ password and set the sa Login to ‘Disabled’.

Learn how to verify and change the system administrator password in MSDE or SQL Server 2005 Express Edition.

4. It’s also a good idea to ensure that ‘xp_cmdshell’ is disabled. Again, note that this should be disabled by default.

5. Block port TCP port 1433 if it is not needed be opened to the internet. From your Azure Portal, take the following steps to configure a Rule to block 1433 in Network Security Group

a. Open the Azure portal

b. Navigate to > (More Services) -> Network security groups

c. If you have opted into the Network Security option, you will see an entry for <ComputerName-nsg> — click it to view your Security Rules

d. Under Settings click "Inbound security rules" and then click +Add on the next pane

e. Enter the Rule name and Port information.Under the ‘Service’ pulldown, choose MS SQL and it will automatically select Port range = 1433 as detailed below.

f. Then apply the newly created rule to the subscription

6. Inspect all stored procedures that may have been enabled in SQL and look for stored procedures that may be implementing ‘xp_cmdshell’ and running unusual command.

For example, in our case, we identified the following commands:

7. Lastly, we highly recommend configuring Azure subscription(s) to receive future alerts and email notifications from Microsoft Azure Security Center. To receive alerts and email notifications of security issues like this in the future, we recommended upgrading from ASC “Free” (basic detection) tier to ASC “Standard” (advanced detection) tier.

Below is an example of the email alert received from ASC when this SQL incident was detected:

Learn more about SQL detection

Azure SQL Database Threat Detection–Advanced DB Security in the Cloud
Protect Azure SQL Databases with Azure Security Center
SQL Threat Detection – Your built-in security expert

Quelle: Azure

Up your SaaS application game

Frustrated with ever-changing customer expectations? Yesterday’s investments losing steam as the market turns to the next shiny thing? How do you scale across cloud and mobile, stay on top of data and security, and ultimately sell successfully in a rapidly transforming landscape?

Microsoft App Accelerate targets exactly these issues for application creators and ISVs (independent software vendors). Whatever type of solution your company delivers, the program provides the resources to help you plan, deploy, launch, and grow your applications.

Learn more by reading Build intelligent applications with help from Microsoft App Accelerate.
Quelle: Azure

One-click disaster recovery of applications using Azure Site Recovery

Disaster recovery is not only about replicating your virtual machines but also about end to end application recovery that is tested multiple times, error free, and stress free when disaster strikes, which are the Azure Site Recovery promises. If you have never seen your application run in Microsoft Azure, chances are that when a real disaster happens, the virtual machines may just boot, but your business may remain down. The importance and complexity involved in recovering applications was described in the previous blog of this series – Disaster recovery for applications, not just virtual machines using Azure Site Recovery. This blog covers how you can use the Azure Site Recovery construct of recovery plans to failover or migrate applications to Microsoft Azure in the most tested and deterministic way, using an example of recovering a real-world application to the public cloud.  

Why use Azure Site Recovery “recovery plans”?

Recovery plans help you plan for a systematic recovery process by creating small independent units that you can manage. These units will typically represent an application in your environment. Recovery plan not only allows you to define the sequence in which the virtual machines start, but also helps you automate common tasks during recovery.

Essentially, one way to check that you are prepared for disaster recovery is by ensuring that every application of yours is part of a recovery plan and each of the recovery plans is tested for recovery to Microsoft Azure. With this preparedness, you can confidently migrate or failover your complete datacenter to Microsoft Azure.
 
Let us look at the three key value propositions of a recovery plan:

Model an application to capture dependencies
Automate most recovery tasks to reduce RTO
Test failover to be ready for a disaster

Model an application to capture dependencies

A recovery plan is a group of virtual machines generally comprising an application that failover together. Using the recovery plan constructs, you can enhance this group to capture your application-specific properties.
 
Let us take the example of a typical three tier application with

one SQL backend
one middleware
one web frontend

The recovery plan can be customized to ensure that the virtual machines come up in the right order post a failover. The SQL backend should come up first, the middleware should come up next, and the web frontend should come up last. This order makes certain that the application is working by the time the last virtual machine comes up. For example, when the middleware comes up, it will try to connect to the SQL tier, and the recovery plan has ensured that the SQL tier is already running. Frontend servers coming up last also ensures that end users do not connect to the application URL by mistake until all the components are up are running and the application is ready to accept requests. To build these dependencies, you can customize the recovery plan to add groups. Then select a virtual machine and change its group to move it between groups.

 

Once you complete the customization, you can visualize the exact steps of the recovery. Here is the order of steps executed during the failover of a recovery plan:

First there is a shutdown step that attempts to turn off the virtual machines on-premises (except in test failover where the primary site needs to continue to be running)
Next it triggers failover of all the virtual machines of the recovery plan in parallel. The failover step prepares the virtual machines’ disks from replicated data.
Finally the startup groups execute in their order, starting the virtual machines in each group – Group 1 first, then Group 2, and finally Group 3. If there are more than one virtual machines in any group (for example, a load-balanced web frontend) all of them are booted up in parallel.

Sequencing across groups ensures that dependencies between various application tiers are honored and parallelism where appropriate improves the RTO of application recovery.

Automate most recovery tasks to reduce RTO

Recovering large applications can be a complex task. It is also difficult to remember the exact customization steps post failover. Sometimes, it is not you, but someone else who is unaware of the application intricacies, who needs to trigger the failover. Remembering too many manual steps in times of chaos is difficult and error prone. A recovery plan gives you a way to automate the required actions you need to take at every step, by using Microsoft Azure Automation runbooks. With runbooks, you can automate common recovery tasks like the examples given below. For those tasks that cannot be automated, recovery plans also provide you the ability to insert manual actions.

Tasks on the Azure virtual machine post failover – these are required typically so that you can connect to the virtual machine, for example:

Create a public IP on the virtual machine post failover
Assign an NSG to the failed over virtual machine’s NIC
Add a load balancer to an availability set

Tasks inside the virtual machine post failover – these reconfigure the application so that it continues to work correctly in the new environment, for example:

Modify the database connection string inside the virtual machine
Change web server configuration/rules

For many common tasks, you can use a single runbook and pass parameters to it for each recovery plan so that one runbook can serve all your applications. To deploy these scripts yourself and try them out, click the button below and import popular scripts into your Microsoft Azure Automation account.

 
With a complete recovery plan that automates the post recovery tasks using automation runbooks, you can achieve one-click failover and optimize the RTO. 

Test failover to be ready for a disaster

A recovery plan can be used to trigger both a failover or a test failover. You should always complete a test failover on the application before doing a failover. Test failover helps you to check whether the application will come up on the recovery site.  If you have missed something, you can easily trigger cleanup and redo the test failover. Do the test failover multiple times until you know with certainty that the application recovers smoothly.

 

Each application is different and you need to build recovery plans that are customized for each. Also, in this dynamic datacenter world, the applications and their dependencies keep changing. Test failover your applications once a quarter to check that the recovery plan is current.

Real-world example – WordPress disaster recovery solution

Watch a quick video of a two-tier WordPress application failover to Microsoft Azure and see the recovery plan with automation scripts, and its test failover in action using Azure Site Recovery.

The WordPress deployment consists of one MySQL virtual machine and one frontend virtual machine with Apache web server, listening on port 80.
WordPress deployed on the Apache web server is configured to communicate with MySQL via the IP address 10.150.1.40.
Upon test failover, the WordPress configuration needs to be changed to communicate with MySQL on the failover IP address 10.1.6.4. To ensure that MySQL acquires the same IP address every time on failover, we will configure the virtual machine properties to have a preferred IP address set to 10.1.6.4.

With relentless focus on ensuring that you succeed with full application recovery, Azure Site Recovery is the one-stop shop for all your disaster recovery needs. Our mission is to democratize disaster recovery with the power of Microsoft Azure, to enable not just the elite tier-1 applications to have a business continuity plan, but offer a compelling solution that empowers you to set up a working end to end disaster recovery plan for 100% of your organization&;s IT applications.

You can check out additional product information and start replicating your workloads to Microsoft Azure using Azure Site Recovery today. You can use the powerful replication capabilities of Azure Site Recovery for 31 days at no charge for every new physical server or virtual machine that you replicate, whether it is running on VMware or Hyper-V. To learn more about Azure Site Recovery, check out our How-To Videos. Visit the Azure Site Recovery forum on MSDN for additional information and to engage with other customers, or use the Azure Site Recovery User Voice to let us know what features you want us to enable next.
Quelle: Azure

Adding offerings and UK Region: Azure rolls deep with PCI DSS v3.2

Check out our AoC

Go here to download Azure’s Payment Card Industry Data Security Standard (PCI DSS) v3.2  Attestation of Compliance (AoC)! When it comes to enabling customers who want or need to operate in a cloud environment AND also need to adhere to the global standards designed to prevent credit card fraud, they need look no further than Azure. 

Why put off until 2018 what you can do today?

When it comes to security and compliance, we are always ready to act. The DSS v3.2 contains several requirements that don’t take effect until January 2018, and while it is possible to get a v3.2 certification without meeting these future requirements, Azure has already adopted them and is currently compliant with all new requirements!

UK, too!

Azure has also added the UK region to our list of PCI-certified datacenters, while expanding coverage within previously certified regions around the world. A new version of our PCI Responsibility Matrix will be released shortly, keep an eye out for that announcement coming soon.

More services = More options for customers

Azure has again increased the coverage of our attestation to keep up with customer needs and we continue to be unmatched amongst Cloud Service Providers in the depth and breadth of offerings with PCI DSS v3.2. A sample of the services added in this attestation include:

Note: Refer to the latest AoC for the full list of services and regions covered. 

DocumentDB
Data Catalog
Machine Learning
Functions
Virtual Machine Scale Sets

AoC FAQs and Highlights

Why does the AoC say “April 2016”?

The front page and footer of the AoC says “April 2016”.  This is the date that the template was published by the PCI SSC, it is not the date of our AoC.  Many customers get confused by this, but we are not able to modify the AoC template. Refer to page 76 of the AoC for the date it was actually signed and issued. 

How should I interpret the service listing in the AoC?

We have received feedback in the past that it was difficult to understand what services were covered in the AoC. This was mainly because the services were listed under the groupings and internal names our Qualified Security Assessor (QSA) used for the assessment, along with the fact that many services got re-branded shortly after our 2015 AoC was released.

We incorporated that feedback in the release of our 2016 AoC, and have again updated the service listing in the 2017 AoC to reflect the current set of Azure offerings. Please be aware that if an Azure service is re-branded we are not able to retroactively update the AoC.  If you have questions about the status of an Azure service, please contact Azure support or your TAM. 

Why isn’t Azure assessed as a “Shared Hosting Provider”?

The shared hosting provider designation in PCI DSS is for situations where multiple customers are being hosted on a single server, but doesn’t take into account hosting of isolated virtualized environments.  An example of shared hosting is if a service provider was hosting multiple customer websites on a single physical web server. In that situation, there is no segregation between the customer environments. Azure is not considered a shared hosting provider for PCI because customer VMs and environments are segregated and isolated from each other. So changes made to “Customer X’s” VM does not affect “Customer Y’s” VM, even under the scenario that both VMs are hosted on the same physical host.  
Quelle: Azure

Azure Data Factory’s Data Movement is now available in the UK

Data Movement is a feature of Azure Data Factory that enables cloud-based data integration, which orchestrates and automates the movement and transformation of data. You can now create data integration solutions using Azure Data Factory that can ingest data from various data stores, transform/process data, and publish results to the data stores. 

Moreover, you can now utilize Azure Data Factory for both your cloud and hybrid data movement needs with the UK data store. For instance, when copying data from a cloud data source to an Azure store located in the UK, Data Movement service in UK South will perform the copy and ensure compliance with data residency.

Note: Azure Data Factory itself does not store any data, but instead lets you create data-driven flows to orchestrate movement of data between supported data stores and the processing of data using compute services in other regions or in an on-premises environment.

To learn more about using Azure Data Factory for data movement, view the Move data by using Copy Activity article. 

You can also go to Azure.com learn more about Azure Data Factory or view more in depth Azure Data Factory information documentation.
Quelle: Azure

Azure Relay Hybrid Connections is generally available

The Azure Relay service was one of the first core Azure services. Today’s announcement shows that it has grown up nicely with the times. For those familiar with the WCF Relay feature of Azure Relay rest assured it will continue to function, but its dependency on Windows Communication Foundation is not for everyone. The Hybrid Connections feature of Azure Relay sheds this dependency by utilizing open standards based protocols.

Hybrid Connections contains a lot of the same functionality as WCF Relay including:

Secure connectivity of on-premises assets and the cloud
Firewall friendliness as it utilizes common outbound ports
Network management friendliness that won&;t require a major reconfiguration of your network

The differences between the two are even better!

Open standards based protocol and not proprietary! WebSockets vs. WCF
Hybrid Connections is cross platform, using Windows, Linux or any platform that supports WebSockets
Hybrid Connections supports .NET Core, JavaScript/Node.js, and multiple RPC programming models to achieve your objectives

Getting started with Azure Relay Hybrid Connections is simple and easy with steps here for .NET and Node.js.

If you want to try it and we hope you do, you can find out more about Hybrid Connections pricing and the Azure Relay offering.
Quelle: Azure

Large-Scale Analysis of DNS Query Logs Reveals Botnets in the Cloud

This post was co-authored by Tomer Teller, Senior Security Program Manager, Azure Security.

The arms race between data security professionals and cybercriminals continues at a rapid pace. More than ever, attackers exploit compute resources for malicious purposes by deploying malware, known as “bots”, in virtual machines running in the cloud. Even a conservative estimate reveals that, at least, 1 in every 10,000 machines are part of some known Botnet.

To better protect VMs in the cloud, Azure Security Center (ASC) applies a novel supervised Machine Learning model for high-precision Botnet detection based on analysis of DNS query logs. This model achieves 95% precision and 43% recall and can detect Botnets before they are reported by antimalware companies.

Communication patterns between Botnets and their CnC server

Bots are controlled by the attacker (Botmaster) using a Command and Control (CnC) server or servers. The Bots which are part of this network are called Botnets (or Zombies). A typical Bot network (Botnet) structure is illustrated in the following figure.

Historically, a CnC server was assigned a static IP address making it very easy to take down or blacklist. To avoid detection, Botmasters responded by creating more complex bot/CnC communication patterns.

For example, attackers developed methods, such as Fast-Flux (Mehta, 2014), that use domain names to locate the CnC server which frequently changes its IP address. In addition, domain generation algorithms (DGA) are also used by various families of malware, with Conficker being perhaps the most notorious example. The DGA pattern works by periodically generating a large number of domain names that can be used by bots as connection points to the CnC servers.

More recently, social networks and other user-generated content sites are being exploited by Botmasters to pass information without ever establishing a direct link with a CnC server. Security professionals can therefore no longer rely on simple rule-based approaches to detect these complex communication patterns.

Opportunity to detect Botnets in the cloud

One of the more common applications of machine learning in the cybersecurity domain is anomaly detection. The idea is that a compromised machine exhibits anomalous behavior. While this assumption is usually correct, the opposite seldom holds. Therefore, such techniques achieve low precision and thus produce many false alarms.

Cloud providers such as Microsoft possess a unique opportunity to detect Botnet activity with much greater accuracy by applying large scale machine learning over multiple data sources as a well as a combined view of all the VM logs. Unlike most other systems which analyze data from each machine in isolation, our approach can effectively uncover patterns that are typical to Botnets.

Gathering the data

We collect DNS query and response data from Azure VMs. The logs contain around 50TB of data per day and includes information such as the query name, queried domain name server, the DNS response, and other DNS logging information.

In addition to DNS query and response data, we also use a Microsoft automated machine-readable feed of threat intelligence (TI). The feed includes information about IP addresses of devices which are likely to be part of a Botnet as well as the IP addresses and domains of known CnC servers.

To achieve optimal results, we model Botnet detection as a 2-class supervised learning problem. That is, to classify that a VM (on a specific date) is part of a Botnet based on that VM’s DNS query log. VM instances are labeled as possibly participating in a Botnet based on the following criteria:

The IP address of the VM appears in the TI Botnet feed of that same day.
The VM issued a DNS query with a domain known to belong to a CnC.
The VM received a DNS response to an issued query and the resulting mapped IP is a known IP address of a CnC.

Feature extraction

The VM instances represent a VM on a specific day and are labeled as possible participants of a botnet based on the TI feed of that day. In our problem, feature extraction is difficult; the number of domains accessed by a given VM can be very large and the total number of possible domains is massive. Hence, the domain space is huge and relatively dense.

Moreover, since the model is used continuously, it needs to identify Botnets even when they query for domains is unseen during training. Based on communication patterns with CnC servers our features should capture the insights laid out in the following table.

Name
Explanation

Rare domains
Domain names of CnC servers are rare since they are seldom requested by legitimate users

Young domains
When a domain generation algorithm (DGA) is used the CnC server frequently acquires new domain names hence they tend to be recently registered. We use a massive daily updated data feed to map domain names to their registration date

Domains Idiosyncratic to Botnets
Botnets controlled by the same CnC server issue DNS queries which contain similarities to each other yet are different from others

Non-existent domain responses

When DGA is used, Botnets query many non-existing domains before they find the actual domain of their CnC server for that time

To efficiently generate the features for each instance we apply two passes over the dataset. In the first pass, we generate a Reputation Table (RT) which maps domains at a given day to:

Rareness scores
Youngness scores
Botnet idiosyncratic scores

In the second pass we calculate the features for each instance based on the reputation scores of domains it queried for. The RT is calculated as follows:

To generate features for a VM at a given day, we create a set Dnx and Dx which are sets of all the non-existent and existent domains queried by the VM, respectively. We produce the feature vectors by summing up the corresponding values in the reputation table for each domain in Dnx and Dx separately.

Similarly, we do the same for the name-servers being queried (i.e., the DNS server that eventually resolved the DNS query). The latter features help identify legitimate scenarios in which rare subdomains of a non-malicious zone are accessed, e.g., rare subdomains of Yahoo.com. For classification, we used Apache Spark’s Gradient-Boosted Trees with default parameters.

Experimental evaluation

We use the Microsoft TI feed to generate the labels for our daily Azure VM instances. However, these labels are not perfect; Botnets can still remain undetected for quite some time. Hence, the labels we produced based on the feed are not comprehensive. This makes our evaluation setting different from that of a standard classification problem, since our goal is not to perfectly match labels that can be extracted based on the feed. This would simply duplicate information which is already available in the daily feed. Instead, our goal is to find compromised VMs before they appear in the TI feed.

With this in mind, we trained our model on a week of data from early June 2016. We let the model classify instances from late June and produce our “ground truth” for evaluation based on labels generated from the TI feed looking forward in time one week (into July). We report the accuracy of our model in the following confusion matrix.

From the matrix, we learn that the model classified 432 (411+21) instances from the test set as being Botnets. Out of these, 95% (411) eventually appear in the Interflow feed (within a week), hence the model achieves 95% precision and 43% recall. Note that the 5% apparent FPs may still potentially be Botnets that have not yet appeared in the feed, hence they require further investigation.

Conclusions

We present a novel supervised ML model for Botnet detection based on DNS logs. We generate the labels for the supervised model based on a threat intelligence feed provided by anti-malware vendors. We show that the model is able to identify with high accuracy the VMs that are part of a botnet well before they become part of the TI feed. This new Botnet detection feature will reduce the risk of Azure VMs becoming infected with malware.
Quelle: Azure

Announcing Support for Multi-member Consortium Blockchain Networks on Azure

As your blockchain application development efforts and pilots mature, we realize that the requirements for the underlying consortium network will change, and that you will need to easily and securely create and deploy across multiple regions and support members comprised from organizations that exist within separate administrative and trust boundaries. We are excited to announce expansion of our blockchain support on Azure to be the first public cloud that enables multi-member consortium blockchain networks addressing enterprise scenarios that require a deployment of a private network across Azure regions, subscriptions, and Azure Active Directory (Azure AD) tenants.

As we work with customers, we see scenarios divide into three common topologies:

1.       Single organization, multiple subscriptions: This is a common topology when divisions in an organization do not trust each other, for example when one division is auditing another division. Each division has its own footprint, but they are physically separated in different subscriptions across the same Azure AD tenant.

2.       Multiple organizations, private: This is the true consortium scenario where each organization will have its own footprint and subscriptions, Azure AD tenants, and regions are all different. Given enterprise IT requirements, the services deployed must not be publicly accessible on the internet, even though communication will occur across organizations.

3.       Multiple organizations, public-facing: Similar to the above topology, but in industries, enterprises, or scenarios where IT requirements allow or require the services deployed to be accessible to the public, over the internet. This simplifies the network connectivity requirements for the distributed system.

Today, we are releasing a set of solution templates in the Azure Marketplace that address the first two topologies configuring multi-region and multi-member Ethereum Consortium Blockchain Networks with a simple multi-step process through the Azure Portal or cmdline. The first template deploys and configures the footprint for the initial consortium member (or region), while the second template deploys, connects, and configures additional members (or regions) to form the overarching private network.  If you are still experimenting, we suggest using the simpler single subscription deployment solution released in November.

These solution templates are designed to make it easier and quicker to deploy and configure a multi-member consortium Ethereum network with minimal Azure and Ethereum knowledge. With a handful of user inputs and a single-click deployment, each member can provision their network footprint, using Microsoft Azure compute, networking, and storage services across the globe. Each member&;s network footprint consists of a set of load-balanced transaction nodes with which an application or user can interact to submit transactions, a set of mining nodes to record transactions, and a VPN gateway. A subsequent connection step connects the gateways to create a fully configured multi-member blockchain network.

A multi-member network architecture is illustrated below.

It is important to note that to whom a member connects is not dictated by the template. The consortium should determine the network connectivity model, whether hub and spoke or mesh.

For more information about the solution, you can visit our guided walkthrough.

As with our first blockchain solution, rather than spending hours building out and configuring the infrastructure and networking across organizations, we have automated these time-consuming pieces to allow you to focus on building out the consortium and your production pilots.

Let us know if you have any questions, feedback, or additional requests once you try out these new blockchain solutions. We are excited to help you expand your blockchain deployments to true multi-party topologies.
Quelle: Azure

Introducing Backup Pre-Checks for Backup of Azure VMs

Over the past couple of weeks, we have announced multiple enhancements for backup and recovery of both Windows and Linux Azure Virtual Machines that reinforce Azure Backup’s cloud-first approach of backing up critical enterprise data in Azure. Enterprise production environments in Azure are becoming increasingly dynamic and are characterized by frequent VM configuration changes (such as network or platform related updates) that can adversely impact backup. Today, we are taking a step to enable customers to monitor the impact of configuration changes and take steps to ensure the continuity of successful backup operations. We are excited to announce the preview of Backup Pre-Checks for Azure Virtual Machines.

Backup Pre-Checks, as the name suggests, check your VMs’ configuration for issues that can adversely affect backups, aggregate this information so you can view it directly from the Recovery Services Vault dashboard and provide recommendations for corrective measures to ensure successful file-consistent or application-consistent backups, wherever applicable. All this without any infrastructure and at no additional cost.

Backup Pre-Checks run as part of the scheduled backup operations for your Azure VMs and complete with one of the following states:

Passed: This state indicates that your VMs configuration is conducive for successful backups and no corrective action needs to be taken.
Warning: This state indicates one or more issues in VM’s configuration that might lead to backup failures and provides recommended steps to ensure successful backups. Not having the latest VM Agent installed, for example, can cause backups to fail intermittently and falls in this class of issues.
Critical: This state indicates one or more critical issues in the VM’s configuration that will lead to backup failures and provides required steps to ensure successful backups. A network issue caused due to an update to the NSG rules of a VM, for example, will fail backups as it prevents the VM from communicating with the Azure Backup service and falls in this class of issues.

Value proposition

Identify and Monitor VM configuration issues at scale: With the aggregated view of the Backup Pre-Check status across all VMs on the Recovery Services Vault, you can keep track of how many VMs need corrective configuration changes to ensure successful backups
Resolve configuration issues more efficiently: Use the Backup pre-check states to rank which VMs need configuration changes. Address the “Critical” Backup Pre-Check status for your VMs first, using the specific required steps and ensure their successful backups before addressing the “Warning” Backup Pre-check states for your VMs.
Automated execution: You don’t need to maintain or apply separate schedules for Backup Pre-Checks as they are integrated with existing backup schedules and therefore are assured to execute automatically and get the latest VM configuration information at the same cadence as their backups.

Getting started

Follow the steps below to start resolving any issues reported by Backup Pre-Checks for Virtual Machine backups on your Recovery Services Vault.

Click on the ‘Backup Pre-Check Status (Azure VMs)’ tile on the Recovery Services Vault dashboard.
Click on any VM with Backup Pre-Check status of either Critical or Warning. This would open the VM details blade.
Click on the blade notification on the top of the blade to reveal the configuration issue description and remedial steps.

Related links and additional content

Learn more about preparing your VMs for successful backups 
Need help? Reach out to Azure Backup forum for support
Tell us how we can improve Azure Backup by contributing new ideas and voting up existing ones
Follow us on Twitter @AzureBackup for the latest news and updates
New to Azure Backup, sign up for a free Azure trial subscription

Quelle: Azure