Cloud Computing Köln - Seite 1839 von 7144 - Neues zu Cloud Computing, Internet of Things und Technologien

Over one third of organizations believe that data analytics and machine learning have the most potential to significantly alter the way they run business over the next 3 to 5 years. However, only 26% of organizations are data driven. One of the biggest reasons for this gap is that a major portion of the data generated today is unstructured, which includes images, documents, and videos. It is estimated to cover roughly up to 80% of all data, which has so far remained untapped by organizations.One of the goals of Google’s data cloud is to help customers realize value from data of all types and formats. Earlier this year, we announced BigLake, which unifies data lakes and warehouses under a single management framework, enabling you to analyze, search, secure, govern and share unstructured data using BigQuery. At Next ‘22, we announced the preview of object tables, a new table type in BigQuery that provides a structured record interface for unstructured data stored in Google Cloud Storage. This enables you to directly run analytics and machine learning on images, audio, documents and other file types using existing frameworks like SQL and remote functions natively in BigQuery itself. Object tables also extend our best practices of securing, sharing and governing structured data to unstructured, without needing to learn or deploy new tools.Directly process unstructured data using BigQuery MLObject tables contain metadata such as URI (Uniform Resource Identifier), content type, and size that can be queried just like other BigQuery tables. You can then derive inferences using machine learning models on unstructured data with BigQuery ML. As part of preview, you can import open source TensorFlow Hub image models, or your own custom models to annotate the images. Very soon, we plan to enable this for audio, video, text and many other formats, and pre-trained models to enable out-of-the box analysis. Check out this video to learn more and watch a demo.code_block[StructValue([(u’code’, u’# Create an object tablernCREATE EXTERNAL TABLE my_dataset.object_tablernWITH CONNECTION us.my_connection rnOPTIONS(uris=[“gs://mybucket/images/*.jpg”],rn object_metadata=”SIMPLE”, metadata_cache_mode=”AUTOMATIC”);rnrn # Generate inferences with BQMLrnSELECT * FROM ML.PREDICT(rn MODEL my_dataset.vision_model, rn (SELECT ML.DECODE_IMAGE(data) AS img FROM my_dataset.object_table)rn);’), (u’language’, u”), (u’caption’, <wagtail.wagtailcore.rich_text.RichText object at 0x3e08491c1750>)])]By analyzing unstructured data natively in BigQuery, businesses canEliminate manual effort as pre-processing steps such as tuning image sizes to model requirements are automatedLeverage the simple and familiar SQL interface to quickly gain insightsSave costs by utilizing existing BigQuery slots without needing to provision new forms of computeAdswerve is a leading Google Marketing, Analytics and Cloud partner on a mission to humanize data. Twiddy & Co. is Adswerve’s client – a vacation rental company in North Carolina. By combining structured and unstructured data, Twiddy and Adswerve used BigQuery ML to analyze images of rental listings and predict the click-through rate, enabling data-driven photo editorial decisions. “Twiddy now has the capability to use advanced image analysis to stay competitive in an ever changing landscape of vacation rental providers – and can do this using their in-house SQL skills.” said Pat Grady, Technology Evangelist, AdswerveProcess unstructured data using remote functionsCustomers today use remote functions (UDFs) to process structured data for languages and libraries that are not supported in BigQuery. We are extending this capability to process unstructured data using object tables. Object tables provide signed URLs to allow remote UDFs running on Cloud Functions or Cloud Run to process the object table content. This is particularly useful for running Google’s pre-trained AI models, including Vision AI, Speech-to-Text, Document AI, open source libraries such as Apache Tika, or deploying your own custom models where performance SLAs are important. Here’s an example of an object table being created over PDF files that are parsed using an open source library running as a remote UDF.code_block[StructValue([(u’code’, u’SELECT uri, extract_title(samples.parse_tika(signed_url)) AS titlernFROM EXTERNAL_OBJECT_TRANSFORM(TABLE pdf_files_object_table, rn [“SIGNED_URL”]);’), (u’language’, u”), (u’caption’, <wagtail.wagtailcore.rich_text.RichText object at 0x3e083ebb4310>)])]Extending more BigQuery capabilities to unstructured dataBusiness intelligence – The results of analyzing unstructured data either directly in BigQuery ML or via UDFs can be combined with your structured data to build unified reports using Looker Studio (at no charge), Looker or any of your preferred BI solutions. This allows you to gain more comprehensive business insights. For example, online retailers can analyze product return rates by correlating them with the images of defective products. Similarly, digital advertisers can correlate ad performance with various attributes of ad creatives to make more informed decisions.BigQuery search index – Customers are increasingly using the search functionality of BigQuery to power search use cases. These capabilities now extend to unstructured data analytics as well. Whether you use BigQueryML to produce inference on images or use remote UDFs with Doc AI to produce document extraction, the results can now be search indexed and used to support search access patterns. Here’s an example of search index on data that is parsed from PDF files:code_block[StructValue([(u’code’, u’CREATE SEARCH INDEX my_index ON pdf_text_extract(ALL COLUMNS);rnrnSELECT * FROM pdf_text_extract WHERE SEARCH(pdf_text, “Google”);’), (u’language’, u”), (u’caption’, <wagtail.wagtailcore.rich_text.RichText object at 0x3e084957e110>)])]Security and governance – We are extending BigQuery’s row-level security capabilities to help you secure objects in Google Cloud Storage. By securing specific rows in an object table, you can restrict the ability of end users to retrieve the signed URLs of corresponding URIs present in the table. This is a shared responsibility security model, for which administrators need to ensure that end users don’t have direct access to Google Cloud Storage, and use signed URLs from object tables as the only access mechanism.Here’s an example of a policy for PII images that are secured to be first processed through a blur pipeline:code_block[StructValue([(u’code’, u’CREATE ROW ACCESS POLICY pii_data ON object_table_imagesrnGRANT TO (“group:admin@example.com”) rnFILTER USING (ARRAY_LENGTH(metadata)=1 AND rn metadata[OFFSET(0)].name=”face_detected”)’), (u’language’, u”), (u’caption’, <wagtail.wagtailcore.rich_text.RichText object at 0x3e083dd17e90>)])]Soon, Dataplex will support object tables, allowing you to automatically create object tables in BigQuery and manage and govern unstructured data at scale.Data sharing – You can now use Analytics Hub to share unstructured data with partners, customers and suppliers while not compromising on security and governance. Subscribers can consume the rows of object tables that are shared with them, and use signed URLs for unstructured data objects. Getting StartedSubmit this form to try these new capabilities that unlock the power of your unstructured data in BigQuery. Watch this demo to learn more about these new capabilities.Special thanks to engineering leaders Amir Hormati, Justin Levandoski and Yuri Volobuev for contributing to this post.Related ArticleBuilt with BigQuery: BigQuery ML enables Faraday to make predictions for any US consumer brandHow Building with BigQuery ML enables Faraday to make predictions for any US consumer brand.Read Article
Quelle: Google Cloud Platform

19. Oktober 2022

da Agency

Announcing Azure DNS Private Resolver general availability

A successful hybrid networking strategy demands DNS services that work seamlessly across on-premises and cloud networks. Azure DNS Private Resolver now provides a fully managed recursive resolution and conditional forwarding service for Azure virtual networks. Using this service, you will be able to resolve DNS names hosted in Azure DNS private zones from on-premises networks as well as DNS queries originating from Azure virtual networks that can be forwarded to a specified destination server to resolve them.

This service will provide a highly available and resilient DNS infrastructure on Azure for a fraction of the price of running traditional IaaS VMs running DNS servers in virtual networks. You will be able to seamlessly integrate with Private DNS Zones and unlock key scenarios with minimal operational overhead.

We are excited to share that Azure DNS Private Resolver is now in general availability.

A quick overview of Azure DNS

We offer two types of Azure DNS Zones—private and public—for hosting your private DNS and public DNS records. In the preceding illustration, multi-region workloads running on Azure with Azure DNS Private Resolver are provisioned in two regional, centralized virtual networks with one or more spokes peered to each centralized virtual network. These virtual networks have inbound and outbound endpoints provisioned. From on-premises, there are two distinct locations (East and West) and each location connects via Express Route to the centralized virtual network where Private Resolver is provisioned. These on-premises locations have one or more local DNS servers configured to do conditional forwarding to the inbound endpoint of Private Resolver. The local DNS servers in East have the IP address of the East inbound endpoint as the primary DNS target, and the West inbound endpoint as secondary. Alternatively, the local DNS servers in West have the IP address of the West inbound endpoint as the primary DNS target, and the East inbound endpoint as secondary. There is a single private DNS zone linked to both regions and both on-premises locations can resolve names from this zone even in the event of a regional failure.

Azure Private DNS: Azure Private DNS provides a reliable and secure DNS service for your virtual network. Azure Private DNS manages and resolves domain names in the virtual network without the need to configure a custom DNS solution. By using private DNS zones, you can use your own custom domain name instead of the Azure-provided names during deployment.
Azure Public DNS: DNS domains in Azure DNS are hosted on Azure's global network of DNS name servers. Azure DNS uses anycast networking. Each DNS query is answered by the closest available DNS server to provide fast performance and high availability for your domain.

What is being announced today?

Azure DNS Private Resolver enables you to query Azure DNS private zones from an on-premises environment and vice versa without deploying virtual machine-based DNS servers.

Azure DNS Private Resolver general availability is being announced to all customers and will have regional availability in the following regions:

East US
East US 2
Central US
South Central US
North Central US
West Central US
West US 3
Canada Central
Brazil South

West Europe
North Europe
UK South
France Central
Sweden Central
Switzerland North

East Asia
Southeast Asia
Japan East
Korea Central
South Africa North
Australia East

What will customers be able to do with Azure Private Resolver?

Apart from the features which were announced earlier in preview, customers will now be able to leverage the following additional functionality and content:

Additional architectural guidance for higher resiliency and enabling disaster recovery scenarios.
In-depth information on how to configure conditional forwarding rules.
Configuring hybrid name resolution from on-premises.

In the following diagram, an on-premises network connects to Azure via ExpressRoute and has on-premises DNS servers configured to conditionally forward queries to the private IP address of the inbound endpoint. The inbound endpoint then resolves names available on Azure Private DNS zones which are linked to the virtual network where private resolver is provisioned. If there is no matching private DNS zone in the virtual network, it will use the outbound endpoint and resolve using the ruleset rules via longest suffix match. If no match in the ruleset is found it will recurse to the internet for public name resolution.

Features and benefits

Cross-subscription support to link virtual networks from different subscriptions to rulesets.
Resource Health Check Integration to provide visibility of endpoint health to our customers.

Visibility of query metrics per endpoint to plan for future capacity:

PrivateLink enabled services integration in conditional forwarding to exclude Azure infra zones from being resolved on-premises.

Private Resolver general availability is also available to use via PowerShell, CLI, .NET, Java, Python, REST, Typescript, Go, ARM, and Terraform.

Key use cases for this service

Conditionally forward from on-premises with Azure ExpressRoute/VPN and resolve names hosted on Azure Private DNS Zones via private IP address.
Seamlessly resolve Private Endpoints which are registered in Azure Private DNS Zones.
Configure default DNS servers and forward all DNS queries to either a Protective DNS service or other target DNS servers with a wildcard rule.
Conditionally forward to any reachable target DNS server using a simple rule.
Access resources on-premises with Azure Bastion using names hosted on DNS servers on-premises or Azure Private DNS zones.

Fully managed

Built-in high availability, zone redundancy, and low latency name resolution.

Reduces cost

Reduce operating costs and run at a fraction of the price of traditional IaaS solutions.

Private access to your Private DNS Zones

Conditionally forward from your Virtual Networks to any reachable DNS server and from on-premises to Azure Private DNS Zones.

Scalability

High performance per endpoint.

Highly available

Availability Zone aware and resilient to failures within a region. Service-legal agreement (SLA) of 99.99 percent during general availability.

DevOps-friendly

Build your pipelines with Terraform, ARM, or Bicep.

Get started and share your feedback

You can try Azure DNS Private Resolver today. For more information about the capabilities available, please visit the Azure DNS Private Resolver technical documentation webpage. Post your ideas and suggestions on the networking community page.
Quelle: Azure

19. Oktober 2022

da Agency

Delivering consistency and transparency for cloud hardware security

This post was co-authored by Mark Russinovich, CTO and Technical Fellow, Azure, and Bryan Kelly, Partner Architect, Azure Hardware Systems and Infrastructure.

When it comes to building the Microsoft Cloud, our work to standardize designs for systems, boards, racks, and other parts of our datacenter infrastructure is paramount to facilitating forward progress and innovation across the computing industry. Microsoft has made a number of contributions to and collaborated with various members of the Open Compute Project (OCP) community, the leading industry group dedicated to open source hardware innovation. This year, we are excited to showcase some of our newest projects at the OCP Global Summit and share our learnings on the path of building a more reliable, trusted, and sustainable cloud. One of the key areas where we’ve seen continued focus and opportunity is driving industrywide standards around platform security. To dive deeper into our contributions in this area, I’ve invited Mark Russinovich, CTO and Technical Fellow, Azure, and Bryan Kelly, Partner Architect, Azure Hardware Systems and Infrastructure, to share more about Microsoft’s newest security contributions to OCP that standardize the foundations of trust, integrity, and reliability in computing.

Securing customer workloads from the cloud to the edge

Microsoft Azure is a leader in cloud security and privacy offering a broad range of confidential computing services to help organizations run workloads that keep business and customer data private with advanced levels of security. As the demand for confidential computing grows from cloud to edge, so do the requirements for consistency and transparency of the security mechanisms that protect workloads. With the rise of edge computing, the resultant growth in the exposed attack surface also presents a need for stronger physical security solutions. In this context, there is an increased need for greater transparency in the infrastructure that underpins these technologies and upholds hardware security promises.

Caliptra: Integrating trust into every chip

At the Open Compute Project (OCP) Summit, we are jointly announcing Caliptra, an open source root of trust (RoT) that produces cryptographic proofs about the hardware protections in place for confidential workloads. Designed with security experts and industry leaders in confidential computing across AMD, Google, Microsoft, and NVIDIA, Caliptra is a forward-looking approach casting transparency into hardware security. As a reusable open source, silicon-level block for integration into systems on a chip (SoCs)—such as CPUs, GPUs, and accelerators—Caliptra provides trustworthy and easily verifiable attestation.

At its core, Caliptra provides foundational security properties that underpin the integrity of higher-level security protection for confidential workloads. The Caliptra RoT has the following essential security properties:

Identity: A unique device manufacturer’s cryptographic identity for attestation endorsement. The identity is consistent with TCG DICE and includes intrinsic attestation of the Caliptra firmware.

Compartmentalization: Hardware protection barriers that isolate Caliptra’s security assets.

Measurement: Cryptographic digests that represent the SoC security configuration in a concise, cryptographically verifiable manner.

Renewable security: The security state of the hardware can be brought forward to a secure state and the life cycle management extends through manufacturing, field operations, circular economy, and eventually end of life.

Ownership: Provides verification of both authenticity and integrity of all mutable firmware loaded into an SoC. This property uses hardware-enforced digital signatures to ensure integrity policies are upheld.

Attestation: The endorsement of measurement claims with Caliptra’s cryptographic identity. These measurements provide precise information about the security state of the trusted computing base.

The initial Caliptra 0.5 contribution release to OCP contains a series of specifications describing architecture, integration, and implementation. An open sourced register-transfer level (RTL) code implementation of Caliptra that can be synthesized into current SoC designs will be made available, along with the cloud-designed firmware written entirely in Rust. With this trusted foundation designed for confidential cloud devices, Caliptra supports the consistent scaling of confidential workloads across distributed systems.

With deep ecosystem collaboration at the heart of Microsoft’s open source philosophy, we look forward to continuing working closely with our partners and engaging the industry to advance Caliptra. Caliptra RTL and firmware project collaboration will be done under the auspices of the CHIPS Alliance.

Hydra: A new secure Baseboard Management Controller (BMC)

We are also introducing Hydra, a new secure BMC in partnership with Nuvoton. A BMC is typically designed into every server system and expansion chassis—for example, JBOD or GPU. As a diagnostic and recovery controller, the BMC has special privileged hardware interfaces for acquiring debug data and telemetry from CPUs. These interfaces present security concerns, as they are targets for attacks that bypass conventional security defenses.

Azure uses Cerberus, a contribution we made to OCP in 2017 for hardware security, to improve BMC security by enforcing firmware integrity and preventing the persistence of malware in the BMC. However, as threat models evolve to restrict admins with physical access to hardware, the BMC needs security properties to establish secure links to an external RoT.

Microsoft collaborated with Nuvoton to design a new security-focused BMC, with enhanced hardware security throughout the BMC SoC. The silicon-integrated root of trust supports TCG DICE identity flows with hardware engines for fast cryptographic operations and hardware-managed keys. The RoT has a one-way bridge for activity monitoring and controlling the BMC security configuration, including which internal security peripherals the BMC can assess. This unique feature allows fine-grained BMC interface authorization, enabling scenarios whereby temporary access to a debug interface can be granted to the BMC only after it attests its trustworthiness.

Kirkland: A secure Trusted Platform Module (TPM)

While Microsoft provides multilayered security across our datacenters, infrastructure, and operations, we believe in defense-in-depth and that all interconnects should be cryptographically secured from interposer-based attack vectors. In partnership with Google, Infineon, and Intel, we are announcing Project Kirkland at OCP. Project Kirkland demonstrates how, using firmware-only updates to the TPM stack and CPU RoT, the interconnect between the TPM and CPU can be secured in a way that prevents substitution attacks, interposing, and eavesdropping. We are open sourcing this methodology and plan to work with the Trusted Computing Group on standardizing this approach while working with other TPM manufacturers to adopt the same methodology, so these techniques become available to all.

A discrete TPM is a chip typically used to protect secrets for the software running on the CPU and conditionally released based on the CPU's boot measurements. Historically, the bus between the CPU and the TPM is susceptible to attack from physical adversaries wishing to falsify attested measurements or obtain TPM-bound secrets. The standards-based firmware techniques used in Project Kirkland defend against such attacks by using cryptography to authenticate the caller and protect the transmission of secrets over the bus.

Open hardware innovation at cloud scale

A community-driven approach to infrastructure innovation is vital—not just for continued advancements in trust, efficiency, and scalability, but in service of a larger vision of empowering the ecosystem towards building the for computing needs of tomorrow.

We are also contributing several new hardware designs such as a new modular chassis (Mt. Shasta), a converged architecture that brings form factor, power, and management interface into a modular design—optimized for advanced workloads like high-performance computing, artificial intelligence, and video codecs. In partnership with Quanta and Molex, Mt. Shasta is designed to be fully compatible with Open Rack V3, with flexibility in changing module-module connectivity. Earlier this year, we also collaborated with Intel and contributed the Scalable I/O Virtualization (SIOV) specification to OCP. SIOV enables device and platform manufacturers to an industry standard for hyperscale virtualization of PCI Express and Compute Express Link devices in cloud servers, enabling more scalable, efficient, and cost-effective hardware designs for datacenters.

As the demand for cloud-scale computing and digital services continues to grow, Microsoft is committing to deep ecosystem collaboration with OCP and industry partners to deliver the systems and infrastructure that maximize performance, trust, and resiliency for cloud customers.

Connect with Microsoft at the OCP Global Summit 2022 and beyond

Visit Microsoft at OCP Global Summit: Booth A1
Check out sessions delivered by Microsoft & partners at OCP Global Summit.
Take a virtual tour of Microsoft datacenters.
Learn more about Microsoft’s global infrastructure.
Learn more about cloud hardware innovation at Microsoft

Quelle: Azure