Indiegames-Rundschau: Von Bananen und Astronauten

In Outer Wilds erlebt ein Astronaut in Murmeltier-Manier das immer gleiche Abenteuer, in My Friend Pedro tötet ein maskierter Auftragskiller im Auftrag einer Banane, dazu gibt es Horror von Lovecraft: Golem.de stellt die Indiegames-Highlights des Sommers vor. Von Rainer Sigl (Indiegames Rundschau, Spieletest)
Quelle: Golem

MileIQ and Azure Event Hubs: Billions of miles streamed

This post was co-authored by Shubha Vijayasarathy, Program Manager, Azure Messaging (Event Hubs)

With billions of miles logged, MileIQ provides stress-free logging and accurate mileage reports for millions of drivers. Logging and reporting miles driven is a necessity for independent contractors to organizations with employees who need to drive for work. MileIQ automates mileage logging to create accurate records of miles driven, minimizing the effort and time needed with manual calculations. Real-time mileage tracking produces over a million location signal events per hour, requiring fast and resilient event processing that scales.

MileIQ leverages Apache Kafka to ingest massive streams of data:

Event processing: Events that demand time-consuming processing are put into Kafka, and multiple processors consume and process these asynchronously.
Communication among micro-services: Events are published by the event-owning micro-service on Kafka topics. The other micro-services, which are interested in these events, subscribe to these topics to consume the events.
Data Analytics: As all the important events are published on Kafka, the data analytics team subscribes to the topics it is interested in and pulls all the data it requires for data processing.

Growth Challenges

As with any successful venture, growth introduces operational challenges as infrastructure struggles to support the growing demand. In MileIQ’s case, the effort and resources required to maintain Apache Kafka clusters multiplied exponentially with adoption. A seemingly simple task, like modifying a topic’s retention configuration, now becomes an operational burden as the number of Kafka clusters scale to meet the increase in data.

Leveraging a managed service, enabled MileIQ to shift resources from operations and maintenance to focus on new ways to drive business impact. A couple reasons why the MileIQ team selected Azure Event Hubs for Kafka:

Fully managed platform as a service (PaaS): With little configuration or management overhead, Event Hubs for Kafka provides a PaaS Kafka experience without the need to manage, configure, or run Kafka clusters.
Supports multiple Kafka use-cases: Event Hubs for Apache Kafka provides support at the protocol level, enabling integration of existing Kafka applications with no code changes and minimal configuration change. MileIQ’s existing Kafka producers and consumers, as well as other streaming applications like Apache Kafka MirrorMaker and Apache Spark, integrated seamlessly with the Kafka-enabled Event Hub.
Deliver streaming data to Azure Blob storage: The Capture feature of Event Hubs automatically send data from Azure Event Hubs for Kafka to Blob storage. MileIQ uses the data in Blob storage for data analytics and backup.
Enterprise performance: The Dedicated-tier cluster offers single-tenant deployments with a guaranteed 99.99% SLA. MileIQ performance tests showed the Dedicated-tier cluster was able to consistently produce a throughput rate of 6,000 events per second.

* Testing based on one event at a time synchronously to address specific use-cases focused on consistency over throughput. Testing batching and produce asynchronously resulted in a much higher throughput.

Set up for success

As a result of migrating Apache Kafka to a managed service, MileIQ now has the infrastructure needed to support future growth.

“To sum up, our experience switching over to Azure Event Hubs Kafka has been excellent. To start with, the onboarding was straightforward, integration was seamless, and we continue to receive great help and support from the Azure Event Hubs Kafka team. In the near future, we look forward to the release of new features that the Azure Event Hubs Kafka team is working on – Geo Replication, Idempotent Producers, Kafka Streams, etc.”

“Migrating to Azure Event Hubs Kafka was a painless experience. Straightforward onboarding seamless integration, and support from the Event Hubs team every step of the way.  We’re excited to see what’s next and look forward to a continued partnership.” – MileIQ

Start streaming data

Data is valuable only when there is an easy way to process and get timely insights from data sources. Azure Event Hubs provides a fully managed distributed stream processing platform with low latency and seamless integration with Apache Kafka applications.

What are you waiting for? Time to get event-ing!

Event Hubs for Apache Kafka overview
Get started: Create Apache Kafka-enabled event hubs
Discover more about Azure Event Hubs
Learn about Dedicated-tier clusters

Enjoyed this blog? Follow us as we update the features list we will start supporting. Leave us your valuable feedback, questions, or comments below.

Happy event-ing!
Quelle: Azure

How HSBC built its PayMe for Business app on Microsoft Azure

Bank-grade security, super-fast transactions, and analytics 

If you live in Asia or have ever traveled there, you’ve probably witnessed the dramatic impact that mobile technology has had on all aspects of day to day life. In Hong Kong in particular, most consumers now use a smart phone daily, presenting new opportunities for organizations to deliver content and services directly to their mobile devices.

As one of the world’s largest international banks, HSBC is building new services on the cloud to enable them to organize their data more efficiently, analyze it to understand their customers better, and make more core customer journeys and features available on mobile first.

HSBC’s retail and business banking teams in Hong Kong have combined the convenience afforded by smart phones with cloud services to allow “cashless” transactions where people can use their smart phone to perform payments digitally. Today, over one and a half million people use HSBC’s PayMe app to exchange money with people in their personal network for free. And businesses are using HSBC’s new PayMe for Business app, built natively on Azure, to collect payments instantly, with 98 percent of all transactions completed in 500 milliseconds or less. Additionally, the businesses can leverage powerful built-in intelligence on the app to improve their sales and operations.

On today’s Microsoft Mechanics episode of “How We Built it,” Alessio Basso, Chief Architect of PayMe from HSBC, explains the approach they took and why.

Bank-grade security, faster time to delivery, dynamic scale and resiliency

The first decision Alessio and team made was to use fully managed services to allow them to go from ideation to a fully operational service in just a few months. Critical to their approach was adopting a microservices-based architecture with Azure Kubernetes Service and Azure Database for MySQL.

They designed each microservice to be independent, with their own instance of Azure managed services, including Azure Database for MySQL, Azure Event Hub, Azure Storage, Azure Key Vault for credentials and secrets management, and more. They architected for this level of isolation to strengthen security and overall application uptime, as shared dependencies are eliminated.

Each microservice can rapidly scale compute and database resources elastically and independently, based on demand. What’s more, Azure Database for MySQL, allows for the creation of read replicas to offload read-only and analytical queries without impacting payment transaction response times.

Also, from a security perspective, because each microservice runs within its own subnet inside of Azure Virtual Network, the team is able to isolate network and communications back and forth between Azure resources with service principals via Virtual Network service endpoints.

Fast and responsive analytics platform

At its core, HSBC’s PayMe is a social app that allows consumers to establish their personal networks, while facilitating the interactions and transactions with the people in their circle and business entities. In order to create more value for both businesses and consumers, Azure Cosmos DB is used for graph data modelled to store customer-merchant-transaction relationships.

Massive amounts of structured and unstructured data from Azure Database for MySQL, Event Hubs, and Storage are streamed and transformed. The team designed an internally developed data ingestion process, feeding an analytical model called S.L.I.M (simple, lightly, integrated model), optimized for analytics queries performance, as well as making data virtually available to the analytics platform, using Azure Databricks Delta’s unmanaged table capability.

Then machine learning within their analytics platform built on Azure Databricks allows for the quick determination of patterns and relationships, as well as for the detection of anomalous activity.

With Azure, organizations can immediately take advantage of new opportunities to deliver content and services directly to mobile devices, including a next-level digital payment platform.

To learn more about how HSBC architected their cashless digital transaction platform, please watch the full episode.
Learn more about achieving microservice independence with your own instance of a Azure managed service like Azure Database for MySQL.

Quelle: Azure