Beyond the printed form: Unlocking insights from documents with Form Recognizer

Data extraction from printed forms is by now a tried and true technology. Form Recognizer extracts key value pairs, tables and text from documents such as W2 tax statements, oil and gas drilling well reports, completion reports, invoices, and purchase orders. However, real-world businesses often rely on a variety of documents for their day-to-day needs that are not always cleanly printed.

We are excited to announce the addition of handwritten and mixed-mode (printed and handwritten) support. Starting now, handling handwritten and mixed-mode forms is the new norm.

Extracting data from handwritten and mixed-mode content with Form Recognizer

Entire data sets that were inaccessible in the past due to the limitations of extraction technology now become available. The handwritten and mixed-mode capability of Form Recognizer is available in preview and enables you to extract structured data out of handwritten text filled in forms such as:

Medical forms: New patient information, doctor notes.
Financial forms: Account opening forms, credit card applications.
Insurance: Claim forms, liability forms.
Manufacturing forms: Packaging slips, testing forms, quality forms.
And more.

By using our vast experience in optical character recognition (OCR) and machine learning for form analysis, our experts created a state-of-the-art solution that goes beyond printed forms. The OCR technology behind the service supports both handwritten and printed. Expanding the scope of Form Recognizer allows you to tap into previously uncharted territories, by making new sources of data available to you. You may extract valuable business information from newly available data, keeping you ahead of your competition.

Whether you are using Form Recognizer for the first time or already integrated it into your organization, you will now have an opportunity to create new business applications:

Expand your available data set: If you are only extracting data from machine printed forms, expand your total data set to mixed-mode forms and historic handwritten forms.
Create one application for a mix of documents: If you use a mix of handwritten and printed forms, you can create one application that applies across all your data.
Avoid manual digitization of handwritten forms: Original forms may be fed to Form Recognizer without any pre-processing, extracting the same key-value pairs and table data you would get from a machine-printed form to reduce costs, errors, and time.

Our customer: Avanade

Avanade values people as their most important asset. They are always on the lookout for talented and passionate professionals to grow their organization. One way they find these people is by attending external events, which may include university career fairs, trade shows, or technical conferences to name a few. 

During these events they often take the details of those interested in finding out more about Avanade, as well as their permission to contact them at a later date. Normally this is completed with a digital form using a set of tablets. But when the stand is particularly busy, they use a short paper form that attendees can fill in with their handwritten details. Unfortunately, these forms needed to be manually entered into the marketing database, requiring a considerable amount of time and resources. With the volume of potential new contacts at these events, multiplied by the number of events Avanade attends, this task can be daunting.

Azure Form Recognizer’s new handwritten support simplifies the process, giving Avanade peace of mind knowing no contact is lost and the information is there for them immediately.

In addition, Avanade integrated Form Recognizer as a skill within their cognitive search solution, enabling them to quickly use the service in their existing platform and follow-up with new leads, while their competitors may be spending time digitizing their handwritten forms.

“Azure Form Recognizer takes a vast amount of effort out of the process, changing the task from data entry to data validation. By integrating Form Recognizer with Azure Search, we are also immediately able to use the service in our existing platforms. If we need to find and check a form for any reason, for example to check for a valid signature there, we can simply search by any of the fields like name or job title and jump straight to that form. In our initial tests, using Form Recognizer has reduced the time taken to digitize the forms and double check the entries by 35 percent, a number we only expect to get better as we work to optimize our tools to work hand in hand with the service, and add in more automation.” – Fergus Kidd, Emerging Technology Engineer, Avanade

Getting started

To learn more about Form Recognizer and the rest of the Azure AI ecosystem, please visit our website and read the documentation.

Get started by contacting us.

For additional questions please reach out to us at formrecog_contact@microsoft.com
Quelle: Azure

Build a dev workflow with Cloud Code on a Pixelbook

Can you use a Pixelbook for serious software development? Do you want a workflow that is simple, doesn’t slow you down, and is portable to other platforms? And do you need support for Google Cloud Platform SDK, Kubernetes and Docker? I switched to a Pixelbook for development, and I love it!Pixelbooks are slim, light, ergonomic, and provide great performance. Chrome OS is simple to use. It brings many advantages over traditional operating systems: frictionless updatesenhanced securityextended battery lifeAnd the most compelling feature for me: almost instant coming to life after sleep. This is great when hopping between meetings and on the road. A little about me – I’m a Developer Programs Engineer. I work on Google Cloud and contribute to many open source projects. I need to accomplish repeatable development tasks: working with Github, build, debug, deploy and observe. Running and testing the code on multiple platforms is also of high importance. I can assure you, the workflow below built on Pixelbook satisfies all the following:Simple, repeatable development workflow with emphasis on developer productivityPortable to other platforms (Linux, MacOS, Windows)—“create once, use everywhere”Support for Google Cloud Platform SDK, Github, Kubernetes and Docker.Let’s dive into how you can set up a development environment on Pixelbook that meets all those requirements using Cloud Code for Visual Studio Code, remote extensions, and several other handy tools. If you are new to the world of Chromebooks and switching from a PC, check out this post to get started.Step 1: Enable Linux apps on PixelbookLinux for Chromebooks (aka Crostini) is a project to let developers do everything they need locally on a Chromebook, with an emphasis on web and Android app development. It adds Linux support.  On your Pixelbook:1. Go to Settings (chrome://settings) in the built-in Chrome browser.2. Scroll down to the “Linux (Beta) ” section (see screenshot below).3. Click “Turn on” and follow the prompts. It may take up to 10 minutes depending on your Wi-Fi connection.4. At the end, a new Terminal window should automatically open to a shell within the container. We’re all set to continue to the next step – installing developer tools!Pin the terminal window to your program bar for convenience.Configure Pixelbook keyboard to respect Function keysFolks coming from Windows or MacOS backgrounds are used to using Function keys for development productivity. On Chrome OS, they are replaced by default to a group of shortcuts. However, we can bring them back:Navigate to chrome://settings. Now, pick “Device” on the left menu, then pick “keyboard”. Toggle “treat top-row keys as function keys”:Step 2: Install development toolsFor Kubernetes development on GCP, we need to install tools like Docker, Google Cloud SDK and kubectl. Pixelbook Linux is Debian Stretch, so we will install prerequisites for docker and gcloud using instructions for Debian Stretch distribution.Install and configure Google Cloud SDK (gcloud):Run these commands from gcloud Debian quickstart to install gcloud sdk:TroubleshootingYou might run into this error:Your keyrings are out of date. Run the following commands and try the Cloud SDK commands again:Add gcloud to PATHInstalling Docker CE for Linux:Follow these instructions.And then add your user to the docker group:NOTE: This allows running docker commands without sudo.Install kubectlInstalling Visual Studio CodeGo to VSCode linux install instructions page.Download the.deb package (64bit) from the link on the page.After the download is complete, install the deb file using “Install app with Linux (beta)”:TroubleshootingIf you don’t see “Install with Linux” as an option for the deb file, double check that you switched to the beta channel.Now let’s install a few extensions that I find helpful when working on a remote container using VS Code:Docker – managing docker images, autocompletion for docker files, and more.Remote Containers – use a docker container as a full-featured development environment. These two, along with Cloud Code, are key extensions in our solution.Step 3: Configuring Github accessConfigure github with SSH keyNow copy and past the key into Github.NOTE:If facing permissions error doing ssh-add, run sudo chown $USER .ssh and re-run all the steps for github setup again.Set the username and email of github:Step 4: Remote developmentNow that we have the tools installed and Github access configured, let’s configure our development workflow. In order to create a solution that is portable to other platforms, we will use remote containers extension. We will create a container that will be used to build, deploy and debug applications that we create. This is how it will work:We will open our codebase in a remote container. This will let VS Code think that it is open in isolated Linux environment, so everything we do (build, deploy, debug, file operations) will be interpreted as if we were working on a dedicated Linux VM with its own file system: every command we execute on VS Code will be sent for execution on our remote container. This way we achieve the goal of portability—remote Linux container can run on both MacOS and Windows just like we do it on Pixelbook with Chrome OS that supports Linux.Dev Container settings for each repoHere’s how to set up a dev container for an existing project. You can find the full source code in the Cloud Code templates repo. This Github repo includes templates for getting started with repeatable Kubernetes development in five programming languages—Node.js, Go, Java, Python and .NET. Each template includes configuration for debugging and deploying the template to Kubernetes cluster using Cloud Code for VS Code and IntelliJ. For simplicity, we work with a HelloWorld template that just serves “Hello World” message from a simple web server in a single container.To enable remote container development, we need to add a .devcontainer folder with two files:Dockerfile — defines container image that holds all developer tools we need installed in a remote development containerDevcontainer.json — Instructs VS Code Remote Tools extension how to run remote development container.Creating a container image for remote developmentOur remote container needs to have the SDK we use for development in the programming language of our choice. In addition, it needs tools that enable Cloud Code and Kubernetes workflows on Google Cloud. Therefore in the Dockerfile we install:Google Cloud SDKSkaffold — tool Cloud Code uses for handling the workflow for building, pushing and deploying apps in containersDocker CLIIn addition, container images are immutable. Every time we open the code in a remote container, we’ll get a clean state—no extra settings will be persisted between remote container reloads by default (kubernetes clusters to work with, gcloud project configuration, github ssh keys). To address that, we mount our host folders as drives in the container (see this part later in devcontainer.json) and copy its content to the folder in the container file system where dev tools expect to find these files. Example from Dockerfile of kubeconfig, gcloud and ssh keys sync between host and remote container:devcontainer.jsonThis file tells Remote Container extension which ports to expose in the container, how to mount drives, which extensions to install in the remote container, and more.A few notable configurations:runArgs contains command line arguments remote extension passes to docker when remote container is launched. This is where we set environment variables and mount external drives in a container. This helps to eliminate authorizations and specifies the kubernetes clusters we want to work with in Cloud Code.In the extensions section, we add a few VS Code extensions for enhanced productivity in the development container. These will be installed on a dev container but not on the host, so you can tailor this choice to the codebase you plan to work on in the dev container. In this case I am setting up for nodejs development.Cloud Code for VS Code — Google’s extension that helps to write, deploy and debug cloud-native applications quickly and easily. It allows deploying code to kubernetes and supports 5 programming languages.Npm support for VS CodeCode Spell CheckerMarkdownlint — Improves the quality of markdown files. Gitlens — Shows the history of code commits along with other relevant useful information.Output colorizer — Colors the output of various commands. Helpful when observing application logs and other info in the IDE.Vscode-icons — Changes icons to known file extensions for better visibility and discoverability of the files.Docker — Manages docker images, autocompletion for docker files and moreTSLint — Linting for typescript (optional)Bracket pair colorizer (optional)Npm intellisense (optional)ESLint Javascript (optional)Hello World in Dev Container on PixelbookLet’s try to build, debug and deploy the sample Hello World nodejs app on Pixelbook using the remote dev container setup we just created:Initialize gcloud by running gcloud init in a command line of your Pixelbook and following the steps. As part of our earlier setup, when we open the code in a remote container, Gcloud settings will be sync’ed into a dev container, so you won’t need to re-initialize every time.Connect to a GKE cluster using the command below. We will use it to deploy our app. This also can be done outside of the dev container and will be sync’ed using our earlier setup in .devsettings.Open the code in dev container: In VS Code command palette, type: Remote-Containers: Open Folder in Container… and select your code location. The code will open in dev container, pre-configured with all the toolset and ready to go!Build and deploy the code to GKE using Cloud Code: In VS Code Command Palette, type: Cloud Code: Deploy and follow the instructions. Cloud Code will build the code, package it into container image, push it into container registry, then deploy it into GKE cluster we initialized earlier—all from the dev container on a Pixelbook!Though slick and small, the Pixelbook might just fit your developer needs. With VS Code, Remote development extension, Docker, Kubernetes and Cloud Code you can lift your development setup to the next level, where there is no need to worry about machine-specific or platform-specific differences affecting your productivity. By sharing dev container setup on Github, developers that clone your code will be able to reopen it in a container (assuming they have the Remote – Containers extension installed).Once done, developers will get an isolated environment with all dependencies baked in — just start coding!If you have a Pixelbook — or if you don’t, and just want to try out Cloud Code — the Hello World app and all config files are available on GitHub. Let me know how it went and what your favorite setup for developer productivity is.Further readingSet up Linux (Beta) on your ChromebookChromebook Developer ToolboxGetting Started with Cloud Code for VS CodeCloud Code Templates RepoDeveloping inside a Container
Quelle: Google Cloud Platform

Last month today: August on GCP

Last month on the Google Cloud Platform (GCP) blog, we dove into hardware, software, and the humans who make technology work. Here’s what topped our charts in August.Exploring the nuts and bolts of cloudGoogle already uses AMD’s EPYC processors for internal workloads, and last month we announced that they’re coming to the data centers that power Google Cloud products. Second-gen AMD EPYC processors will soon power our new virtual machines—the largest general-purpose VMs we’ve ever offered. There will be a range of sizes for these AMD VMs so you can choose accordingly, and can also configure them as custom machine types. Improvements like these can help you get more performance for the price for your workloads. One small button can make it easy for other developers to deploy your app to GCP using Cloud Run, our managed compute platform that lets you deploy containerized serverless apps. You can add the new Cloud Run Button to any source code repository that has a dockerfile or that can be built with Cloud Native Buildpacks. One click will package the app source code as a container image, push it to Google Container Registry, then deploy it on Cloud Run. Looking at the human side of technologyThis blog post offered a look at the tradeoffs that CIOs and CTOs have to make in their pursuit of business acceleration in a hybrid world, based on recent McKinsey research. While digital transformation and new tech capabilities are in high demand, leaders can avoid making tradeoffs by choosing technology wisely and making necessary operational changes too, including fostering a change mindset. There are tips here on embracing a DevOps model, using a flexible hybrid cloud model, and adopting open-source architectures to avoid common pitfalls.This year’s Accelerate State of DevOps Report is available now, and offers a look at the latest in DevOps, with tips for organizations at all stages of DevOps maturity. This year, data shows that the percentage of elite performers is at its highest ever, and that these elite performers are more likely to use cloud. The report found that most cloud users still aren’t getting all of its benefits, though. DevOps should be a team effort, too, with both organizational and team-level efforts important for success.How customers are developing with cloudGoogle Cloud customers are pushing innovation further to serve customers in lots of interesting ways. First up this month is Macy’s, which uses Google Cloud to help provide customers with great online and in-person experiences. The company is streamlining retail operations across its network with cloud, and uses GCP’s data warehousing and analytics to optimize all kinds of merchandise tasks at its new distribution center.We also heard this month from Itau Unibanco of Brazil, which developed a digital customer service tool to offer instant help to bank users. They use Google Cloud to build a Kubeflow-based CI/CD pipeline to deploy machine learning models and serve customers quickly and accurately. The post offers a look at their architecture and offers tips for replicating the pipeline.Last but not least, check out this story on how web developers are using Google Maps Platform and custom Street View imagery to offer virtual tours to the top of Zugspitze, the tallest mountain in Germany. Along with exploring APIs and deciding how to use the technology, the developers took a ton of 360° photos while hiking up and down parts of the 10,000-foot mountain. Take the tour yourself on their site.That’s a wrap for August! Stay tuned on the blog for all the latest.
Quelle: Google Cloud Platform