da Agency

Amazon SageMaker Studio now integrates with Hugging Face for one-click model deployment and customization

Amazon SageMaker Studio now supports direct integration from Hugging Face, letting you go from discovering a model to working with it inside a fully configured Studio environment in a single click. Select any supported model on Hugging Face and choose “Customize on SageMaker AI” or “Deploy on SageMaker AI” to land directly on the corresponding workflow page with the model pre-loaded and ready to use.
Previously, getting from model discovery to a working environment required navigating the AWS Console to find SageMaker AI, configuring an environment, setting up IAM permissions for serverless model customization, and in many cases requesting GPU quota increases through Service Quotas before running a first job. Now, new customers complete a standard AWS sign-up and receive a SageMaker Studio environment created in seconds with pre-configured permissions for serverless model customization jobs including fine-tuning with custom reward functions for reinforcement learning, model evaluation, and deployment to SageMaker or Bedrock endpoints. Verified customers receive default GPU access to G5, G6, and G4dn instances across endpoint deployments, training jobs, and notebooks without requesting quota increases, and quota limit and utilization information is visible for each instance type directly inside the Studio environment. Returning customers signing in from Hugging Face or SageMaker product pages select their environment and land directly inside SageMaker Studio with the model ready to use.
This feature is available in all AWS Commercial Regions where Amazon SageMaker Studio is supported. To get started, visit any supported model on Hugging Face and select “Customize on SageMaker AI” or “Deploy on SageMaker AI,” or click Get Started from the SageMaker Studio page. To learn more, see Service quotas for Studio in the Amazon SageMaker documentation.
Quelle: aws.amazon.com

da Agency

Amazon Cognito now supports self-service provisioned API rate limits

Amazon Cognito now allows you to increase or decrease your provisioned API rate limits on demand. Cognito has default rate limits for the maximum number of operations per second that you can perform in your user pools in each AWS Region, and you can purchase additional limits on adjustable API categories. With the new on-demand model, you can adjust your rate limits up or down more quickly to match your application’s traffic patterns. Previously, to adjust your Cognito API rate limits, you would request an increase through Service Quotas, where requests are manually reviewed. This meant you had to plan rate limits in advance ahead of anticipated traffic spikes. Now, you have a new self-service experience to set your desired Cognito rate limit up to the account-level max limit using the Amazon Cognito console or the new limit provisioning API operations. Rate limit changes take effect immediately. Self-service provisioned limits are available for adjustable API categories in all AWS Regions where Amazon Cognito is available. For pricing details of this add-on feature, see Amazon Cognito pricing page. To get started, see developer guide.
Quelle: aws.amazon.com

da Agency

Amazon SageMaker HyperPod now supports disaggregated prefill and decode

Amazon SageMaker HyperPod now supports Disaggregated Prefill and Decode (DPD), an inference optimization that separates the two phases of large language model (LLM) inference — prefill and decode — onto dedicated GPU pools and transfers the key-value (KV) cache between them over Elastic Fabric Adapter (EFA) using GPU-Direct RDMA. Customers running LLMs in production for chat assistants, agentic pipelines, retrieval-augmented generation, and long-document analysis need consistent per-token latency and predictable throughput under mixed traffic, but when prefill and decode share the same GPU, a single long-context request can stall token generation for every concurrent request and force customers to over-provision one phase to protect the other. With DPD, customers run compute-bound prefill on one set of GPUs and memory-bandwidth-bound decode on another, so the two phases no longer contend for the same resources. This delivers more consistent per-token latency under sustained concurrency, higher goodput at strict latency SLOs, and the ability to scale prefill and decode capacity independently to match the input and output distribution of the workload. An intelligent router automatically directs long-context requests through the disaggregated path and sends shorter prompts directly to the decoder, so customers get the benefit on the traffic that needs it without paying transfer overhead on short prompts. Customers enable DPD by adding a `pdSpec` section to the same `InferenceEndpointConfig` custom resource they already use for inference endpoints on the HyperPod Inference Operator, and DPD is composable with the existing KV cache offloading and intelligent routing features on HyperPod. DPD is available for SageMaker HyperPod clusters using the EKS orchestrator on EFA-capable instance types in all AWS Regions where Amazon SageMaker HyperPod is available. To learn more, see Disaggregated Prefill and Decode for HyperPod inference in the Amazon SageMaker AI Developer Guide.
Quelle: aws.amazon.com