Overview
Crusoe Cloud currently offers L4 passthrough load balancers through our managed load balancers, allowing you to distribute TCP traffic across one or more backends. These load balancers can be used to distribute inference traffic across your GPU fleet, provide ingress for dashboards and services used to monitor model training runs or front self-managed cluster control planes.
This capability is currently in Limited Availability, and may not be visible to your organization by default. To request access, please contact support.
Details
Load balancers on Crusoe Cloud are regional resources and are tied to a specific VPC network. Each load balancer receives a static public IPv4 address and be configured to listed for traffic on one or more ports. Any traffic received to these ports can be routed to one or more backends.
Key Concepts
Backends and Health Checks
Each load balancer backend is represented by a tuple of private IPv4 address:port. These backends typically represent services running on Crusoe Cloud virtual machines, either directly or via a pod orchestrated by a Kuberentes cluster.
The health of each of these backends is monitored by a configurable TCP health check. Any backends that are detected as non-responsive are marked as 'Offline', with traffic no longer routed to them by the load balancer. The table below provides an overview of the fields that can be configured for the health check.
| Field | Description | Default Value |
|---|---|---|
Timeout | The maximum amount of time, in seconds, that the load balancer will wait for a healthy response from a backend. | 5 seconds |
Interval | The frequency, in seconds, at which the load balancer will send health check probes to a backend server. | 5 seconds |
Success Count | The number of consecutive successful health checks required for a backend to be brought back online | 3 |
Failure Count | The number of consecutive failed health checks that will cause a backend server to be marked as unhealthy and taken out of rotation. | 2 |
Protocols
Only TCP traffic load balancing is supported currently.
Load Balancing Algorithms
Traffic is distributed across each load balancer using IP-hash based load balancing based on the source IP of inbound traffic. This is not currently configurable.
Firewall Rules
Firewall rules are not currently created by default when backends are added to a load balancer. To ensure that your backends receive inbound traffic on the desired ports, please create the appropriate firewall rules.
Firewall rules must also be applied directly to the backends that are attached to the load balancer. We do not currently support specifying firewall rules directly for the load balancer resource.
Kubernetes LoadBalancer Service Integration
We support a dedicated Load Balancer controller that creates and manages Load Balancers in your project / organization when LoadBalancer services are created within your Kubernetes cluster. Installation instructions are outlined here.