Supported Annotations

Voyager operator allows customization of Ingress resource using annotation keys with ingress.appscode.com/ prefix. The ingress annotaiton keys are always string. Annotation values might have the following data types:

Value TypeDescriptionExample YAML
stringany valid string‘v1’; “v2”
integerany valid integer‘1’; “2”
bool1, t, T, TRUE, true, True considered true; everything else is considered false‘true’
arrayjson formatted array of string‘[“v1”, “v2”]’
mapjson formatted string to string map’{ “k1” : “v1”, “k2”: “v2” }’
enumstring which has a predefined set of valid values‘E1’; “E2”

If you are using YAML to write your Ingress, you can use any valid YAML syntax, including multi-line string. Here is an example:

annotations:
  ingress.appscode.com/type: LoadBalancer
  ingress.appscode.com/replicas: '2'
  ingress.appscode.com/load-balancer-ip: '100.101.102.103'
  ingress.appscode.com/stats: 'true'
  ingress.appscode.com/stats-port: '2017'
  ingress.appscode.com/stats-secret-name: my-secret
  ingress.appscode.com/annotations-service: |
    {
        "service.beta.kubernetes.io/aws-load-balancer-backend-protocol": "http",
        "service.beta.kubernetes.io/aws-load-balancer-proxy-protocol": "*",
        "service.beta.kubernetes.io/aws-load-balancer-ssl-cert": "arn:aws:acm:..."
    }

Below is the full list of supported annotation keys, voyager also support standard ingress annotations:

KeysValueDefaultDescription
ingress.appscode.com/typeLoadBalancer, HostPort, NodePort, InternalLoadBalancerRequired. Indicates type of service used to expose HAProxy to the internet
ingress.appscode.com/replicasinteger1Optional. Indicates number of replicas of HAProxy pods
ingress.appscode.com/load-balancer-ipstringxOptional. For “gce”, “gke”, “azure”, “acs” cloud provider, if this value is set to a valid IPv4 address, it will be assigned to loadbalancer used to expose HAProxy. The IP should be pre-allocated in cloud provider account but not assigned to the load-balancer. Usually this is set to a static IP to preserve DNS configuration
ingress.appscode.com/node-selectormapxIndicates which hosts are selected to run HAProxy pods. This is a recommended annotation for HostPort type ingress.
ingress.appscode.com/sticky-sessionboolfalseOptional. Indicates the session affinity for the traffic. If set, session affinity will apply to all the rulses.
ingress.appscode.com/annotations-servicemapxOptional. Annotaiotns applied to service used to expose HAProxy
ingress.appscode.com/annotations-podmapxOptional. Annotations applied to pods used to run HAProxy
ingress.appscode.com/keep-source-ipboolfalseOptional. If set, preserves source IP for LoadBalancer type ingresses. The actual configuration generated depends on the underlying cloud provider. For gce, gke, azure: Adds annotation service.beta.kubernetes.io/external-traffic: OnlyLocal to services used to expose HAProxy. For aws, enforces the use of the PROXY protocol.
ingress.appscode.com/accept-proxyboolfalseOptional. If set, enforces the use of the PROXY protocol.
ingress.appscode.com/statsboolfalseOptional. If set, HAProxy stats will be exposed
ingress.appscode.com/stats-portinteger56789Optional. Port used to expose HAProxy stats
ingress.appscode.com/stats-secret-namestringxOptional. Secret used to provide username & password to secure HAProxy stats endpoint. Secret must contain keys username and password
ingress.appscode.com/stats-service-namestringvoyager-<ingress-name>-statsClusterIP type service used to expose HAproxy stats. This allows to avoid exposing stats to internet.
ingress.appscode.com/ipRemoved since 1.5.6. Use ingress.appscode.com/load-balancer-ip
ingress.appscode.com/persistRemoved since 1.5.6.
ingress.appscode.com/daemon.nodeSelectorRemoved since 1.5.6. Use ingress.appscode.com/node-selector
ingress.appscode.com/stickySessionRemoved since 1.5.6. Use ingress.appscode.com/sticky-session
ingress.appscode.com/annotationsServiceRemoved since 1.5.6. Use ingress.appscode.com/annotations-service
ingress.appscode.com/annotationsPodRemoved since 1.5.6. Use ingress.appscode.com/annotations-pod
ingress.appscode.com/statsSecretNameRemoved since 1.5.6. Use ingress.appscode.com/stats-secret-name

Following annotations for ingress are not modifiable. The configuration is applied only when an Ingress object is created. If you need to update these annotations, then first delete the Ingress and then recreate.

ingress.appscode.com/type
ingress.appscode.com/node-selector
ingress.appscode.com/load-balancer-ip

The issue is being tracked here.

Custom Annotations to LoadBalancer Service or Pods

If the LoadBalancer service and Pods needs to be set custom annotations, those can be set via two ingress options.

ingress.appscode.com/annotations-service Json encoded annotations map that will be applied to LoadBalancer service.

ie.

ingress.appscode.com/annotations-service = {"foo": "bar", "service-annotation": "set"}

This will add the foo:bar and service-annotation:set to the Service annotation.

ingress.appscode.com/annotations-pod Json encoded annotations map that will be applied to LoadBalancer pods.

ie.

ingress.appscode.com/annotations-pod = {"foo": "bar", "pod-annotation": "set"}

This will add the foo:bar and pod-annotation:set to all the pods’ annotation.

Ingress Annotations Support

This file defines the list of supported annotations by voyager that are used in other ingress controllers. voyager intent to ensure maximum amount compatibility between different implementations.

Voyager supports applying specified annotations in ingress or in backend service. If applied to ingress configuration will be applied on all backend of the ingress, if applied to service configurations will only apply on those backends.

Ingress Annotations

All following annotations are assumed to be prefixed with ingress.kubernetes.io/. Voyager also supports some particular defines annotation, which are described here.

NameDetailsAnnotation applies to
TLS
ssl-passthroughPass TLS connections directly to backend; do not offload.ingress
hstsEnable HSTSingress
hsts-max-ageSpecifies the time (in seconds) the browser should connect to the server using the HTTPS connection.ingress
hsts-preloadEnable HSTS preloadingress
hsts-include-subdomainsHSTS rule applies to all of the site’s sub domainsingress
Authentication
auth-typeEnable Basic Authingress, service
auth-secretBasic Auth user secretingress, service
auth-realmBasic Auth realmingress, service
Miscellaneous
enable-corsEnables CORS headers in HTTP responseingress
affinitySticky session. only supported value is cookieingress, service
session-cookie-nameSticky session cookie name to setingress, service
session-cookie-hashSticky session cookie typeingress, service
proxy-body-sizeMaximum http request body size. This limits the advertised length of the HTTP request’s body in bytes.ingress

Voyager Annotations

voyager supports other annotations to serve different purpose prefixed with ingress.appscode.com/. Annotations can be applied on ingress or backends.

NameDetailsAnnotation applies to
typeDefines loadbalancer typeingress
replicasScale load balancer replicaingress
backend-weightWeighted Loadbalancing for Canary Deploymentpod
annotations-serviceAdd Custom Annotation to LoadBalancer Serviceingress
annotations-podAdd Custom Annotation to LoadBalancer Podsingress
accept-proxyAccept proxy protocolingress
default-timeoutConfigure Custom Timeouts for HAProxyingress
default-optionConfigure Options for HAProxyingress
backend-tlsTLS enabled Backendservice, ingress
sticky-session (deprecated)Configure Sticky session to Backendsservice, ingress
use-dns-resolverSupports redirects/DNS resolution for ExternalName type serviceingress
dns-resolver-nameserversSupports redirects/DNS resolution for ExternalName type serviceingress
dns-resolver-check-healthSupports redirects/DNS resolution for ExternalName type serviceingress
dns-resolver-retriesSupports redirects/DNS resolution for ExternalName type serviceingress
dns-resolver-timeoutSupports redirects/DNS resolution for ExternalName type serviceingress
dns-resolver-holdSupports redirects/DNS resolution for ExternalName type serviceingress
statsExpose HAProxy statsingress
stats-portExpose HAProxy statsingress
stats-secret-nameExpose HAProxy statsingress
stats-service-nameExpose HAProxy statsingress
monitoring-agentExpose HAProxy stats using prometheusingress
service-monitor-labelsExpose HAProxy stats using prometheusingress
service-monitor-namespaceExpose HAProxy stats using prometheusingress
service-monitor-endpoint-portExpose HAProxy stats using prometheusingress
service-monitor-endpoint-scrape-intervalExpose HAProxy stats using prometheusingress

Acknowledgements