New to Kubed? Please start here.
Supported Notifiers
Kubed can send notifications via Email, SMS or Chat for various operations using appscode/go-notify library. To connect to these services, you need to create a Secret with the appropriate keys. Then pass the secret name to Kubed by setting notifierSecretName field in Kubed cluster config. Also, set clusterName to a meaningful name for you. This cluster name will be prefixed to any notification sent via Email/SMS/Chat so that you can identify the source easily.
Hipchat
To receive chat notifications in Hipchat, create a Secret with the following key:
| Name | Description |
|---|
| HIPCHAT_AUTH_TOKEN | Required Hipchat api access token. You can use room notification tokens, if you are planning to send notifications to a single room. |
| HIPCHAT_BASE_URL | Optional Base url of Hipchat server |
| HIPCHAT_CA_CERT_DATA | Optional PEM encoded CA certificate used by Hipchat server |
| HIPCHAT_INSECURE_SKIP_VERIFY | Optional If set to true, skips SSL verification |
| HIPCHAT_NOTIFY | Optional If set to true, a notification will be triggered |
| HIPCHAT_COLOR | Optional Set color of message sent. Possible values: yellow, green, red, purple, gray, random. |
$ echo -n 'your-hipchat-auth-token' > HIPCHAT_AUTH_TOKEN
$ kubectl create secret generic notifier-config -n kube-system \
--from-file=./HIPCHAT_AUTH_TOKEN
secret "notifier-config" created
apiVersion: v1
data:
HIPCHAT_AUTH_TOKEN: eW91ci1oaXBjaGF0LWF1dGgtdG9rZW4=
kind: Secret
metadata:
creationTimestamp: 2017-07-25T01:54:37Z
name: notifier-config
namespace: kube-system
resourceVersion: "2244"
selfLink: /api/v1/namespaces/kube-system/secrets/notifier-config
uid: 372bc159-70dc-11e7-9b0b-080027503732
type: Opaque
Now, to receive notifications via Hipchat, configure receiver as below:
- notifier:
Hipchat - to: a list of chat room names
clusterName: unicorn
eventForwarder:
receivers:
- notifier: Hipchat
to:
- ops-alerts
notifierSecretName: notifier-config
Stride
To receive chat notifications in Stride, create a Secret with the following key:
| Name | Description |
|---|
| STRIDE_CLOUD_ID | Required You can get stride cloud_id from your organization’s URL |
| STRIDE_ROOM_TOKEN | Optional Stride room token |
If you login into your stride web app, you will see an URL with this format https://app.stride.com/{{cloud ID}}/lobby. From here, you will get cloud_id.
$ echo -n 'your-stride-cloud-id' > STRIDE_CLOUD_ID
$ echo -n 'your-stride-room-token' > STRIDE_ROOM_TOKEN
$ kubectl create secret generic notifier-config -n demo \
--from-file=./STRIDE_CLOUD_ID \
--from-file=./STRIDE_ROOM_TOKEN
secret "notifier-config" created
apiVersion: v1
data:
STRIDE_CLOUD_ID: eW91ci1zdHJpZGUtY2xvdWQtaWQ=
STRIDE_ROOM_TOKEN: eW91ci1zdHJpZGUtcm9vbS10b2tlbg==
kind: Secret
metadata:
creationTimestamp: 2017-07-25T01:54:37Z
name: notifier-config
namespace: demo
resourceVersion: "2244"
selfLink: /api/v1/namespaces/demo/secrets/notifier-config
uid: 372bc159-70dc-11e7-9b0b-080027503732
type: Opaque
Now, to receiver notifications via Stride, configure receiver as below:
- notifier:
Stride - to: a single room name
clusterName: unicorn
eventForwarder:
receivers:
- notifier: Stride
to:
- ops-alerts
notifierSecretName: notifier-config
Using Stride App
You can also use Stride App as client to send notification. This will allow you to send notifications to multiple rooms.
You need to do followings:
To receive chat notifications in stride using App, create a Secret with the following key:
| Name | Description |
|---|
| STRIDE_CLOUD_ID | Required You can get stride cloud_id from your organization’s URL |
| STRIDE_CLIENT_ID | Optional Stride app clientId |
| STRIDE_CLIENT_SECRET | Optional Stride app clientSecret |
$ echo -n 'your-stride-cloud-id' > STRIDE_CLOUD_ID
$ echo -n 'your-stride-app-client-id' > STRIDE_CLIENT_ID
$ echo -n 'your-stride-app-client-secret' > STRIDE_CLIENT_SECRET
$ kubectl create secret generic notifier-config -n demo \
--from-file=./STRIDE_CLOUD_ID \
--from-file=./STRIDE_CLIENT_ID \
--from-file=./STRIDE_CLIENT_SECRET
secret "notifier-config" created
apiVersion: v1
data:
STRIDE_CLOUD_ID: eW91ci1zdHJpZGUtY2xvdWQtaWQ=
STRIDE_CLIENT_ID: eW91ci1zdHJpZGUtYXBwLWNsaWVudC1pZA==
STRIDE_CLIENT_SECRET: eW91ci1zdHJpZGUtYXBwLWNsaWVudC1zZWNyZXQ=
kind: Secret
metadata:
creationTimestamp: 2017-07-25T01:54:37Z
name: notifier-config
namespace: demo
resourceVersion: "2244"
selfLink: /api/v1/namespaces/demo/secrets/notifier-config
uid: 372bc159-70dc-11e7-9b0b-080027503732
type: Opaque
Now, to receiver notifications via Stride App, configure receiver as below:
- notifier:
Stride - to: a list of room names where this app is installed
clusterName: unicorn
eventForwarder:
receivers:
- notifier: Stride
to:
- ops-alerts
- dev-alerts
notifierSecretName: notifier-config
Mailgun
To receive email notifications via Mailgun, create a Secret with the following keys:
| Name | Description |
|---|
| MAILGUN_DOMAIN | Required domain name for Mailgun account |
| MAILGUN_API_KEY | Required Mailgun API Key |
| MAILGUN_FROM | Required sender email address |
| MAILGUN_PUBLIC_API_KEY | Optional Mailgun public API Key |
$ echo -n 'your-mailgun-domain' > MAILGUN_DOMAIN
$ echo -n '[email protected]' > MAILGUN_FROM
$ echo -n 'your-mailgun-api-key' > MAILGUN_API_KEY
$ echo -n 'your-mailgun-public-api-key' > MAILGUN_PUBLIC_API_KEY
$ kubectl create secret generic notifier-config -n kube-system \
--from-file=./MAILGUN_DOMAIN \
--from-file=./MAILGUN_FROM \
--from-file=./MAILGUN_API_KEY \
--from-file=./MAILGUN_PUBLIC_API_KEY
secret "notifier-config" created
apiVersion: v1
data:
MAILGUN_API_KEY: eW91ci1tYWlsZ3VuLWFwaS1rZXk=
MAILGUN_DOMAIN: eW91ci1tYWlsZ3VuLWRvbWFpbg==
MAILGUN_FROM: bm8tcmVwbHlAZXhhbXBsZS5jb20=
MAILGUN_PUBLIC_API_KEY: bWFpbGd1bi1wdWJsaWMtYXBpLWtleQ==
kind: Secret
metadata:
creationTimestamp: 2017-07-25T01:31:24Z
name: notifier-config
namespace: kube-system
resourceVersion: "714"
selfLink: /api/v1/namespaces/kube-system/secrets/notifier-config
uid: f8e91037-70d8-11e7-9b0b-080027503732
type: Opaque
Now, to receive notifications via Mailgun, configure receiver as below:
- notifier:
Mailgun - to: a list of email addresses
clusterName: unicorn
eventForwarder:
receivers:
- notifier: Mailgun
to:
- [email protected]
notifierSecretName: notifier-config
SMTP
To configure any email provider, use SMTP notifier. To receive email notifications via SMTP services, create a Secret with the following keys:
| Name | Description |
|---|
| SMTP_HOST | Required Host address of smtp server |
| SMTP_PORT | Required Port of smtp server |
| SMTP_INSECURE_SKIP_VERIFY | Required If set to true, skips SSL verification |
| SMTP_USERNAME | Required Username |
| SMTP_PASSWORD | Required Password |
| SMTP_FROM | Required Sender email address |
$ echo -n 'your-smtp-host' > SMTP_HOST
$ echo -n 'your-smtp-port' > SMTP_PORT
$ echo -n 'your-smtp-insecure-skip-verify' > SMTP_INSECURE_SKIP_VERIFY
$ echo -n 'your-smtp-username' > SMTP_USERNAME
$ echo -n 'your-smtp-password' > SMTP_PASSWORD
$ echo -n 'your-smtp-from' > SMTP_FROM
$ kubectl create secret generic notifier-config -n kube-system \
--from-file=./SMTP_HOST \
--from-file=./SMTP_PORT \
--from-file=./SMTP_INSECURE_SKIP_VERIFY \
--from-file=./SMTP_USERNAME \
--from-file=./SMTP_PASSWORD \
--from-file=./SMTP_FROM
secret "notifier-config" created
To configure Kubed to send email notifications using a GMail account, set the Secrets like below:
$ echo -n 'smtp.gmail.com' > SMTP_HOST
$ echo -n '587' > SMTP_PORT
$ echo -n 'your-gmail-adress' > SMTP_USERNAME
$ echo -n 'your-gmail-password' > SMTP_PASSWORD
$ echo -n 'your-gmail-address' > SMTP_FROM
Now, to receive notifications via SMTP, configure receiver as below:
- notifier:
SMTP - to: a list of email addresses
clusterName: unicorn
eventForwarder:
receivers:
- notifier: SMTP
to:
- [email protected]
notifierSecretName: notifier-config
Twilio
To receive SMS notifications via Twilio, create a Secret with the following keys:
| Name | Description |
|---|
| TWILIO_ACCOUNT_SID | Required Twilio account SID |
| TWILIO_AUTH_TOKEN | Required Twilio authentication token |
| TWILIO_FROM | Required Sender mobile number |
$ echo -n 'your-twilio-account-sid' > TWILIO_ACCOUNT_SID
$ echo -n 'your-twilio-auth-token' > TWILIO_AUTH_TOKEN
$ echo -n 'your-twilio-from' > TWILIO_FROM
$ kubectl create secret generic notifier-config -n kube-system \
--from-file=./TWILIO_ACCOUNT_SID \
--from-file=./TWILIO_AUTH_TOKEN \
--from-file=./TWILIO_FROM
secret "notifier-config" created
apiVersion: v1
data:
TWILIO_ACCOUNT_SID: eW91ci10d2lsaW8tYWNjb3VudC1zaWQ=
TWILIO_AUTH_TOKEN: eW91ci10d2lsaW8tYXV0aC10b2tlbg==
TWILIO_FROM: eW91ci10d2lsaW8tZnJvbQ==
kind: Secret
metadata:
creationTimestamp: 2017-07-26T17:38:38Z
name: notifier-config
namespace: kube-system
resourceVersion: "27787"
selfLink: /api/v1/namespaces/kube-system/secrets/notifier-config
uid: 41f57a61-7229-11e7-af79-08002738e55e
type: Opaque
Now, to receive notifications via Twilio, configure receiver as below:
- notifier:
Twilio - to: a list of receiver mobile numbers
clusterName: unicorn
eventForwarder:
receivers:
- notifier: Twilio
to:
- +1-999-888-1234
notifierSecretName: notifier-config
Slack
To receive chat notifications in Slack, create a Secret with the following keys:
$ echo -n 'your-slack-auth-token' > SLACK_AUTH_TOKEN
$ kubectl create secret generic notifier-config -n kube-system \
--from-file=./SLACK_AUTH_TOKEN
secret "notifier-config" created
apiVersion: v1
data:
SLACK_AUTH_TOKEN: eW91ci1zbGFjay1hdXRoLXRva2Vu
kind: Secret
metadata:
creationTimestamp: 2017-07-25T01:58:58Z
name: notifier-config
namespace: kube-system
resourceVersion: "2534"
selfLink: /api/v1/namespaces/kube-system/secrets/notifier-config
uid: d2571817-70dc-11e7-9b0b-080027503732
type: Opaque
Now, to receive notifications via Slack, configure receiver as below:
- notifier:
Slack - to: a list of chat room names
clusterName: unicorn
eventForwarder:
receivers:
- notifier: Slack
to:
- '#ops-alerts'
notifierSecretName: notifier-config
Plivo
To receive SMS notifications via Plivo, create a Secret with the following keys:
| Name | Description |
|---|
| PLIVO_AUTH_ID | Required Plivo auth ID |
| PLIVO_AUTH_TOKEN | Required Plivo authentication token |
| PLIVO_FROM | Required Sender mobile number |
$ echo -n 'your-plivo-auth-id' > PLIVO_AUTH_ID
$ echo -n 'your-plivo-auth-token' > PLIVO_AUTH_TOKEN
$ echo -n 'your-plivo-from' > PLIVO_FROM
$ kubectl create secret generic notifier-config -n kube-system \
--from-file=./PLIVO_AUTH_ID \
--from-file=./PLIVO_AUTH_TOKEN \
--from-file=./PLIVO_FROM
secret "notifier-config" created
apiVersion: v1
data:
PLIVO_AUTH_ID: eW91ci1wbGl2by1hdXRoLWlk
PLIVO_AUTH_TOKEN: eW91ci1wbGl2by1hdXRoLXRva2Vu
PLIVO_FROM: eW91ci1wbGl2by1mcm9t
kind: Secret
metadata:
creationTimestamp: 2017-07-25T02:00:02Z
name: notifier-config
namespace: kube-system
resourceVersion: "2606"
selfLink: /api/v1/namespaces/kube-system/secrets/notifier-config
uid: f8dade1c-70dc-11e7-9b0b-080027503732
type: Opaque
Now, to receive notifications via Plivo, configure receiver as below:
- notifier:
Plivo - to: a list of receiver mobile numbers
clusterName: unicorn
eventForwarder:
receivers:
- notifier: Plivo
to:
- +1-999-888-1234
notifierSecretName: notifier-config
Pushover.net
To receive push notifications via Pushover.net, create a Secret with the following keys:
| Name | Description |
|---|
| PUSHOVER_TOKEN | Required Pushover.net token. |
| PUSHOVER_USER_KEY | Required User key or group key. |
| PUSHOVER_TITLE | Optional Message’s title, otherwise your app’s name is used. |
| PUSHOVER_URL | Optional A supplementary URL to show with your message. |
| PUSHOVER_URL_TITLE | Optional A title for the supplementary URL, otherwise just the URL is shown. |
| PUSHOVER_PRIORITY | Optional Send as -2 to generate no notification/alert, -1 to always send as a quiet notification, 1 to display as high-priority and bypass the user’s quiet hours, or 2 to also require confirmation from the user. |
| PUSHOVER_SOUND | Optional The name of one of the sounds supported by device clients to override the user’s default sound choice. |
$ echo -n 'your-pushover-token' > PUSHOVER_TOKEN
$ echo -n 'your-pushover-user-key' > PUSHOVER_USER_KEY
$ echo -n 'your-pushover-title' > PUSHOVER_TITLE
$ echo -n 'your-pushover-url' > PUSHOVER_URL
$ echo -n 'your-pushover-url-title' > PUSHOVER_URL_TITLE
$ echo -n 'your-pushover-priority' > PUSHOVER_PRIORITY
$ echo -n 'your-pushover-sound' > PUSHOVER_SOUND
$ kubectl create secret generic notifier-config -n kube-system \
--from-file=./PUSHOVER_TOKEN \
--from-file=./PUSHOVER_USER_KEY \
--from-file=./PUSHOVER_TITLE \
--from-file=./PUSHOVER_URL \
--from-file=./PUSHOVER_URL_TITLE \
--from-file=./PUSHOVER_PRIORITY \
--from-file=./PUSHOVER_SOUND
secret "notifier-config" created
apiVersion: v1
data:
PUSHOVER_PRIORITY: eW91ci1wdXNob3Zlci1wcmlvcml0eQ==
PUSHOVER_SOUND: eW91ci1wdXNob3Zlci1zb3VuZA==
PUSHOVER_TITLE: eW91ci1wdXNob3Zlci10aXRsZQ==
PUSHOVER_TOKEN: eW91ci1wdXNob3Zlci10b2tlbg==
PUSHOVER_URL: eW91ci1wdXNob3Zlci11cmw=
PUSHOVER_URL_TITLE: eW91ci1wdXNob3Zlci11cmwtdGl0bGU=
PUSHOVER_USER_KEY: eW91ci1wdXNob3Zlci11c2VyLWtleQ==
kind: Secret
metadata:
creationTimestamp: 2017-08-04T05:13:07Z
name: notifier-config
namespace: kube-system
resourceVersion: "33711872"
selfLink: /api/v1/namespaces/kube-system/secrets/notifier-config
uid: 99df75a8-78d3-11e7-acfa-42010af00141
type: Opaque
Now, to receive notifications via Pushover.net, configure receiver as below:
- notifier:
Pushover - to: a list of devices where notifications will be sent. If list is empty, all devices will be notified.
clusterName: unicorn
eventForwarder:
receivers:
- notifier: Pushover
to:
- my-phone
notifierSecretName: notifier-config
Telegram
To receive notifications in your Telegram app, please follow the steps below:
- Sign up for Telegram.
- Now, use @botfather to create your own Telegram Bot and get the authorization token.
- Create a Kubernetes Secret with the following keys:
| Name | Description |
|---|
| TELEGRAM_TOKEN | Required Telegram Bot authorization token. |
$ echo -n 'your-telegram-bot-token' > TELEGRAM_TOKEN
$ kubectl create secret generic notifier-config -n kube-system \
--from-file=./TELEGRAM_TOKEN
secret "notifier-config" created
apiVersion: v1
data:
TELEGRAM_TOKEN: NDkxoooooooooooooooooxJ
kind: Secret
metadata:
creationTimestamp: 2018-01-16T13:41:04Z
name: notifier-config
namespace: kube-system
resourceVersion: "768"
selfLink: /api/v1/namespaces/kube-system/secrets/notifier-config
uid: e6066076-fac2-11e7-b3e7-0800276ee39b
type: Opaque
- Add the Bot as an Admin for your channel where notifications will be sent.
Now, to receiver notifications in your public Telegram channels, configure receiver as below:
- notifier:
Telegram - to: a list of channels where notifications will be sent.
clusterName: unicorn
eventForwarder:
receivers:
- notifier: Telegram
to:
- '@my-channel'
notifierSecretName: notifier-config
To receive notifications if your private channel, follow the steps below:
- Create a new public channel or change your private channel to public temporarily and assign a link.
- Now, run the following command to determine the
id for your channel.
TOKEN=<token>
curl -X POST https://api.telegram.org/bot${TOKEN}/sendMessage -d "[email protected]&text=hello"
{"ok":true,"result":{"message_id":10,"chat":{"id":-1001210429328,"title":"mytest","username":"mytest123489","type":"channel"},"date":1516103121,"text":"hello"}}
- Make your channel private.
Now, to receiver notifications in your private Telegram channels, configure channle id as receiver like below:
clusterName: unicorn
eventForwarder:
receivers:
- notifier: Telegram
to:
- '-1001210429328'
notifierSecretName: notifier-config
Webhook Notifier
To receive chat notifications via Webhook, create a Secret with the following keys:
| Name | Description |
|---|
| WEBHOOK_URL | Required URL of webhook server where notification is sent |
| WEBHOOK_USERNAME | Optional Username for basic auth |
| WEBHOOK_PASSWORD | Optional Password for basic auth |
| WEBHOOK_TOKEN | Optional Token for bearer auth |
| WEBHOOK_CA_CERT_DATA | Optional PEM encoded CA certificate used by Webhook server |
| WEBHOOK_CLIENT_CERT_DATA | Optional PEM encoded client certificate used to authenticate to Webhook server |
| WEBHOOK_CLIENT_KEY_DATA | Optional PEM encoded client private key used to authenticate to Webhook server |
| WEBHOOK_INSECURE_SKIP_VERIFY | Optional If set to true, skips SSL verification of Webhook server certificate |
$ echo -n '' > WEBHOOK_URL
$ echo -n '' > WEBHOOK_TOKEN
$ echo -n '' > WEBHOOK_CA_CERT_DATA
$ kubectl create secret generic notifier-config -n kube-system \
--from-file=./WEBHOOK_URL \
--from-file=./WEBHOOK_TOKEN \
--from-file=./WEBHOOK_CA_CERT_DATA
secret "notifier-config" created
apiVersion: v1
data:
WEBHOOK_URL: eW91ci1oaXBjaGF0LWF1dGgtdG9rZW4=
WEBHOOK_TOKEN: eW91ci1oaXBjaGF0LWF1dGgtdG9rZW4=
WEBHOOK_CA_CERT_DATA: eW91ci1oaXBjaGF0LWF1dGgtdG9rZW4=
kind: Secret
metadata:
creationTimestamp: 2017-07-25T01:54:37Z
name: notifier-config
namespace: kube-system
resourceVersion: "2244"
selfLink: /api/v1/namespaces/kube-system/secrets/notifier-config
uid: 372bc159-70dc-11e7-9b0b-080027503732
type: Opaque
Now, to receiver notifications via Webhook, configure receiver as below:
- notifier:
Webhook - to: a list of receiver names
clusterName: unicorn
eventForwarder:
receivers:
- notifier: Webhook
to:
- ops-alerts
notifierSecretName: notifier-config
Using multiple notifiers
Kubed supports using different notifiers in different scenarios. First add the credentials for the different notifiers in the same Secret notifier-config and deploy that to Kubernetes. Then in the Kubed cluster config, specify the appropriate notifier for each feature.
apiVersion: v1
data:
MAILGUN_API_KEY: eW91ci1tYWlsZ3VuLWFwaS1rZXk=
MAILGUN_DOMAIN: eW91ci1tYWlsZ3VuLWRvbWFpbg==
MAILGUN_FROM: bm8tcmVwbHlAZXhhbXBsZS5jb20=
SLACK_AUTH_TOKEN: eW91ci1zbGFjay1hdXRoLXRva2Vu
kind: Secret
metadata:
creationTimestamp: 2017-07-25T01:58:58Z
name: notifier-config
namespace: kube-system
resourceVersion: "2534"
selfLink: /api/v1/namespaces/kube-system/secrets/notifier-config
uid: d2571817-70dc-11e7-9b0b-080027503732
type: Opaque
clusterName: unicorn
eventForwarder:
warningEvents:
handle: true
namespaces:
- kube-system
receivers:
- notifier: Mailgun
to:
- [email protected]
- notifier: Slack
to:
- '#ops-alerts'
notifierSecretName: notifier-config
Next Steps
- Want to keep an eye on your cluster with automated notifications? Setup Kubed event forwarder.
- Learn how to use Kubed to protect your Kubernetes cluster from disasters here.
- Need to keep configmaps/secrets synchronized across namespaces or clusters? Try Kubed config syncer.
- Out of disk space because of too much logs in Elasticsearch or metrics in InfluxDB? Configure janitors to delete old data.
- Wondering what features are coming next? Please visit here.
- Want to hack on Kubed? Check our contribution guidelines.
