Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Allow alert evaluation to only occur during certain times/days #141

Closed
lpalm opened this issue Jan 11, 2016 · 6 comments
Closed

Allow alert evaluation to only occur during certain times/days #141

lpalm opened this issue Jan 11, 2016 · 6 comments

Comments

@lpalm
Copy link

lpalm commented Jan 11, 2016

It would be really useful to have a way to prevent alerts/expressions from being evaluated during weekends or scheduled maintenance windows. Non-24/7 services typically will go down when not being used so we'd like to not have to receive emails about it.

@nathanielc
Copy link
Contributor

My first thought on how to approach this would be to create a time of day functions for use in filters. Something like:

stream.from()...
    // process data stream
   ...
    // filter data based on time features. Specifically limit data points to M-F 8am-5pm
   .where(lambda: hour("time") > 8 AND hour("time") < 17 AND day("time") >= 1 AND day("time") <= 5)
   .alert()
      //process alert.

Thoughts?

@lpalm
Copy link
Author

lpalm commented Jan 11, 2016

Yeah, that would certainly be general enough for our use case (though perhaps weekDay() would be clearer than day()?). A cron-like syntax (e.g. * 8-17 * * 1-5) would also be fine.

One thing to keep in mind is that eventually we'd like to alert for missing data during those periods, so we'd need to be careful that this would play nice with whatever implementation is used for #137

@gargooza
Copy link

+1 for a dedicated function to filter alerts to specific days/times. Something cron-like as per @lpalm 's suggestion would be awesome.

@nathanielc
Copy link
Contributor

@gargooza @lpalm Is this something you would like globally for all alerts within Kapacitor or would it be different per alert?

I could just add a quite-hours cron-like option to the config. Would that be enough or do you want it per alert?

@gargooza
Copy link

A filter would need to be configured specifically for each alert. For example, some of my alerts are no good on weekends. Some alerts are only during business hours. Some alerts shouldn't fire on holidays. Some alerts might be combinations of all of those.

@lpalm
Copy link
Author

lpalm commented Jan 22, 2016

Per alert would be better for us as well as maintenance windows vary depending on the task being monitored and their geographical location.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

3 participants