Adding the option to disable the DNS processor failure or success cache

[mjmbischoff]

Proposed commit message

Adds the option to disable the success and failure cache.

Motivation

This is to enable use cases that require capturing the current point in time dns record regardless of cache or ttl of the record. Such as the case of monitoring the dns server, or with recorded events that need to capture the current state of the environment. TTL captures the time frame over which the old value might be used over the current DNS record, in other words the frame time in which the agent might observe the old or new record based upon whenever the previous request was made. This unpredictability can be undesired when optimizing time-to-intervention.

Disabling the cache will have throughput implications, serial processing an event will be greater than DNS roundtrip time. For example if round-trip time to perform an DNS request is 1 ms, max throughput it limited to 1000/sec. Known use cases have are low throughput requirements. Parallelization, by for example deploying multiple agents, can be used to stretch this number. We would urge to reevaluate the use case and the use of the cache at this point.

NOTE: setting the ttl on the failure cache to 1ns achieves a similar, but imperfect effect.
NOTE: setting the ttl on the success cache is a valid option as per code, it is however ignored as also document in the code. in the documentation it is omitted as an option. Honoring setting and the ttl (min(ttl, dns_record_ttl)) is a different route. Similar to other dns client behaviour.

Checklist

• My code follows the style guidelines of this project
• I have commented my code, particularly in hard-to-understand areas
• I have made corresponding changes to the documentation
• I have made corresponding change to the default configuration files
• I have added tests that prove my fix is effective or that my feature works
• I have added an entry in CHANGELOG.next.asciidoc or CHANGELOG-developer.next.asciidoc.

Disruptive User Impact

non known, the default values leave the old behavior intact and the setting to trigger the new behavior is added in this PR

How to test this PR locally

Define the DNS processor, observe cache stats / resolver requests.

Related issues

• https://github.com/elastic/enhancements/issues/24988

[botelastic]

This pull request doesn't have a Team:<team> label.

[github-actions]

🤖 GitHub comments

Expand to view the GitHub comments

Just comment with:

• run docs-build : Re-trigger the docs validation. (use unformatted text in the comment!)

[mergify]

This pull request does not have a backport label.
If this is a bug or security fix, could you label this PR @mjmbischoff? 🙏.
For such, you'll need to label your PR with:

• The upcoming major version of the Elastic Stack
• The upcoming minor version of the Elastic Stack (if you're not pushing a breaking change)

To fixup this pull request, you need to add the backport labels for the needed
branches, such as:

• backport-8./d is the label to automatically backport to the 8./d branch. /d is the digit
• backport-active-all is the label that automatically backports to all active branches.
• backport-active-8 is the label that automatically backports to all active minor branches for the 8 major.
• backport-active-9 is the label that automatically backports to all active minor branches for the 9 major.

[andrewkroh]

Can you please add to the proposed commit message to explain the why part and what is the use case for cache disablement.

beats/.github/PULL_REQUEST_TEMPLATE.md

Line 19 in c203b82

- WHY: the rationale/motivation for the changes

Turning off the caching will significantly limit the throughput of the pipeline. Even if each request takes 1ms to complete, that means the maximum throughput is 1000 EPS.

Also, the documentation for the processor will need updated to include the new configuration parameter.

[mjmbischoff]

Can you please add to the proposed commit message to explain the why part and what is the use case for cache disablement.

beats/.github/PULL_REQUEST_TEMPLATE.md

Line 19 in c203b82

- WHY: the rationale/motivation for the changes

Turning off the caching will significantly limit the throughput of the pipeline. Even if each request takes 1ms to complete, that means the maximum throughput is 1000 EPS.

Also, the documentation for the processor will need updated to include the new configuration parameter.

Added motivation.

TODO: documentation Added documentation c5de66a ab103b2 e1f60c9