-
Notifications
You must be signed in to change notification settings - Fork 105
ingest pipeline docs "ingest lag" #1672
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
base: main
Are you sure you want to change the base?
Conversation
🔍 Preview links for changed docs: 🔔 The preview site may take up to 3 minutes to finish building. These links will become live once it completes. |
|
||
The basic calculation is: | ||
|
||
`event.ingested - @timestamp` |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Since @timestamp
is usually overrode to data source timestamp, my team usually recommends event.created
instead which is when it hits its first Elastic product (although I believe exception is Logstash pipelines don't qualify as pipeline in that doc and don't induce the field although Agent+Elasticsearch would.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Yes, but event.created
is not created automatically in Elastic Agent anymore, thus you need to add this to all the integrations manually. We can add a recommendation for it.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I rewrote it a bit and added the code needed to add the event.created
.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
First review pass.
@@ -95,6 +95,7 @@ toc: | |||
- file: ingest/upload-data-files.md | |||
- file: ingest/transform-enrich.md | |||
children: | |||
- file: ingest/transform-enrich/ingest-lag.md |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
You should edit https://www.elastic.co/docs/manage-data/ingest/transform-enrich to add the appropriate section for the new doc.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
No idea what do you mean by that to be honest. Can you take care of that for me?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
You'll see that in https://www.elastic.co/docs/manage-data/ingest/transform-enrich each of the subpages is introduced. Not mandatory, but might worth doing it for this one as well. A paragraph could suffice.
Co-authored-by: Fabrizio Ferri-Benedetti <[email protected]>
Co-authored-by: Fabrizio Ferri-Benedetti <[email protected]>
Co-authored-by: Fabrizio Ferri-Benedetti <[email protected]>
Co-authored-by: Fabrizio Ferri-Benedetti <[email protected]>
Co-authored-by: Fabrizio Ferri-Benedetti <[email protected]>
Co-authored-by: Fabrizio Ferri-Benedetti <[email protected]>
Co-authored-by: Fabrizio Ferri-Benedetti <[email protected]>
Co-authored-by: Fabrizio Ferri-Benedetti <[email protected]>
Co-authored-by: Fabrizio Ferri-Benedetti <[email protected]>
Co-authored-by: Fabrizio Ferri-Benedetti <[email protected]>
Co-authored-by: Fabrizio Ferri-Benedetti <[email protected]>
Co-authored-by: Fabrizio Ferri-Benedetti <[email protected]>
Co-authored-by: Fabrizio Ferri-Benedetti <[email protected]>
Co-authored-by: Fabrizio Ferri-Benedetti <[email protected]>
Co-authored-by: Fabrizio Ferri-Benedetti <[email protected]>
Co-authored-by: Fabrizio Ferri-Benedetti <[email protected]>
Co-authored-by: Fabrizio Ferri-Benedetti <[email protected]>
Co-authored-by: Fabrizio Ferri-Benedetti <[email protected]>
Co-authored-by: Fabrizio Ferri-Benedetti <[email protected]>
Co-authored-by: Fabrizio Ferri-Benedetti <[email protected]>
#1381 relates to this, we pulled this file out as it's standalone for better review