feat: enhances S3Download to filter by traced table names (backport #1374) #1380
Conversation
![mergify[bot]](https://avatars.githubusercontent.com/in/10562?s=60&v=4){kind=small}
This PR updates the `S3Download` implementation to allow specifying the names of the traced tables when downloading from an S3 bucket. This enhancement enables file filtering and reduces the amount of data downloaded. It is a desirable feature when interacting with the traced data of large network tests. If no table name is provided, then `S3Download` downloads all the traced tables. (cherry picked from commit 800924f)
mergify
bot
requested review from
rootulp and
staheri14
and removed request for
a team
| func S3Download(dst, prefix string, cfg S3Config) error { | ||
| // fileNames is a list of traced jsonl file names to download. If it is empty, all traces are downloaded. | ||
| // fileNames should not have .jsonl suffix. | ||
| func S3Download(dst, prefix string, fileNames []string, cfg S3Config) error { |
There was a problem hiding this comment.
this is technically a Go API breaking change and pkg/trace is public. Are you sure this should be backported as is? @staheri14
There was a problem hiding this comment.
That is a good point, I'll defer it to @evan-forbes, do you think we should avoid merging this change? I can also re-implement it in a backward compatible way i.e., by introducing two different S3Download methods, with and without fileNames.
There was a problem hiding this comment.
nice let's do all things backwards compatible if we can out of good habit. If there significant cost to doing them backwards compatible then we can reevaluate.
ofc, we just need to add a deprecated note to this function
|
We can close this PR, I will make another PR which introduces the same feature in a backward compatible fashion. |
This PR updates the
S3Downloadimplementation to allow specifying the names of the traced tables when downloading from an S3 bucket. This enhancement enables file filtering and reduces the amount of data downloaded. It is a desirable feature when interacting with the traced data of large network tests. If no table name is provided, thenS3Downloaddownloads all the traced tables.This is an automatic backport of pull request #1374 done by Mergify.