-
Notifications
You must be signed in to change notification settings - Fork 2.1k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
feat: csv to document row level conversion #8916
base: main
Are you sure you want to change the base?
feat: csv to document row level conversion #8916
Conversation
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
@mdrazak2001 Thanks for the contribution. I have requested some small changes, otherwise the PR looks good.
Pull Request Test Coverage Report for Build 13658995133Details
💛 - Coveralls |
@mdrazak2001 We discussed this feature internally with the team and decided to move it to For the linked issue, we plan to implement it in a way that provides a conversion feature:
I'll update this PR to move the feature to |
Related Issues
Proposed Changes:
Enhance the CSVToDocument component to support row-level conversion.
- Adds a 'split_by_row' parameter to convert each row of a CSV file into a separate Haystack Document.
- Retains the header row (field names) as the first line of the 'content' in each row-level Document.
How did you test it?
added unit test to existing test_csv_todocument.py
Notes for the reviewer
Checklist
fix:
,feat:
,build:
,chore:
,ci:
,docs:
,style:
,refactor:
,perf:
,test:
and added!
in case the PR includes breaking changes.