Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Outlook Mail: map Outlook IDs to auto-incrementing numerical IDs #46

Merged
merged 5 commits into from
Sep 4, 2024

Conversation

g-linville
Copy link
Collaborator

@g-linville g-linville commented Sep 3, 2024

gpt-4o was getting confused by the long base64 IDs that Outlook uses. This change swaps all of those for auto-incrementing numerical IDs, which are much easier for the LLM to handle. We store the mapping in a SQLite DB stored in the workspace directory, and we create new entries on the fly as needed. The mapping is indexed on both fields so we can do reverse lookups performantly as well, to avoid unnecessarily creating the same entry in the mapping multiple times.

For now, this is implemented only in Mail, but I made the mapping stuff its own package so that we can easily include it in Calendar. Based on my testing, this seems to work very well, but I want a couple other people to try it before I bother to do it in the Calendar code.

go 1.23.0

require (
github.com/glebarez/sqlite v1.11.0
Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is the pure Go (no CGo) implementation of the SQLite driver for GORM.

Signed-off-by: Grant Linville <[email protected]>
@@ -2,8 +2,11 @@ module github.com/gptscript-ai/tools/apis/outlook/mail/code

go 1.23.0

replace github.com/gptscript-ai/tools/apis/outlook/common => github.com/g-linville/tools/apis/outlook/common v0.0.0-20240903203431-d7bfd26dc70d
Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is pointing at my fork for now. Unfortunately I can't really change this until after I merge this PR. Should I open up a second PR to introduce the common package by itself first?

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think it's fine to merge and immediately create another PR.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

any reason this common package has to be separate go module?

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I had separate Go modules for the manage and read tools, and want this code to be usable by both of them without one of them being the distinct "owner" of it

} else {
printers.PrintMessages(messages, false)
return printers.PrintMessages(messages, false)
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

nit: don't need the else block anymore.

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

fixed

@@ -74,3 +82,47 @@ func PrintMessage(msg models.Messageable, detailed bool) error {
func recipientableToString(r models.Recipientable) string {
return fmt.Sprintf("%s (%s)", util.Deref(r.GetEmailAddress().GetName()), util.Deref(r.GetEmailAddress().GetAddress()))
}

func fixMailFolder(folder *models.MailFolderable) error {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I would appreciate a more descriptive name for this, or a comment explaining exactly what is being fixed.

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yeah I should have named these better lol

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

fixed

return nil
}

func fixMessage(msg *models.Messageable) error {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Same comment here.

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

fixed

return fmt.Errorf("failed to set folder ID: %w", err)
}

(*folder).SetId(util.Ptr(newFolderID))
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hmm I think folder.SetId should just work?

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

No clue why, but it wasn't working, which is why I had to do it this way.

Copy link
Contributor

@StrongMonkey StrongMonkey left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM once donnie's comment has been addressed.

Signed-off-by: Grant Linville <[email protected]>
@g-linville g-linville requested a review from thedadams September 4, 2024 01:23
// replaceMailFolderIDs replaces the ID values of the mail folder itself and its parent
// with the corresponding numerical ID that we generate in the database.
// This is necessary to do prior to printing it for the LLM.
func replaceMailFolderIDs(folder *models.MailFolderable) error {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Daishan's comment below caused me to look into this: models.MailFolderable is an interface, so you're passing a pointer to an interface (which is why you have to dereference to use it).

Since you are "addressing" the variable yourself when you pass to these "replace IDs" functions, I would suggest not doing that and passing the variable itself (which is likely already a pointer because it's methods are mutating).

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Fixed

// with the corresponding numerical ID that we generate in the database.
// This is necessary to do prior to printing it for the LLM.
func replaceMailFolderIDs(folder *models.MailFolderable) error {
if folder == nil {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Because of the way Go's type system works, after making the changes suggested above, this nil check doesn't really make sense. It won't protect you from a nil-pointer dereference.

You may already know that, but just in case: https://go.dev/play/p/FGVP-LqqzpT

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Fixed

Signed-off-by: Grant Linville <[email protected]>
@g-linville g-linville requested a review from thedadams September 4, 2024 15:57
Signed-off-by: Grant Linville <[email protected]>
@g-linville g-linville merged commit 0ba4310 into obot-platform:main Sep 4, 2024
@g-linville g-linville deleted the mail-id branch September 4, 2024 16:17
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants