Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Changed recreate_db to skip non sqlite attributes #2168

Merged
merged 6 commits into from
Dec 4, 2024

Conversation

twentyninehairs
Copy link
Contributor

@twentyninehairs twentyninehairs commented Dec 2, 2024

Summary by CodeRabbit

  • New Features

    • Enhanced flexibility for handling multiple database types (MySQL and PostgreSQL) in the database management script.
  • Bug Fixes

    • Improved error handling for environment variables to ensure only specific values are accepted.
  • Chores

    • Updated cleanup tasks to correctly target the appropriate database type during operations.

Copy link
Contributor

coderabbitai bot commented Dec 2, 2024

📝 Walkthrough

Walkthrough

The recreate_db script has been significantly modified to enhance its logic for handling different database types. A new conditional check has been introduced to process the database URI only if the SPIFFWORKFLOW_BACKEND_DATABASE_TYPE is not set to "sqlite". This change allows for structured handling of MySQL and PostgreSQL databases while retaining SQLite functionality. Error handling for the SPIFFWORKFLOW_BACKEND_ENV variable remains unchanged, and cleanup tasks have been updated to target the correct database type during operations.

Changes

File Path Change Summary
spiffworkflow-backend/bin/recreate_db Introduced conditional check for database type; modified URI processing for MySQL and PostgreSQL; retained SQLite handling; updated cleanup tasks for database operations.

Sequence Diagram(s)

sequenceDiagram
    participant User
    participant Script
    participant Database

    User->>Script: Run recreate_db
    Script->>Script: Check SPIFFWORKFLOW_BACKEND_DATABASE_TYPE
    alt If not SQLite
        Script->>Database: Process database URI
        Database-->>Script: Connection established
        Script->>Database: Perform operations (create/drop)
    else If SQLite
        Script->>Database: Handle SQLite operations
    end
    Script-->>User: Complete database setup
Loading

Thank you for using CodeRabbit. We offer it for free to the OSS community and would appreciate your support in helping us grow. If you find it useful, would you consider giving us a shout-out on your favorite social media?

❤️ Share
🪧 Tips

Chat

There are 3 ways to chat with CodeRabbit:

  • Review comments: Directly reply to a review comment made by CodeRabbit. Example:
    • I pushed a fix in commit <commit_id>, please review it.
    • Generate unit testing code for this file.
    • Open a follow-up GitHub issue for this discussion.
  • Files and specific lines of code (under the "Files changed" tab): Tag @coderabbitai in a new review comment at the desired location with your query. Examples:
    • @coderabbitai generate unit testing code for this file.
    • @coderabbitai modularize this function.
  • PR comments: Tag @coderabbitai in a new PR comment to ask questions about the PR branch. For the best results, please provide a very specific query, as very limited context is provided in this mode. Examples:
    • @coderabbitai gather interesting stats about this repository and render them as a table. Additionally, render a pie chart showing the language distribution in the codebase.
    • @coderabbitai read src/utils.ts and generate unit testing code.
    • @coderabbitai read the files in the src/scheduler package and generate a class diagram using mermaid and a README in the markdown format.
    • @coderabbitai help me debug CodeRabbit configuration file.

Note: Be mindful of the bot's finite context window. It's strongly recommended to break down tasks such as reading entire modules into smaller chunks. For a focused discussion, use review comments to chat about specific files and their changes, instead of using the PR comments.

CodeRabbit Commands (Invoked using PR comments)

  • @coderabbitai pause to pause the reviews on a PR.
  • @coderabbitai resume to resume the paused reviews.
  • @coderabbitai review to trigger an incremental review. This is useful when automatic reviews are disabled for the repository.
  • @coderabbitai full review to do a full review from scratch and review all the files again.
  • @coderabbitai summary to regenerate the summary of the PR.
  • @coderabbitai resolve resolve all the CodeRabbit review comments.
  • @coderabbitai configuration to show the current CodeRabbit configuration for the repository.
  • @coderabbitai help to get help.

Other keywords and placeholders

  • Add @coderabbitai ignore anywhere in the PR description to prevent this PR from being reviewed.
  • Add @coderabbitai summary to generate the high-level summary at a specific location in the PR description.
  • Add @coderabbitai anywhere in the PR title to generate the title automatically.

Documentation and Community

  • Visit our Documentation for detailed information on how to use CodeRabbit.
  • Join our Discord Community to get help, request features, and share feedback.
  • Follow us on X/Twitter for updates and announcements.

Copy link
Contributor Author

@twentyninehairs twentyninehairs left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Script was crashing when using SQLite. Nested setting database variables irrelevant to SQLite so that script can continue running when using SQLite.

@twentyninehairs twentyninehairs marked this pull request as ready for review December 3, 2024 23:30
Copy link
Contributor

@coderabbitai coderabbitai bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 1

📜 Review details

Configuration used: .coderabbit.yaml
Review profile: CHILL

📥 Commits

Reviewing files that changed from the base of the PR and between ee3d545 and 0585fbd.

📒 Files selected for processing (1)
  • spiffworkflow-backend/bin/recreate_db (1 hunks)

Comment on lines +40 to +52
if [[ "${SPIFFWORKFLOW_BACKEND_DATABASE_TYPE:-mysql}" != "sqlite" ]]; then
if [[ -n "${SPIFFWORKFLOW_BACKEND_DATABASE_URI:-}" ]]; then
database_host_and_port=$(grep -oP "^[^:]+://.*@\K(.+?)[/]" <<<"$SPIFFWORKFLOW_BACKEND_DATABASE_URI" | sed -E 's/[\/]$//')
database_host=$(awk -F ':' '{print $1}' <<<"$database_host_and_port")
database_port=$(awk -F ':' '{print $2}' <<<"$database_host_and_port")
database_username_and_password=$(grep -oP "^[^:]+://\K([^@]+)[@]" <<<"$SPIFFWORKFLOW_BACKEND_DATABASE_URI" | sed -E 's/[@]$//')
database_username=$(awk -F ':' '{print $1}' <<<"$database_username_and_password")
database_password=$(awk -F ':' '{print $2}' <<<"$database_username_and_password")
database_name_from_uri=$(grep -oP "/\K(\w+)$" <<<"$SPIFFWORKFLOW_BACKEND_DATABASE_URI")
if ! grep "\<$database_name_from_uri\>" <<<"$databases_to_run_on"; then
databases_to_run_on="$database_name_from_uri"
fi
fi
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🛠️ Refactor suggestion

Improve Database URI Parsing for Robustness and Portability

The current implementation for parsing SPIFFWORKFLOW_BACKEND_DATABASE_URI uses multiple grep -oP, awk, and sed commands with complex regular expressions. This approach may:

  • Be fragile when handling URIs with special characters, different formats, or URL-encoded components.
  • Suffer from portability issues since the -P option (Perl-compatible regex) in grep is not supported on all systems by default.

Consider the following improvements:

  • Use a dedicated URI parsing tool or utility: Instead of manually parsing the URI, utilize a more robust method or tool designed for URI parsing. For instance, you can use Python's urllib.parse module within the script to reliably extract URI components.

  • Enhance portability: Replace grep -oP with POSIX-compliant tools or pure awk to ensure compatibility across different environments.

Example Refactor Using awk:

Here's a refactored version that uses awk for parsing without relying on non-standard options:

 if [[ "${SPIFFWORKFLOW_BACKEND_DATABASE_TYPE:-mysql}" != "sqlite" ]]; then
     if [[ -n "${SPIFFWORKFLOW_BACKEND_DATABASE_URI:-}" ]]; then
-      database_host_and_port=$(grep -oP "^[^:]+://.*@\K(.+?)[/]" <<<"$SPIFFWORKFLOW_BACKEND_DATABASE_URI" | sed -E 's/[\/]$//')
-      database_host=$(awk -F ':' '{print $1}' <<<"$database_host_and_port")
-      database_port=$(awk -F ':' '{print $2}' <<<"$database_host_and_port")
-      database_username_and_password=$(grep -oP "^[^:]+://\K([^@]+)[@]" <<<"$SPIFFWORKFLOW_BACKEND_DATABASE_URI" | sed -E 's/[@]$//')
-      database_username=$(awk -F ':' '{print $1}' <<<"$database_username_and_password")
-      database_password=$(awk -F ':' '{print $2}' <<<"$database_username_and_password")
-      database_name_from_uri=$(grep -oP "/\K(\w+)$" <<<"$SPIFFWORKFLOW_BACKEND_DATABASE_URI")
+      database_username_and_password=$(echo "$SPIFFWORKFLOW_BACKEND_DATABASE_URI" | awk -F'[@:]+' '{print $2}')
+      database_username=$(echo "$database_username_and_password" | awk -F':' '{print $1}')
+      database_password=$(echo "$database_username_and_password" | awk -F':' '{print $2}')
+      database_host_and_port=$(echo "$SPIFFWORKFLOW_BACKEND_DATABASE_URI" | awk -F'[@/]' '{print $2}')
+      database_host=$(echo "$database_host_and_port" | awk -F':' '{print $1}')
+      database_port=$(echo "$database_host_and_port" | awk -F':' '{print $2}')
+      database_name_from_uri=$(echo "$SPIFFWORKFLOW_BACKEND_DATABASE_URI" | awk -F'/' '{print $NF}')
       if ! grep -q -w "$database_name_from_uri" <<<"$databases_to_run_on"; then
         databases_to_run_on="$database_name_from_uri"
       fi
     fi
 fi

Benefits of This Refactor:

  • Portability: Eliminates the use of grep -P, enhancing compatibility with systems that have standard grep implementations.
  • Robustness: Reduces reliance on complex regular expressions, minimizing the risk of parsing errors with different URI formats.
  • Readability: Simplifies the parsing logic, making the script easier to maintain and understand.

Additional Recommendations:

  • Quote Variables Properly: Ensure all variable expansions are properly quoted to prevent word splitting or globbing issues, especially when dealing with data that may contain spaces or special characters.
  • Validate Parsed Components: Add validation checks after parsing to ensure that each component (username, password, host, port, database name) has been extracted correctly before proceeding.

Please test the refactored code thoroughly to confirm it handles all expected URI formats correctly.

📝 Committable suggestion

‼️ IMPORTANT
Carefully review the code before committing. Ensure that it accurately replaces the highlighted code, contains no missing lines, and has no issues with indentation. Thoroughly test & benchmark the code to ensure it meets the requirements.

Suggested change
if [[ "${SPIFFWORKFLOW_BACKEND_DATABASE_TYPE:-mysql}" != "sqlite" ]]; then
if [[ -n "${SPIFFWORKFLOW_BACKEND_DATABASE_URI:-}" ]]; then
database_host_and_port=$(grep -oP "^[^:]+://.*@\K(.+?)[/]" <<<"$SPIFFWORKFLOW_BACKEND_DATABASE_URI" | sed -E 's/[\/]$//')
database_host=$(awk -F ':' '{print $1}' <<<"$database_host_and_port")
database_port=$(awk -F ':' '{print $2}' <<<"$database_host_and_port")
database_username_and_password=$(grep -oP "^[^:]+://\K([^@]+)[@]" <<<"$SPIFFWORKFLOW_BACKEND_DATABASE_URI" | sed -E 's/[@]$//')
database_username=$(awk -F ':' '{print $1}' <<<"$database_username_and_password")
database_password=$(awk -F ':' '{print $2}' <<<"$database_username_and_password")
database_name_from_uri=$(grep -oP "/\K(\w+)$" <<<"$SPIFFWORKFLOW_BACKEND_DATABASE_URI")
if ! grep "\<$database_name_from_uri\>" <<<"$databases_to_run_on"; then
databases_to_run_on="$database_name_from_uri"
fi
fi
if [[ "${SPIFFWORKFLOW_BACKEND_DATABASE_TYPE:-mysql}" != "sqlite" ]]; then
if [[ -n "${SPIFFWORKFLOW_BACKEND_DATABASE_URI:-}" ]]; then
database_username_and_password=$(echo "$SPIFFWORKFLOW_BACKEND_DATABASE_URI" | awk -F'[@:]+' '{print $2}')
database_username=$(echo "$database_username_and_password" | awk -F':' '{print $1}')
database_password=$(echo "$database_username_and_password" | awk -F':' '{print $2}')
database_host_and_port=$(echo "$SPIFFWORKFLOW_BACKEND_DATABASE_URI" | awk -F'[@/]' '{print $2}')
database_host=$(echo "$database_host_and_port" | awk -F':' '{print $1}')
database_port=$(echo "$database_host_and_port" | awk -F':' '{print $2}')
database_name_from_uri=$(echo "$SPIFFWORKFLOW_BACKEND_DATABASE_URI" | awk -F'/' '{print $NF}')
if ! grep -q -w "$database_name_from_uri" <<<"$databases_to_run_on"; then
databases_to_run_on="$database_name_from_uri"
fi
fi
fi

@burnettk burnettk merged commit 7cd38a3 into sartography:main Dec 4, 2024
@burnettk
Copy link
Contributor

burnettk commented Dec 4, 2024

thank you, @twentyninehairs !

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants