Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Fix progress managers episode 2 attack of the clones #156

Open
wants to merge 26 commits into
base: dev
Choose a base branch
from

Conversation

mdr223
Copy link
Collaborator

@mdr223 mdr223 commented Feb 26, 2025

Summary of PR

optimizer-demo.py

  • reflect changes to processing strategy names and addition of sentinel execution strategy

dataclasses.py

  • improve ExecutionStats by more explicitly declaring sentinel_plan_stats (rather than grouping plan stats and sentinel plan stats together in the same dict)

filters.py

  • bugfix

records.py

  • add field_to_score_fn to DataRecordSet to make it easier to access for target records in sentinel execution

execution_strategy.py and execution_strategy_type.py

  • move ExecutionStrategyType to its own file
  • also created SentinelExecutionStrategyType
  • created base class for ExecutionStrategy
  • pulled shared execution logic (between MAB and Random) into SentinelExecutionStrategy
  • modified / simplified some logic within SentinelExecutionStrategy for scoring computed records
    • I will debug this logic as part of my research work, so don't feel required to comb through the logic of e.g. score_quality and compute_quality

mab_execution_strategy.py and random_sampling_execution_strategy.py

  • most of the (algo specific) code that was previously in mab_sentinel_processor.py and random_sampling_sentinel_processor.py, but refactored a bit
  • also added progress manager logic

parallel_execution_strategy.py and single_threaded_execution_strategy.py

  • pushed execution logic into _execute_plan() method so that progress manager could be stopped no matter what if there was an exception thrown during execution

cost_model.py

  • simplified SampleBasedCostModel to process a SentinelPlanStats object (rather than some bespoke dictionary)

optimizer.py

  • update name to enable constructor to work with **config.to_dict() in query processor factory

optimizer_strategy.py and optimizer_strategy_type.py

  • put types in their own file and simplified

tasks.py

  • make use of new enum methods on OptimizerStrategyType

config.py

  • modify names and types of *_type variables
  • remove dead config params which are no longer used

mab_sentinel_processor.py and random_sampling_sentinel_processor.py

  • 💀

nosentinel_processor.py

  • remove double inheritance classes now that QueryProcessor takes an execution_strategy

query_processor.py

  • pass in kwargs directly (instead of passing in config object)
  • add execution_strategy and sentinel_execution_strategy

query_processor_factory.py

  • made config normalization logic a bit more robust
  • removed lookup mapping for processors
  • instead, we construct the optimizer, execution_strategy, and sentinel_execution_strategy and feed them into the processor constructor

sentinel_processor.py

  • pulled up shared logic from (formerly) MABSentinelProcessor and RandomSamplingSentinelProcessor

progress.py

  • support live display for sentinel plan progress monitoring

@mdr223 mdr223 requested a review from sivaprasadsudhir March 6, 2025 02:10
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

1 participant