sum_to_zero_vector case study #229
Conversation
|
hi @spinkney and @WardBrian - not sure if we need reviews to add case studies, but I would appreciate any feedback you might have, if you have time. |
|
|
||
|
|
There was a problem hiding this comment.
I found it hard to understand what all the code drop downs are. Can you add a sentence saying what this is. For example, Code to load libraries and setup environment.
| >A sum to zero vector is exactly what the name suggests. A vector where the sum of the elements equals 0. | ||
| If you put a normal prior on the zero-sum vector the resulting variance will be less than the intended normal variance. | ||
| To get the same variance as the intended normal prior do | ||
|
|
There was a problem hiding this comment.
Please add the discourse username for quotes
| and the hard and soft sum-to-zero implementations. | ||
| We fit each model to the same dataset, using the same random seed, and then | ||
| compare the summary statistics for the constrained parameter values. | ||
| Since the models are equivalent, we expect that all three implementations |
There was a problem hiding this comment.
The models are nearly equivalent. The hard sum-to-zero, the soft sum-to-zero, and the built-in all have different implied prior distributions that are being placed on the vector.
|
|
||
| * The specified test sensitivity and specificity | ||
|
|
||
| In order to fit this model, we need to put a sum-to-zero constraint on the categorical variables. |
There was a problem hiding this comment.
Not true, we could drop a category or do some other type of contrast coding. I suggest rewriting to "We find the sum-to-zero constraint on teh categorical variables to be preferable to dropping a category for reference or other contrast coding strategy because it let's us model all the categories as offset from the mean.
|
|
||
|
|
||
| ##### Instantiate the data generating model. | ||
|
|
There was a problem hiding this comment.
Add a sentence here about what this is
| #### Model 1: `sum_to_zero_vector` | ||
|
|
||
| This model is in file [binomial_4_preds_ozs.stan](https://github.com/stan-dev/example-models/tree/master/jupyter/sum-to-zero/stan/binomial_4_preds_ozs.stan). | ||
|
|
There was a problem hiding this comment.
I suggest showing at least the parameters block for the sum_to_zero_vector. For the below python code, I suggest putting all the following code in one block.
| ``` | ||
|
|
||
| #### Model 2: Hard sum-to-zero constraint | ||
|
|
There was a problem hiding this comment.
Highlight what has changed from the previous Stan code. I suggest putting all the python into one block. This goes for the following section as well.
| ``` | ||
|
|
||
| #### Runtime performance | ||
|
|
There was a problem hiding this comment.
Why even have this section if there isn't a table to show?
There was a problem hiding this comment.
I suggest merging this with the below and changing the title to something like: Model Checking, Comparison, and Efficiency
| ``` | ||
|
|
||
| **Eth** | ||
|
|
There was a problem hiding this comment.
What is this? What does all the code do? Don't assume your readers know python that well!
| display_side_by_side(small_html, large_html) | ||
| ``` | ||
|
|
||
| All models have R-hat values of 1.00 for all group-level parameters and high effective sample sizes. |
There was a problem hiding this comment.
I highly suggest to put all the results into a table so the reader can see the differences across all the models without having to read the code output. I don't think you even need to show this code. Just a table and talk about the differences.
Transferring the contents of github repo: https://github.com/mitzimorris/sum_to_zero_vector to this repo.
This case study introduces the
sum_to_zero_vector. It demonstrates a simple workflow for evaluating performance of different ways to impose a sum-to-zero constraint on a parameter vector.The HTML file is self-contained. To re-render the HTML, this requires the
stan-dev/quarto-configrepo for the Stan website styling.