LikeLike

LikeLike

LikeLike

LikeLike

LikeLike

LikeLike

LikeLike

LikeLike

LikeLike

First, as to the adjustment based on test date, I don’t think you correct in assuming that the fall scores are uniformly lower than for the preceding spring. E.g., the 50th percentile math score for K in the spring is 159 and the 50th percentile for 1st in the fall is 163. There are also certainly examples where the RIT score decreases from spring to fall. It varies by subject, grade and percentile (and does not vary in the straightforward way I would have expected). Just eyeballing things I think it’s largely a wash and wouldn’t be the source of major changes. So I wouldn’t make much of it.

Second, depending on how the growth expectations were generated, it could be appropriate to make an adjustment. Using your “John” example, a spring score of 203 would generate a certain distribution for John’s growth. If John actually scored 200 from a fall test (in your hypothetical), then it would be wrong to treat that 200 as a spring score and generate a growth distribution from it. I don’t know that that’s how the growth distributions were generated but it’s plausible. I think you could do this fairly reliably as you have a mapping by grade and subject of percentiles from fall back to spring. And it would not have the bias in the direction you claim because you assumption that scores generally decline from spring to fall is incorrect. But as I said, the spring to fall changes are I think largely a wash (though I could be proven wrong with a more careful assessment), so this discussion is largely academic.

Third, I am at the moment agnostic on whether CPS did something improper and/or unreliable in adjusting for different versions of the test. If you had reliable nationally normed percentiles for both versions, it seems to me not unreasonable to map one onto the other. E.g., if you had a student scoring at the 50th percentile in reading on the IL state version, you could look up what the 50th percentile was for reading on the CC version (adjusting as needed for the spring-fall issue), then get a growth expectation from that score and compare to the students actual performance on the CC version the subsequent spring. I am not saying this rough approach is necessarily exactly right, but something along these lines should be considered. If the IL version is indeed “easier” as claimed in the comments, then it would be legitimate to think about an adjustment. There may be further considerations to be assessed but to dismiss an adjustment out of hand is not warranted.

I therefore think your conclusion that the scores should not have been adjusted (you use the more pejorative “altered”) is likely wrong or, at a minimum, premature. There can be legitimate reasons to adjust scores. I have suggested some above. I do agree with you that CPS has to date been lacking in transparency and I applaud your efforts to push them in this regard.

LikeLike