In this chart, we take the PCA coordinates and color the participant locations by the number of total votes. Hopefully, it looks random. If it doesn't, we might imagine the following scenario:
- 1000 people vote, and there are very few controversial statements. They do not return.
- 1 person submits a statement which is incredibly controversial.
- 1000 more people vote, the space begins to take on structure, PCA is closely linked to vote count.
We know this scenario - that voters don't see controversial comments - happens. Polis mitigates in two ways:
- polis eliminates participants who don't vote at least 7 times from the analysis
- polis shows several highly controversial comments (large egeinvalue) in the first 10 comments participants see