- **Paper**: - **Type**: - **Opinion**: - **Reference site**: - **Contents** ## How does SimSiam avoid collapse? 1. SimSiam claim two components, stop gradient and predictor (which can be regarded as expectation over augmentations (EOA) = diverse augmented views from the same image의 기댓값). 2. Methods (1) Decompose a representation vector (2) an extra gradient component 1. Findings1) it center vector helps prevent collapse via the **de-centering** effect. 2. Findings2) its residual vector achieves dimensional **de-correlation** with also alleviates collapse. 3. Findings3) the extra gradient caused by **negative samples** in InfoNCE also achieves **de-centering and de-correlation** in the same manner. 4. Findings4) Towards **simplifying the predictor**, a single bias layer is sufficient enough for preventing collapse. 3. Revisiting SimSiam (abstract of Section 2) 1. The symmetric architectures with and without predictor both lead to collapse. So, how SimSiam avoids collapse lies in its **asymmetric** architecture. 2. We can not define the predictor as EOA. 4.
