According to reports from Edge users, the icon appeared after updating Edge to version 121.0.2277.128, ... Now simply restart Edge, and the pesky new sidebar button will have disappeared.