My rear panel is off now because it's summer and I switched to a half windshield. My soft top is what I really decide to remove or not. At 70+ it's inflated like a balloon. It only takes two minutes to remove so I usually do. If @HUCK and @JTW say the rear panel holds up, I trust them and I'd leave it on.