Yes, as long as you get the wiring polarity correct. But isn't the accessory port switched? If it is, the ignition would have to be on to charge. That wouldn't be good.
Why not add a pig tail connected directly to the battery?
I have a short lead (pigtail) attached to the battery and extending out under the back of the front cover. When I take the buggy out I yank on the wire on the trickle charger to remove it. when the buggy returns to the garage I reconnect the charger. Always.